2024-09-17T07:17:30.518 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T07:17:30.534 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T07:17:30.613 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dis-2024-09-17_07:04:29-rbd-wip-dis-testing-distro-default-smithi/7908707 branch: wip-dis-testing description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writethrough} email: idryomov@gmail.com first_in_suite: false job_id: '7908707' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: dis-2024-09-17_07:04:29-rbd-wip-dis-testing-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-dis-testing ceph: conf: client: rbd cache: true rbd cache max dirty: 0 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-dis-testing sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 owner: scheduled_dis@teuthology priority: 70 repo: https://github.com/ceph/ceph-ci roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 2961 sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 sleep_before_teardown: 0 subset: 66/100 suite: rbd suite_branch: wip-dis-testing suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_6e8474a02f2b86353394076ab80c6f50ea28e361/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci suite_sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE8zmjdF4PdtILv+2zCeT/tUVrwhzBkkt3E2R4lNgccEXDhBLdxbyGqOroGqwXuVhymK6WH4UAzDcBbdHEel4is= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDk2D3YUPVg1wW2pfKgbPzFSIrhcVYe4tc7gPcjnfvUD2eOhWXII+Q6ZenBr8LnSilr7vqSxijAInFQe2xfgrOQ= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_07:04:29 tube: smithi user: dis verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T07:17:30.613 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_6e8474a02f2b86353394076ab80c6f50ea28e361/qa; will attempt to use it 2024-09-17T07:17:30.614 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_6e8474a02f2b86353394076ab80c6f50ea28e361/qa/tasks 2024-09-17T07:17:30.614 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T07:17:30.616 INFO:teuthology.task.internal:Checking packages... 2024-09-17T07:17:30.637 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6e8474a02f2b86353394076ab80c6f50ea28e361' 2024-09-17T07:17:30.637 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T07:17:30.637 INFO:teuthology.packaging:ref: None 2024-09-17T07:17:30.637 INFO:teuthology.packaging:tag: None 2024-09-17T07:17:30.638 INFO:teuthology.packaging:branch: wip-dis-testing 2024-09-17T07:17:30.638 INFO:teuthology.packaging:sha1: 6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-17T07:17:30.638 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-dis-testing 2024-09-17T07:17:30.786 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4883.g6e8474a0 2024-09-17T07:17:30.788 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T07:17:30.794 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T07:17:30.794 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T07:17:30.817 INFO:teuthology.task.internal:Saving configuration 2024-09-17T07:17:30.830 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T07:17:30.836 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T07:17:30.857 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2024-09-17_07:04:29-rbd-wip-dis-testing-distro-default-smithi/7908707', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 07:12:26.870237', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE8zmjdF4PdtILv+2zCeT/tUVrwhzBkkt3E2R4lNgccEXDhBLdxbyGqOroGqwXuVhymK6WH4UAzDcBbdHEel4is='} 2024-09-17T07:17:30.877 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2024-09-17_07:04:29-rbd-wip-dis-testing-distro-default-smithi/7908707', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 07:12:26.868356', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDk2D3YUPVg1wW2pfKgbPzFSIrhcVYe4tc7gPcjnfvUD2eOhWXII+Q6ZenBr8LnSilr7vqSxijAInFQe2xfgrOQ='} 2024-09-17T07:17:30.877 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T07:17:30.883 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-17T07:17:30.883 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-17T07:17:30.883 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T07:17:30.934 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe1bfcd7ac0>, signals=[15]) 2024-09-17T07:17:30.934 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T07:17:30.939 INFO:teuthology.task.internal:Opening connections... 2024-09-17T07:17:30.939 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-09-17T07:17:30.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:17:31.037 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-09-17T07:17:31.037 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:17:31.117 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T07:17:31.123 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-09-17T07:17:31.141 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-09-17T07:17:31.142 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-09-17T07:17:31.199 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-09-17T07:17:31.199 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-09-17T07:17:31.199 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-09-17T07:17:31.199 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-09-17T07:17:31.199 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-09-17T07:17:31.199 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-09-17T07:17:31.199 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T07:17:31.199 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-09-17T07:17:31.199 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-09-17T07:17:31.199 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T07:17:31.199 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-09-17T07:17:31.200 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T07:17:31.200 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T07:17:31.200 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T07:17:31.200 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-09-17T07:17:31.231 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-09-17T07:17:31.250 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-09-17T07:17:31.250 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-09-17T07:17:31.308 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-09-17T07:17:31.308 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2024-09-17T07:17:31.308 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-09-17T07:17:31.308 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-09-17T07:17:31.308 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2024-09-17T07:17:31.308 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2024-09-17T07:17:31.308 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T07:17:31.308 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-09-17T07:17:31.309 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2024-09-17T07:17:31.309 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T07:17:31.309 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-09-17T07:17:31.309 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T07:17:31.309 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T07:17:31.309 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T07:17:31.309 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-09-17T07:17:31.339 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T07:17:31.348 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T07:17:31.353 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T07:17:31.353 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-09-17T07:17:31.359 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-09-17T07:17:31.377 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T07:17:31.382 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T07:17:31.382 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-09-17T07:17:31.417 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-09-17T07:17:31.448 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T07:17:31.496 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T07:17:31.509 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T07:17:31.509 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T07:17:31.510 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T07:17:31.510 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T07:17:31.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T07:17:31.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T07:17:31.528 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-09-17T07:17:31.528 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-09-17T07:17:31.585 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-503.el9.x86_64 2024-09-17T07:17:31.585 INFO:teuthology.task.kernel:Running kernel on smithi045: 5.14.0-503.el9.x86_64 2024-09-17T07:17:31.585 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2024-09-17T07:17:31.586 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-503.el9.x86_64 2024-09-17T07:17:31.586 INFO:teuthology.task.kernel:Running kernel on smithi192: 5.14.0-503.el9.x86_64 2024-09-17T07:17:31.586 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum install -y kernel 2024-09-17T07:17:33.844 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.3 MB 00:00 2024-09-17T07:17:34.153 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.3 MB 00:00 2024-09-17T07:17:35.971 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-17T07:17:37.336 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 8.6 MB/s | 20 MB 00:02 2024-09-17T07:17:40.222 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-17T07:17:41.762 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-17T07:17:41.839 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 19 kB 00:00 2024-09-17T07:17:42.636 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-17T07:17:43.279 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 110 kB/s | 19 kB 00:00 2024-09-17T07:17:44.158 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-17T07:17:48.058 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-17T07:17:49.814 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-09-17T07:17:49.965 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T07:17:49.965 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T07:17:50.055 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:17:50.065 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:17:50.065 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-09-17T07:17:50.065 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:17:50.065 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:17:50.065 INFO:teuthology.orchestra.run.smithi192.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T07:17:50.066 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-17T07:17:50.066 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T07:17:50.066 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T07:17:50.066 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T07:17:50.066 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:17:50.066 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:17:50.066 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:17:50.066 INFO:teuthology.orchestra.run.smithi192.stdout:Install 4 Packages 2024-09-17T07:17:50.066 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:17:50.067 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 79 M 2024-09-17T07:17:50.067 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 126 M 2024-09-17T07:17:50.067 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:17:50.493 INFO:teuthology.orchestra.run.smithi192.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 122 kB/s | 36 kB 00:00 2024-09-17T07:17:51.779 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T07:17:51.779 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T07:17:51.871 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:17:51.875 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:17:51.875 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-09-17T07:17:51.875 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:17:51.875 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:17:51.875 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T07:17:51.875 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-17T07:17:51.875 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T07:17:51.875 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T07:17:51.876 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T07:17:51.876 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:17:51.876 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:17:51.876 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:17:51.876 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2024-09-17T07:17:51.876 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:17:51.876 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 79 M 2024-09-17T07:17:51.877 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 126 M 2024-09-17T07:17:51.877 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:17:52.232 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 472 kB/s | 36 kB 00:00 2024-09-17T07:17:56.007 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 4.1 MB/s | 16 MB 00:03 2024-09-17T07:17:59.507 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 3.9 MB/s | 29 MB 00:07 2024-09-17T07:18:01.198 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 3.8 MB/s | 35 MB 00:09 2024-09-17T07:18:01.199 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:18:01.199 INFO:teuthology.orchestra.run.smithi045.stdout:Total 8.5 MB/s | 79 MB 00:09 2024-09-17T07:18:01.914 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:18:02.322 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:18:02.323 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:18:03.299 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:18:03.300 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:18:05.866 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:18:06.546 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T07:18:06.625 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T07:18:08.033 INFO:teuthology.orchestra.run.smithi192.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 899 kB/s | 16 MB 00:17 2024-09-17T07:18:09.387 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T07:18:09.511 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T07:18:14.637 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T07:18:14.678 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:18:19.820 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:18:23.006 INFO:teuthology.orchestra.run.smithi192.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 900 kB/s | 29 MB 00:32 2024-09-17T07:18:28.148 INFO:teuthology.orchestra.run.smithi192.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 932 kB/s | 35 MB 00:37 2024-09-17T07:18:28.148 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:18:28.148 INFO:teuthology.orchestra.run.smithi192.stdout:Total 2.1 MB/s | 79 MB 00:38 2024-09-17T07:18:28.831 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:18:29.168 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:18:29.168 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:18:30.140 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:18:30.141 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:18:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:18:33.377 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T07:18:33.466 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T07:18:36.195 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T07:18:36.337 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T07:18:41.157 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T07:18:41.271 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:18:46.098 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:18:59.107 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:18:59.148 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:18:59.497 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:18:59.498 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T07:18:59.498 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T07:18:59.498 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T07:19:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:19:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:19:00.363 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:19:00.363 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T07:19:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T07:19:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T07:19:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T07:19:00.364 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:19:00.364 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:19:00.695 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T07:19:00.792 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T07:19:00.792 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T07:19:00.792 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T07:19:00.792 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T07:19:00.792 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T07:19:00.792 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T07:19:00.793 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T07:19:00.793 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T07:19:00.851 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T07:19:00.851 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T07:19:00.851 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2024-09-17T07:19:00.883 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2024-09-17T07:19:00.883 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2024-09-17T07:19:00.883 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.06 2024-09-17T07:19:00.883 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 82.el9 2024-09-17T07:19:00.883 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2024-09-17T07:19:00.883 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T07:19:00.883 INFO:teuthology.orchestra.run.smithi045.stdout:Group : Unspecified 2024-09-17T07:19:00.884 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 8274670 2024-09-17T07:19:00.884 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2024-09-17T07:19:00.884 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T07:19:00.884 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T07:19:00.884 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T07:19:00.884 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T07:19:00.884 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : builder@centos.org 2024-09-17T07:19:00.884 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2024-09-17T07:19:00.884 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T07:19:00.884 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2024-09-17T07:19:00.884 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2024-09-17T07:19:00.884 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:19:00.885 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T07:19:00.885 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T07:19:00.885 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T07:19:00.885 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2024-09-17T07:19:00.885 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:19:00.885 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T07:19:00.886 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T07:19:00.886 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 5.14.0-508.el9.x86_64 2024-09-17T07:19:00.886 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T07:19:01.533 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2024-09-17T07:19:03.019 INFO:teuthology.orchestra.run.smithi045.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T07:19:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:done 2024-09-17T07:19:03.042 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-17T07:19:03.057 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.AOgQw27a9C 2024-09-17T07:19:03.058 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AOgQw27a9C 2024-09-17T07:19:03.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.AOgQw27a9C 2024-09-17T07:19:03.272 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.AOgQw27a9C is 6KB 2024-09-17T07:19:03.320 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.AOgQw27a9C 2024-09-17T07:19:03.335 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T07:19:03.404 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T07:19:03.404 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T07:19:03.404 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T07:19:03.404 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T07:19:03.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T07:19:03.563 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2024-09-17T07:19:24.505 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:19:24.545 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:19:24.927 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:19:24.927 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T07:19:24.928 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T07:19:24.928 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T07:19:25.790 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T07:19:25.791 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:19:25.791 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:19:25.791 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T07:19:25.791 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T07:19:25.791 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T07:19:25.791 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T07:19:25.791 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:19:25.791 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:19:26.125 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T07:19:26.245 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T07:19:26.245 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T07:19:26.245 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T07:19:26.246 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T07:19:26.246 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T07:19:26.246 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T07:19:26.246 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi192.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T07:19:26.246 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T07:19:26.304 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T07:19:26.305 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T07:19:26.305 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -qi grub2-tools 2024-09-17T07:19:26.337 INFO:teuthology.orchestra.run.smithi192.stdout:Name : grub2-tools 2024-09-17T07:19:26.337 INFO:teuthology.orchestra.run.smithi192.stdout:Epoch : 1 2024-09-17T07:19:26.338 INFO:teuthology.orchestra.run.smithi192.stdout:Version : 2.06 2024-09-17T07:19:26.338 INFO:teuthology.orchestra.run.smithi192.stdout:Release : 82.el9 2024-09-17T07:19:26.338 INFO:teuthology.orchestra.run.smithi192.stdout:Architecture: x86_64 2024-09-17T07:19:26.338 INFO:teuthology.orchestra.run.smithi192.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T07:19:26.338 INFO:teuthology.orchestra.run.smithi192.stdout:Group : Unspecified 2024-09-17T07:19:26.338 INFO:teuthology.orchestra.run.smithi192.stdout:Size : 8274670 2024-09-17T07:19:26.338 INFO:teuthology.orchestra.run.smithi192.stdout:License : GPLv3+ 2024-09-17T07:19:26.338 INFO:teuthology.orchestra.run.smithi192.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T07:19:26.338 INFO:teuthology.orchestra.run.smithi192.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T07:19:26.339 INFO:teuthology.orchestra.run.smithi192.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T07:19:26.339 INFO:teuthology.orchestra.run.smithi192.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T07:19:26.339 INFO:teuthology.orchestra.run.smithi192.stdout:Packager : builder@centos.org 2024-09-17T07:19:26.339 INFO:teuthology.orchestra.run.smithi192.stdout:Vendor : CentOS 2024-09-17T07:19:26.339 INFO:teuthology.orchestra.run.smithi192.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T07:19:26.339 INFO:teuthology.orchestra.run.smithi192.stdout:Summary : Support tools for GRUB. 2024-09-17T07:19:26.339 INFO:teuthology.orchestra.run.smithi192.stdout:Description : 2024-09-17T07:19:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:19:26.340 INFO:teuthology.orchestra.run.smithi192.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T07:19:26.340 INFO:teuthology.orchestra.run.smithi192.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T07:19:26.340 INFO:teuthology.orchestra.run.smithi192.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T07:19:26.340 INFO:teuthology.orchestra.run.smithi192.stdout:hardware devices. 2024-09-17T07:19:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:19:26.340 INFO:teuthology.orchestra.run.smithi192.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T07:19:26.341 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T07:19:26.341 INFO:teuthology.task.kernel:Updating grub on smithi192 to boot 5.14.0-508.el9.x86_64 2024-09-17T07:19:26.341 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T07:19:27.047 INFO:teuthology.orchestra.run.smithi192.stderr:Generating grub configuration file ... 2024-09-17T07:19:28.349 INFO:teuthology.orchestra.run.smithi192.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T07:19:28.381 INFO:teuthology.orchestra.run.smithi192.stderr:done 2024-09-17T07:19:28.384 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-09-17T07:19:28.400 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.NwHy9XsIH1 2024-09-17T07:19:28.400 DEBUG:teuthology.orchestra.run.smithi192:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.NwHy9XsIH1 2024-09-17T07:19:28.467 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.NwHy9XsIH1 2024-09-17T07:19:28.608 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.NwHy9XsIH1 is 6KB 2024-09-17T07:19:28.655 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.NwHy9XsIH1 2024-09-17T07:19:28.671 DEBUG:teuthology.orchestra.run.smithi192:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T07:19:28.738 INFO:teuthology.orchestra.run.smithi192.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T07:19:28.738 INFO:teuthology.orchestra.run.smithi192.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T07:19:28.738 INFO:teuthology.orchestra.run.smithi192.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T07:19:28.738 INFO:teuthology.orchestra.run.smithi192.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T07:19:28.739 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T07:19:28.908 DEBUG:teuthology.orchestra.run.smithi192:> sudo shutdown -r now 2024-09-17T07:19:33.569 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T07:19:33.569 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-17T07:19:33.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:19:58.937 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T07:19:58.937 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-17T07:19:58.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:20:33.601 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2024-09-17T07:20:33.602 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T07:20:42.612 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-17T07:20:42.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:20:42.615 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-17T07:20:42.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:20:42.957 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-09-17T07:20:43.378 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-17T07:20:43.378 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T07:20:43.378 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-09-17T07:20:43.394 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-508.el9.x86_64 2024-09-17T07:20:43.395 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T07:20:43.395 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T07:20:43.395 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T07:20:44.395 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T07:20:44.396 DEBUG:teuthology.orchestra.run.smithi045:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T07:20:44.487 INFO:teuthology.orchestra.run.smithi045.stdout:ttyS1 2024-09-17T07:20:44.510 DEBUG:teuthology.parallel:result is None 2024-09-17T07:21:00.989 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2024-09-17T07:21:12.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-17T07:21:12.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:21:13.376 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-09-17T07:21:13.800 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-17T07:21:13.800 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-17T07:21:13.800 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-09-17T07:21:13.815 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-508.el9.x86_64 2024-09-17T07:21:13.815 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T07:21:13.815 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T07:21:13.815 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T07:21:14.816 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T07:21:14.817 DEBUG:teuthology.orchestra.run.smithi192:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T07:21:14.907 INFO:teuthology.orchestra.run.smithi192.stdout:ttyS1 2024-09-17T07:21:14.941 DEBUG:teuthology.parallel:result is None 2024-09-17T07:21:14.941 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T07:21:14.948 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T07:21:14.948 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T07:21:14.952 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T07:21:14.970 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T07:21:14.976 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T07:21:14.986 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T07:21:14.986 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T07:21:15.013 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T07:21:15.045 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T07:21:15.051 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T07:21:15.051 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T07:21:15.076 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T07:21:15.113 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T07:21:15.124 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T07:21:15.125 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T07:21:15.134 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T07:21:15.136 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T07:21:15.147 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T07:21:15.147 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T07:21:15.170 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T07:21:15.199 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T07:21:15.209 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T07:21:15.209 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T07:21:15.239 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T07:21:15.254 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T07:21:15.336 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T07:21:15.391 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T07:21:15.476 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T07:21:15.517 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:21:15.517 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T07:21:15.584 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T07:21:15.627 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T07:21:15.673 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T07:21:15.750 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T07:21:15.788 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:21:15.788 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T07:21:15.850 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-09-17T07:21:15.853 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-09-17T07:21:15.897 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T07:21:15.934 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T07:21:16.215 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T07:21:16.222 INFO:teuthology.task.internal:Starting timer... 2024-09-17T07:21:16.222 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T07:21:16.234 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T07:21:16.243 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-09-17T07:21:16.314 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-09-17T07:21:16.316 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-09-17T07:21:16.365 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-09-17T07:21:16.366 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T07:21:16.366 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-09-17T07:21:16.391 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-09-17T07:21:16.391 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-09-17T07:21:16.417 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-09-17T07:21:16.417 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2024-09-17T07:21:16.417 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T07:21:16.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T07:21:16.461 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T07:21:16.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T07:21:16.486 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T07:21:16.486 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-09-17T07:21:16.538 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-09-17T07:21:16.580 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T07:21:16.590 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T07:21:16.601 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T07:21:16.602 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T07:25:05.597 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2024-09-17T07:25:05.599 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-17T07:25:05.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:25:05.680 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-09-17T07:25:05.757 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-17T07:25:05.758 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-17T07:25:05.758 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T07:25:05.836 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-09-17T07:25:05.919 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-17T07:25:05.920 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T07:25:05.930 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T07:25:05.931 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T07:25:05.931 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T07:25:05.934 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T07:25:05.934 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T07:25:05.965 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T07:25:05.981 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T07:25:05.987 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T07:25:06.001 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T07:25:06.008 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-09-17T07:25:06.020 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-09-17T07:25:06.026 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-09-17T07:25:06.036 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T07:25:06.037 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-09-17T07:25:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T07:25:06.051 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T07:25:06.065 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T07:25:06.109 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-09-17T07:25:06.113 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T07:25:06.113 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-09-17T07:25:06.113 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T07:25:06.113 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T07:25:06.113 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T07:25:06.113 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T07:25:06.120 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-09-17T07:25:06.123 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T07:25:06.123 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-09-17T07:25:06.123 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T07:25:06.123 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T07:25:06.123 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T07:25:06.124 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T07:25:06.124 INFO:teuthology.run_tasks:Running task install... 2024-09-17T07:25:06.133 DEBUG:teuthology.task.install:project ceph 2024-09-17T07:25:06.133 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}} 2024-09-17T07:25:06.133 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'} 2024-09-17T07:25:06.133 INFO:teuthology.task.install:Using flavor: default 2024-09-17T07:25:06.139 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T07:25:06.139 INFO:teuthology.task.install:extra packages: [] 2024-09-17T07:25:06.139 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361', 'tag': None, 'wait_for_package': False} 2024-09-17T07:25:06.140 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-17T07:25:06.141 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361', 'tag': None, 'wait_for_package': False} 2024-09-17T07:25:06.142 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-17T07:25:06.285 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/ 2024-09-17T07:25:06.286 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4883.g6e8474a0 2024-09-17T07:25:06.290 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/ 2024-09-17T07:25:06.290 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4883.g6e8474a0 2024-09-17T07:25:06.399 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T07:25:06.399 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:25:06.399 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T07:25:06.402 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-dis-testing/6e8474a02f2b86353394076ab80c6f50ea28e361/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T07:25:06.403 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:25:06.403 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T07:25:06.426 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T07:25:06.426 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6e8474a02f2b86353394076ab80c6f50ea28e361/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T07:25:06.431 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T07:25:06.431 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6e8474a02f2b86353394076ab80c6f50ea28e361/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T07:25:06.492 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T07:25:06.498 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T07:25:06.567 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T07:25:06.575 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T07:25:06.632 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-09-17T07:25:06.634 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-09-17T07:25:06.640 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-09-17T07:25:06.642 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-09-17T07:25:06.868 INFO:teuthology.orchestra.run.smithi192.stdout:45 files removed 2024-09-17T07:25:06.892 INFO:teuthology.orchestra.run.smithi045.stdout:45 files removed 2024-09-17T07:25:06.894 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw 2024-09-17T07:25:06.919 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw 2024-09-17T07:25:07.440 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 321 kB/s | 84 kB 00:00 2024-09-17T07:25:07.488 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 313 kB/s | 84 kB 00:00 2024-09-17T07:25:07.715 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 104 kB/s | 24 kB 00:00 2024-09-17T07:25:07.780 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-17T07:25:07.982 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-17T07:25:08.064 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-17T07:25:08.658 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-17T07:25:09.956 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.3 MB 00:01 2024-09-17T07:25:10.592 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-17T07:25:13.365 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 8.9 MB/s | 20 MB 00:02 2024-09-17T07:25:16.260 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-17T07:25:17.727 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 85 kB/s | 19 kB 00:00 2024-09-17T07:25:18.682 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-17T07:25:18.717 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.5 MB 00:01 2024-09-17T07:25:20.167 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 111 kB/s | 19 kB 00:00 2024-09-17T07:25:21.580 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-17T07:25:24.120 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-17T07:25:26.355 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:25:26.381 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:25:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-17T07:25:26.381 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:25:26.381 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:25:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 11 M 2024-09-17T07:25:26.382 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-09-17T07:25:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.5 M 2024-09-17T07:25:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.2 M 2024-09-17T07:25:26.382 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-17T07:25:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T07:25:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 5.5 M 2024-09-17T07:25:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 22 M 2024-09-17T07:25:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 25 k 2024-09-17T07:25:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T07:25:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T07:25:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T07:25:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 755 k 2024-09-17T07:25:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T07:25:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T07:25:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T07:25:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T07:25:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 508 k 2024-09-17T07:25:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T07:25:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 5.5 M 2024-09-17T07:25:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T07:25:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T07:25:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T07:25:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 45 k 2024-09-17T07:25:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 143 k 2024-09-17T07:25:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 162 k 2024-09-17T07:25:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T07:25:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 320 k 2024-09-17T07:25:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 307 k 2024-09-17T07:25:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 99 k 2024-09-17T07:25:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T07:25:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T07:25:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T07:25:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T07:25:26.386 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-09-17T07:25:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T07:25:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:25:26.387 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:25:26.387 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:25:26.387 INFO:teuthology.orchestra.run.smithi192.stdout:Install 31 Packages 2024-09-17T07:25:26.387 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 2 Packages 2024-09-17T07:25:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:25:26.388 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 62 M 2024-09-17T07:25:26.388 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:25:27.344 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-17T07:25:27.380 INFO:teuthology.orchestra.run.smithi192.stdout:(1/33): ceph-base-19.3.0-4883.g6e8474a0.el9.x86 6.9 MB/s | 5.5 MB 00:00 2024-09-17T07:25:27.430 INFO:teuthology.orchestra.run.smithi192.stdout:(2/33): ceph-selinux-19.3.0-4883.g6e8474a0.el9. 507 kB/s | 25 kB 00:00 2024-09-17T07:25:27.597 INFO:teuthology.orchestra.run.smithi192.stdout:(3/33): libcephfs2-19.3.0-4883.g6e8474a0.el9.x8 4.4 MB/s | 755 kB 00:00 2024-09-17T07:25:27.664 INFO:teuthology.orchestra.run.smithi192.stdout:(4/33): libradosstriper1-19.3.0-4883.g6e8474a0. 7.5 MB/s | 508 kB 00:00 2024-09-17T07:25:27.981 INFO:teuthology.orchestra.run.smithi192.stdout:(5/33): ceph-radosgw-19.3.0-4883.g6e8474a0.el9. 7.8 MB/s | 11 MB 00:01 2024-09-17T07:25:28.031 INFO:teuthology.orchestra.run.smithi192.stdout:(6/33): python3-ceph-argparse-19.3.0-4883.g6e84 905 kB/s | 45 kB 00:00 2024-09-17T07:25:28.123 INFO:teuthology.orchestra.run.smithi192.stdout:(7/33): librgw2-19.3.0-4883.g6e8474a0.el9.x86_6 12 MB/s | 5.5 MB 00:00 2024-09-17T07:25:28.149 INFO:teuthology.orchestra.run.smithi192.stdout:(8/33): python3-ceph-common-19.3.0-4883.g6e8474 1.2 MB/s | 143 kB 00:00 2024-09-17T07:25:28.182 INFO:teuthology.orchestra.run.smithi192.stdout:(9/33): python3-cephfs-19.3.0-4883.g6e8474a0.el 2.7 MB/s | 162 kB 00:00 2024-09-17T07:25:28.216 INFO:teuthology.orchestra.run.smithi192.stdout:(10/33): python3-rados-19.3.0-4883.g6e8474a0.el 4.7 MB/s | 320 kB 00:00 2024-09-17T07:25:28.241 INFO:teuthology.orchestra.run.smithi192.stdout:(11/33): python3-rbd-19.3.0-4883.g6e8474a0.el9. 5.1 MB/s | 307 kB 00:00 2024-09-17T07:25:28.266 INFO:teuthology.orchestra.run.smithi192.stdout:(12/33): python3-rgw-19.3.0-4883.g6e8474a0.el9. 1.9 MB/s | 99 kB 00:00 2024-09-17T07:25:28.467 INFO:teuthology.orchestra.run.smithi192.stdout:(13/33): ceph-common-19.3.0-4883.g6e8474a0.el9. 12 MB/s | 22 MB 00:01 2024-09-17T07:25:28.517 INFO:teuthology.orchestra.run.smithi192.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 388 kB/s | 107 kB 00:00 2024-09-17T07:25:28.592 INFO:teuthology.orchestra.run.smithi192.stdout:(15/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-09-17T07:25:28.618 INFO:teuthology.orchestra.run.smithi192.stdout:(16/33): libnbd-1.20.2-2.el9.x86_64.rpm 486 kB/s | 171 kB 00:00 2024-09-17T07:25:28.643 INFO:teuthology.orchestra.run.smithi192.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 912 kB/s | 160 kB 00:00 2024-09-17T07:25:28.693 INFO:teuthology.orchestra.run.smithi192.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 844 kB/s | 42 kB 00:00 2024-09-17T07:25:28.718 INFO:teuthology.orchestra.run.smithi192.stdout:(19/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-17T07:25:28.761 INFO:teuthology.orchestra.run.smithi192.stdout:(20/33): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-17T07:25:28.786 INFO:teuthology.orchestra.run.smithi192.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 536 kB/s | 36 kB 00:00 2024-09-17T07:25:28.811 INFO:teuthology.orchestra.run.smithi192.stdout:(22/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-17T07:25:28.861 INFO:teuthology.orchestra.run.smithi192.stdout:(23/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.0 MB/s | 308 kB 00:00 2024-09-17T07:25:28.895 INFO:teuthology.orchestra.run.smithi192.stdout:(24/33): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 662 kB 00:00 2024-09-17T07:25:28.920 INFO:teuthology.orchestra.run.smithi192.stdout:(25/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 457 kB/s | 26 kB 00:00 2024-09-17T07:25:28.954 INFO:teuthology.orchestra.run.smithi192.stdout:(26/33): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-17T07:25:28.987 INFO:teuthology.orchestra.run.smithi192.stdout:(27/33): liboath-2.6.7-2.el9.x86_64.rpm 527 kB/s | 49 kB 00:00 2024-09-17T07:25:29.021 INFO:teuthology.orchestra.run.smithi192.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-17T07:25:29.063 INFO:teuthology.orchestra.run.smithi192.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 839 kB 00:00 2024-09-17T07:25:29.172 INFO:teuthology.orchestra.run.smithi192.stdout:(30/33): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-17T07:25:29.214 INFO:teuthology.orchestra.run.smithi192.stdout:(31/33): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2024-09-17T07:25:29.455 INFO:teuthology.orchestra.run.smithi192.stdout:(32/33): librados2-19.3.0-4883.g6e8474a0.el9.x8 8.8 MB/s | 3.5 MB 00:00 2024-09-17T07:25:29.514 INFO:teuthology.orchestra.run.smithi192.stdout:(33/33): librbd1-19.3.0-4883.g6e8474a0.el9.x86_ 9.5 MB/s | 3.2 MB 00:00 2024-09-17T07:25:29.518 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:25:29.518 INFO:teuthology.orchestra.run.smithi192.stdout:Total 20 MB/s | 62 MB 00:03 2024-09-17T07:25:29.628 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:25:29.631 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:25:29.640 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:25:29.640 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-17T07:25:29.640 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:25:29.640 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:25:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 11 M 2024-09-17T07:25:29.641 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-09-17T07:25:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.5 M 2024-09-17T07:25:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.2 M 2024-09-17T07:25:29.641 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-17T07:25:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T07:25:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 5.5 M 2024-09-17T07:25:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 22 M 2024-09-17T07:25:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 25 k 2024-09-17T07:25:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T07:25:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T07:25:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T07:25:29.641 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 755 k 2024-09-17T07:25:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T07:25:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T07:25:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T07:25:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T07:25:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 508 k 2024-09-17T07:25:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T07:25:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 5.5 M 2024-09-17T07:25:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T07:25:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T07:25:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T07:25:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 45 k 2024-09-17T07:25:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 143 k 2024-09-17T07:25:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 162 k 2024-09-17T07:25:29.642 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T07:25:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 320 k 2024-09-17T07:25:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 307 k 2024-09-17T07:25:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 99 k 2024-09-17T07:25:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T07:25:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T07:25:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T07:25:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T07:25:29.643 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-09-17T07:25:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T07:25:29.643 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:25:29.643 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:25:29.643 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:25:29.643 INFO:teuthology.orchestra.run.smithi045.stdout:Install 31 Packages 2024-09-17T07:25:29.644 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 2 Packages 2024-09-17T07:25:29.644 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:25:29.644 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 62 M 2024-09-17T07:25:29.644 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:25:29.664 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:25:29.665 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:25:30.356 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:25:30.356 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:25:31.245 INFO:teuthology.orchestra.run.smithi045.stdout:(1/33): ceph-radosgw-19.3.0-4883.g6e8474a0.el9. 7.1 MB/s | 11 MB 00:01 2024-09-17T07:25:31.329 INFO:teuthology.orchestra.run.smithi045.stdout:(2/33): ceph-base-19.3.0-4883.g6e8474a0.el9.x86 3.4 MB/s | 5.5 MB 00:01 2024-09-17T07:25:31.346 INFO:teuthology.orchestra.run.smithi045.stdout:(3/33): ceph-selinux-19.3.0-4883.g6e8474a0.el9. 251 kB/s | 25 kB 00:00 2024-09-17T07:25:31.405 INFO:teuthology.orchestra.run.smithi045.stdout:(4/33): libradosstriper1-19.3.0-4883.g6e8474a0. 8.5 MB/s | 508 kB 00:00 2024-09-17T07:25:31.505 INFO:teuthology.orchestra.run.smithi045.stdout:(5/33): libcephfs2-19.3.0-4883.g6e8474a0.el9.x8 4.2 MB/s | 755 kB 00:00 2024-09-17T07:25:31.547 INFO:teuthology.orchestra.run.smithi045.stdout:(6/33): python3-ceph-argparse-19.3.0-4883.g6e84 1.1 MB/s | 45 kB 00:00 2024-09-17T07:25:31.597 INFO:teuthology.orchestra.run.smithi045.stdout:(7/33): python3-ceph-common-19.3.0-4883.g6e8474 2.8 MB/s | 143 kB 00:00 2024-09-17T07:25:31.689 INFO:teuthology.orchestra.run.smithi045.stdout:(8/33): librgw2-19.3.0-4883.g6e8474a0.el9.x86_6 19 MB/s | 5.5 MB 00:00 2024-09-17T07:25:31.714 INFO:teuthology.orchestra.run.smithi045.stdout:(9/33): python3-cephfs-19.3.0-4883.g6e8474a0.el 1.4 MB/s | 162 kB 00:00 2024-09-17T07:25:31.740 INFO:teuthology.orchestra.run.smithi045.stdout:(10/33): python3-rados-19.3.0-4883.g6e8474a0.el 6.2 MB/s | 320 kB 00:00 2024-09-17T07:25:31.790 INFO:teuthology.orchestra.run.smithi045.stdout:(11/33): python3-rgw-19.3.0-4883.g6e8474a0.el9. 1.9 MB/s | 99 kB 00:00 2024-09-17T07:25:31.815 INFO:teuthology.orchestra.run.smithi045.stdout:(12/33): python3-rbd-19.3.0-4883.g6e8474a0.el9. 3.0 MB/s | 307 kB 00:00 2024-09-17T07:25:31.856 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:25:31.915 INFO:teuthology.orchestra.run.smithi045.stdout:(13/33): libnbd-1.20.2-2.el9.x86_64.rpm 1.7 MB/s | 171 kB 00:00 2024-09-17T07:25:31.939 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-17T07:25:31.941 INFO:teuthology.orchestra.run.smithi045.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 709 kB/s | 107 kB 00:00 2024-09-17T07:25:31.967 INFO:teuthology.orchestra.run.smithi045.stdout:(15/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-17T07:25:31.969 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-17T07:25:31.992 INFO:teuthology.orchestra.run.smithi045.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-09-17T07:25:31.994 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-17T07:25:32.025 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T07:25:32.034 INFO:teuthology.orchestra.run.smithi045.stdout:(17/33): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-17T07:25:32.046 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T07:25:32.076 INFO:teuthology.orchestra.run.smithi045.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-17T07:25:32.101 INFO:teuthology.orchestra.run.smithi045.stdout:(19/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-17T07:25:32.126 INFO:teuthology.orchestra.run.smithi045.stdout:(20/33): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-17T07:25:32.152 INFO:teuthology.orchestra.run.smithi045.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 715 kB/s | 36 kB 00:00 2024-09-17T07:25:32.177 INFO:teuthology.orchestra.run.smithi045.stdout:(22/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-09-17T07:25:32.235 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T07:25:32.236 INFO:teuthology.orchestra.run.smithi045.stdout:(23/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.6 MB/s | 308 kB 00:00 2024-09-17T07:25:32.261 INFO:teuthology.orchestra.run.smithi045.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-17T07:25:32.286 INFO:teuthology.orchestra.run.smithi045.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-17T07:25:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T07:25:32.312 INFO:teuthology.orchestra.run.smithi045.stdout:(26/33): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-17T07:25:32.379 INFO:teuthology.orchestra.run.smithi045.stdout:(27/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-17T07:25:32.412 INFO:teuthology.orchestra.run.smithi045.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-17T07:25:32.488 INFO:teuthology.orchestra.run.smithi045.stdout:(29/33): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-17T07:25:32.563 INFO:teuthology.orchestra.run.smithi045.stdout:(30/33): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-17T07:25:32.711 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-17T07:25:32.754 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-17T07:25:32.763 INFO:teuthology.orchestra.run.smithi045.stdout:(31/33): ceph-common-19.3.0-4883.g6e8474a0.el9. 7.1 MB/s | 22 MB 00:03 2024-09-17T07:25:32.829 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-17T07:25:32.889 INFO:teuthology.orchestra.run.smithi045.stdout:(32/33): librbd1-19.3.0-4883.g6e8474a0.el9.x86_ 10 MB/s | 3.2 MB 00:00 2024-09-17T07:25:32.920 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 8/35 2024-09-17T07:25:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 9/35 2024-09-17T07:25:33.019 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 9/35 2024-09-17T07:25:33.056 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-17T07:25:33.100 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9. 11/35 2024-09-17T07:25:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 12/35 2024-09-17T07:25:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 13/35 2024-09-17T07:25:33.214 INFO:teuthology.orchestra.run.smithi045.stdout:(33/33): librados2-19.3.0-4883.g6e8474a0.el9.x8 4.8 MB/s | 3.5 MB 00:00 2024-09-17T07:25:33.220 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:25:33.220 INFO:teuthology.orchestra.run.smithi045.stdout:Total 17 MB/s | 62 MB 00:03 2024-09-17T07:25:33.264 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 13/35 2024-09-17T07:25:33.298 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-17T07:25:33.337 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:25:33.381 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:25:33.381 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:25:33.409 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-17T07:25:33.441 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-17T07:25:33.500 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-17T07:25:33.539 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-17T07:25:33.568 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-17T07:25:33.602 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T07:25:33.642 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-17T07:25:33.966 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-17T07:25:34.007 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 23/35 2024-09-17T07:25:34.094 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:25:34.094 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:25:34.097 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 23/35 2024-09-17T07:25:34.132 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 24/35 2024-09-17T07:25:34.172 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-17T07:25:34.664 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-17T07:25:34.701 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 27/35 2024-09-17T07:25:34.772 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 27/35 2024-09-17T07:25:34.859 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 28/35 2024-09-17T07:25:34.895 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x8 29/35 2024-09-17T07:25:35.693 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:25:35.774 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-17T07:25:35.811 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-17T07:25:35.836 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-17T07:25:35.859 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T07:25:35.884 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-17T07:25:36.046 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T07:25:36.098 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-17T07:25:36.453 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-17T07:25:36.563 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-17T07:25:36.656 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-17T07:25:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 8/35 2024-09-17T07:25:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 9/35 2024-09-17T07:25:36.838 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 9/35 2024-09-17T07:25:36.874 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-17T07:25:36.936 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9. 11/35 2024-09-17T07:25:37.005 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 12/35 2024-09-17T07:25:37.034 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 13/35 2024-09-17T07:25:37.082 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 13/35 2024-09-17T07:25:37.116 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-17T07:25:37.203 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-17T07:25:37.236 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-17T07:25:37.249 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-17T07:25:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-17T07:25:37.320 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-17T07:25:37.365 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-17T07:25:37.395 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-17T07:25:37.428 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T07:25:37.470 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-17T07:25:37.819 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-17T07:25:37.846 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-17T07:25:37.858 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 23/35 2024-09-17T07:25:37.886 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 31/35 2024-09-17T07:25:37.942 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 23/35 2024-09-17T07:25:37.968 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 31/35 2024-09-17T07:25:37.968 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T07:25:37.968 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T07:25:37.969 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:25:37.977 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 24/35 2024-09-17T07:25:38.013 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-17T07:25:38.016 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-17T07:25:38.537 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-17T07:25:38.569 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 27/35 2024-09-17T07:25:38.641 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 27/35 2024-09-17T07:25:38.731 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 28/35 2024-09-17T07:25:38.763 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x8 29/35 2024-09-17T07:25:41.230 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-17T07:25:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-17T07:25:41.878 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 30/35 2024-09-17T07:25:41.922 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 31/35 2024-09-17T07:25:42.002 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 31/35 2024-09-17T07:25:42.003 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T07:25:42.003 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T07:25:42.003 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:25:42.032 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-17T07:25:48.138 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-17T07:25:48.138 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-09-17T07:25:48.138 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-09-17T07:25:48.139 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-09-17T07:25:48.139 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-09-17T07:25:48.139 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-09-17T07:25:48.139 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2024-09-17T07:25:48.139 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-09-17T07:25:48.139 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:25:49.154 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/35 2024-09-17T07:25:49.180 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/35 2024-09-17T07:25:49.180 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:25:49.180 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T07:25:49.180 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T07:25:49.180 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T07:25:49.180 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:25:49.182 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T07:25:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T07:25:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T07:25:50.456 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T07:25:50.456 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/35 2024-09-17T07:25:50.456 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/35 2024-09-17T07:25:50.456 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/35 2024-09-17T07:25:50.456 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/35 2024-09-17T07:25:50.457 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 5/35 2024-09-17T07:25:50.457 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 6/35 2024-09-17T07:25:50.457 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-17T07:25:50.457 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9. 8/35 2024-09-17T07:25:50.457 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x8 9/35 2024-09-17T07:25:50.457 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 10/35 2024-09-17T07:25:50.457 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 11/35 2024-09-17T07:25:50.457 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 12/35 2024-09-17T07:25:50.457 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 13/35 2024-09-17T07:25:50.457 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-17T07:25:50.457 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-17T07:25:50.457 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-17T07:25:50.457 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-17T07:25:50.458 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-17T07:25:50.458 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-17T07:25:50.458 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T07:25:50.458 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-17T07:25:50.458 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-17T07:25:50.458 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-17T07:25:50.458 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-17T07:25:50.458 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-17T07:25:50.458 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-17T07:25:50.458 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-17T07:25:50.458 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-17T07:25:50.458 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-17T07:25:50.458 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-17T07:25:50.459 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-17T07:25:50.459 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-17T07:25:50.459 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-17T07:25:50.459 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/35 2024-09-17T07:25:51.397 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T07:25:51.397 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:25:51.397 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-09-17T07:25:51.397 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:51.397 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:51.397 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:25:51.397 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T07:25:51.397 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:51.398 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:51.398 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:51.398 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:51.398 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T07:25:51.398 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T07:25:51.398 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T07:25:51.398 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:51.398 INFO:teuthology.orchestra.run.smithi192.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T07:25:51.398 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T07:25:51.398 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T07:25:51.398 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T07:25:51.398 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:51.398 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T07:25:51.399 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:51.399 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T07:25:51.399 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T07:25:51.399 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T07:25:51.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:51.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:51.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:51.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T07:25:51.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:51.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:51.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:51.399 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T07:25:51.399 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T07:25:51.399 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T07:25:51.400 INFO:teuthology.orchestra.run.smithi192.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T07:25:51.400 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T07:25:51.400 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:25:51.400 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:25:51.795 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-test 2024-09-17T07:25:52.385 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:28 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:25:52.396 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-17T07:25:52.396 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-09-17T07:25:52.396 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-09-17T07:25:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-09-17T07:25:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-09-17T07:25:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-09-17T07:25:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-09-17T07:25:52.397 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-09-17T07:25:52.397 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:25:52.519 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:25:52.519 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:25:52.519 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-17T07:25:52.519 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:25:52.519 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:25:52.520 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 50 M 2024-09-17T07:25:52.520 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-17T07:25:52.520 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 161 k 2024-09-17T07:25:52.520 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T07:25:52.520 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T07:25:52.520 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:25:52.520 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:25:52.520 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:25:52.520 INFO:teuthology.orchestra.run.smithi192.stdout:Install 4 Packages 2024-09-17T07:25:52.520 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:25:52.521 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 51 M 2024-09-17T07:25:52.521 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 217 M 2024-09-17T07:25:52.521 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:25:52.787 INFO:teuthology.orchestra.run.smithi192.stdout:(1/4): libcephsqlite-19.3.0-4883.g6e8474a0.el9. 755 kB/s | 161 kB 00:00 2024-09-17T07:25:53.354 INFO:teuthology.orchestra.run.smithi192.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 389 kB/s | 304 kB 00:00 2024-09-17T07:25:53.441 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/35 2024-09-17T07:25:53.456 INFO:teuthology.orchestra.run.smithi192.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 95 kB/s | 64 kB 00:00 2024-09-17T07:25:53.468 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/35 2024-09-17T07:25:53.468 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:25:53.468 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T07:25:53.468 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T07:25:53.469 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T07:25:53.469 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:25:53.470 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T07:25:53.568 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-17T07:25:53.568 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T07:25:54.096 INFO:teuthology.orchestra.run.smithi192.stdout:(4/4): ceph-test-19.3.0-4883.g6e8474a0.el9.x86_ 33 MB/s | 50 MB 00:01 2024-09-17T07:25:54.098 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:25:54.098 INFO:teuthology.orchestra.run.smithi192.stdout:Total 32 MB/s | 51 MB 00:01 2024-09-17T07:25:54.127 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:25:54.148 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:25:54.148 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:25:54.530 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:25:54.530 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:25:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T07:25:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/35 2024-09-17T07:25:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/35 2024-09-17T07:25:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/35 2024-09-17T07:25:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/35 2024-09-17T07:25:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 5/35 2024-09-17T07:25:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_6 6/35 2024-09-17T07:25:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 7/35 2024-09-17T07:25:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9. 8/35 2024-09-17T07:25:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x8 9/35 2024-09-17T07:25:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 10/35 2024-09-17T07:25:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 11/35 2024-09-17T07:25:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 12/35 2024-09-17T07:25:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 13/35 2024-09-17T07:25:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-17T07:25:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-17T07:25:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-17T07:25:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-17T07:25:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-17T07:25:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-17T07:25:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-17T07:25:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-17T07:25:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-17T07:25:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-17T07:25:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-17T07:25:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-17T07:25:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-17T07:25:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-17T07:25:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-17T07:25:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-17T07:25:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-17T07:25:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-17T07:25:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 32/35 2024-09-17T07:25:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-17T07:25:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/35 2024-09-17T07:25:55.093 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:25:55.172 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T07:25:55.220 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T07:25:55.248 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/4 2024-09-17T07:25:55.825 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-17T07:25:55.825 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:25:55.825 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-09-17T07:25:55.825 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:55.825 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:55.825 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:25:55.825 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T07:25:55.826 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:55.826 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:55.826 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:55.826 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:55.826 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T07:25:55.826 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T07:25:55.826 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T07:25:55.826 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:55.826 INFO:teuthology.orchestra.run.smithi045.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T07:25:55.826 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T07:25:55.826 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T07:25:55.827 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T07:25:55.827 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:55.827 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T07:25:55.827 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:55.827 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T07:25:55.827 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T07:25:55.827 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T07:25:55.827 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:55.827 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:55.827 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:55.827 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T07:25:55.828 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:55.828 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:55.828 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:25:55.828 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T07:25:55.828 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T07:25:55.828 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T07:25:55.828 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T07:25:55.828 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T07:25:55.828 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:25:55.828 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:25:56.217 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-test 2024-09-17T07:25:56.824 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:29 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:25:56.959 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:25:56.959 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:25:56.959 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-17T07:25:56.960 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:25:56.960 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:25:56.960 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 50 M 2024-09-17T07:25:56.960 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-17T07:25:56.960 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 161 k 2024-09-17T07:25:56.960 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T07:25:56.960 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T07:25:56.960 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:25:56.960 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:25:56.960 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:25:56.960 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2024-09-17T07:25:56.960 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:25:56.961 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 51 M 2024-09-17T07:25:56.961 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 217 M 2024-09-17T07:25:56.961 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:25:57.391 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.7 MB/s | 304 kB 00:00 2024-09-17T07:25:57.433 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-17T07:25:57.500 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): libcephsqlite-19.3.0-4883.g6e8474a0.el9. 571 kB/s | 161 kB 00:00 2024-09-17T07:25:58.842 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): ceph-test-19.3.0-4883.g6e8474a0.el9.x86_ 31 MB/s | 50 MB 00:01 2024-09-17T07:25:58.844 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:25:58.844 INFO:teuthology.orchestra.run.smithi045.stdout:Total 27 MB/s | 51 MB 00:01 2024-09-17T07:25:58.873 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:25:58.895 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:25:58.895 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:25:59.297 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:25:59.298 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:25:59.994 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:26:00.031 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/4 2024-09-17T07:26:00.093 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T07:26:00.113 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/4 2024-09-17T07:26:00.134 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T07:26:00.161 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/4 2024-09-17T07:26:02.508 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/4 2024-09-17T07:26:02.508 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/4 2024-09-17T07:26:02.508 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/4 2024-09-17T07:26:02.508 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T07:26:02.844 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T07:26:02.844 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:02.844 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:26:02.845 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:02.845 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:02.845 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T07:26:02.845 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T07:26:02.845 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:02.845 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:26:03.139 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph 2024-09-17T07:26:03.752 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:26:03.893 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:26:03.895 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:03.895 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-17T07:26:03.896 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:03.896 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:26:03.896 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 6.6 k 2024-09-17T07:26:03.896 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-17T07:26:03.896 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 2.4 M 2024-09-17T07:26:03.896 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 1.5 M 2024-09-17T07:26:03.896 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 291 k 2024-09-17T07:26:03.896 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 4.8 M 2024-09-17T07:26:03.896 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 17 M 2024-09-17T07:26:03.896 INFO:teuthology.orchestra.run.smithi192.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T07:26:03.896 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T07:26:03.896 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T07:26:03.897 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T07:26:03.897 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T07:26:03.897 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T07:26:03.897 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T07:26:03.897 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T07:26:03.897 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T07:26:03.897 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T07:26:03.897 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections 2024-09-17T07:26:03.897 INFO:teuthology.orchestra.run.smithi192.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-17T07:26:03.897 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T07:26:03.897 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T07:26:03.897 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T07:26:03.897 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T07:26:03.897 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T07:26:03.898 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T07:26:03.898 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T07:26:03.898 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T07:26:03.898 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T07:26:03.898 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T07:26:03.898 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T07:26:03.898 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T07:26:03.898 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T07:26:03.898 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T07:26:03.898 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T07:26:03.898 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T07:26:03.898 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T07:26:03.898 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T07:26:03.898 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T07:26:03.899 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T07:26:03.899 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:03.899 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:26:03.899 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:03.899 INFO:teuthology.orchestra.run.smithi192.stdout:Install 37 Packages 2024-09-17T07:26:03.899 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:03.899 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 31 M 2024-09-17T07:26:03.900 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 109 M 2024-09-17T07:26:03.900 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:26:04.317 INFO:teuthology.orchestra.run.smithi192.stdout:(1/37): ceph-19.3.0-4883.g6e8474a0.el9.x86_64.r 46 kB/s | 6.6 kB 00:00 2024-09-17T07:26:04.484 INFO:teuthology.orchestra.run.smithi192.stdout:(2/37): ceph-mgr-19.3.0-4883.g6e8474a0.el9.x86_ 4.9 MB/s | 1.5 MB 00:00 2024-09-17T07:26:04.534 INFO:teuthology.orchestra.run.smithi192.stdout:(3/37): ceph-mds-19.3.0-4883.g6e8474a0.el9.x86_ 6.6 MB/s | 2.4 MB 00:00 2024-09-17T07:26:04.626 INFO:teuthology.orchestra.run.smithi192.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4883.g6e84 3.1 MB/s | 291 kB 00:00 2024-09-17T07:26:04.693 INFO:teuthology.orchestra.run.smithi192.stdout:(5/37): ceph-mon-19.3.0-4883.g6e8474a0.el9.x86_ 13 MB/s | 4.8 MB 00:00 2024-09-17T07:26:04.860 INFO:teuthology.orchestra.run.smithi192.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-17T07:26:05.052 INFO:teuthology.orchestra.run.smithi192.stdout:(7/37): ceph-osd-19.3.0-4883.g6e8474a0.el9.x86_ 30 MB/s | 17 MB 00:00 2024-09-17T07:26:05.077 INFO:teuthology.orchestra.run.smithi192.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 491 kB/s | 106 kB 00:00 2024-09-17T07:26:05.127 INFO:teuthology.orchestra.run.smithi192.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-17T07:26:05.161 INFO:teuthology.orchestra.run.smithi192.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 2.6 MB/s | 1.2 MB 00:00 2024-09-17T07:26:05.207 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/4 2024-09-17T07:26:05.211 INFO:teuthology.orchestra.run.smithi192.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-17T07:26:05.237 INFO:teuthology.orchestra.run.smithi192.stdout:(12/37): python3-pycparser-2.20-6.el9.noarch.rp 734 kB/s | 135 kB 00:00 2024-09-17T07:26:05.295 INFO:teuthology.orchestra.run.smithi192.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-09-17T07:26:05.303 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/4 2024-09-17T07:26:05.312 INFO:teuthology.orchestra.run.smithi192.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 460 kB/s | 35 kB 00:00 2024-09-17T07:26:05.338 INFO:teuthology.orchestra.run.smithi192.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.8 MB/s | 77 kB 00:00 2024-09-17T07:26:05.363 INFO:teuthology.orchestra.run.smithi192.stdout:(16/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-17T07:26:05.380 INFO:teuthology.orchestra.run.smithi192.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 761 kB/s | 51 kB 00:00 2024-09-17T07:26:05.397 INFO:teuthology.orchestra.run.smithi192.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 708 kB/s | 42 kB 00:00 2024-09-17T07:26:05.447 INFO:teuthology.orchestra.run.smithi192.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-17T07:26:05.464 INFO:teuthology.orchestra.run.smithi192.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 651 kB/s | 43 kB 00:00 2024-09-17T07:26:05.489 INFO:teuthology.orchestra.run.smithi192.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 4.2 MB/s | 172 kB 00:00 2024-09-17T07:26:05.514 INFO:teuthology.orchestra.run.smithi192.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 429 kB/s | 11 kB 00:00 2024-09-17T07:26:05.548 INFO:teuthology.orchestra.run.smithi192.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 528 kB/s | 18 kB 00:00 2024-09-17T07:26:05.573 INFO:teuthology.orchestra.run.smithi192.stdout:(24/37): python3-cherrypy-18.6.1-2.el9.noarch.r 3.2 MB/s | 358 kB 00:00 2024-09-17T07:26:05.599 INFO:teuthology.orchestra.run.smithi192.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-17T07:26:05.624 INFO:teuthology.orchestra.run.smithi192.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-09-17T07:26:05.649 INFO:teuthology.orchestra.run.smithi192.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-09-17T07:26:05.682 INFO:teuthology.orchestra.run.smithi192.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 792 kB/s | 46 kB 00:00 2024-09-17T07:26:05.741 INFO:teuthology.orchestra.run.smithi192.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 925 kB/s | 79 kB 00:00 2024-09-17T07:26:05.767 INFO:teuthology.orchestra.run.smithi192.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 670 kB/s | 16 kB 00:00 2024-09-17T07:26:05.792 INFO:teuthology.orchestra.run.smithi192.stdout:(31/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-17T07:26:05.817 INFO:teuthology.orchestra.run.smithi192.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-17T07:26:05.842 INFO:teuthology.orchestra.run.smithi192.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-09-17T07:26:05.868 INFO:teuthology.orchestra.run.smithi192.stdout:(34/37): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-17T07:26:05.893 INFO:teuthology.orchestra.run.smithi192.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 799 kB/s | 20 kB 00:00 2024-09-17T07:26:05.918 INFO:teuthology.orchestra.run.smithi192.stdout:(36/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.5 MB/s | 427 kB 00:00 2024-09-17T07:26:06.027 INFO:teuthology.orchestra.run.smithi192.stdout:(37/37): lua-devel-5.4.4-4.el9.x86_64.rpm 34 kB/s | 22 kB 00:00 2024-09-17T07:26:06.031 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:26:06.031 INFO:teuthology.orchestra.run.smithi192.stdout:Total 14 MB/s | 31 MB 00:02 2024-09-17T07:26:06.174 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:26:06.214 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:26:06.215 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:26:06.701 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:26:06.701 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:26:07.739 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/4 2024-09-17T07:26:07.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/4 2024-09-17T07:26:07.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/4 2024-09-17T07:26:07.739 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T07:26:08.050 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:26:08.086 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T07:26:08.086 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:08.087 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:26:08.087 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:08.087 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:08.087 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T07:26:08.087 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T07:26:08.087 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:08.087 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:26:08.122 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T07:26:08.167 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T07:26:08.207 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T07:26:08.280 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T07:26:08.325 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T07:26:08.358 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T07:26:08.383 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph 2024-09-17T07:26:08.451 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T07:26:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T07:26:08.586 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T07:26:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T07:26:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T07:26:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T07:26:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T07:26:08.759 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T07:26:08.811 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T07:26:08.853 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T07:26:08.902 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T07:26:08.934 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T07:26:09.005 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:42 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:26:09.008 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T07:26:09.147 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:26:09.149 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:26:09.149 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-17T07:26:09.149 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:26:09.149 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:26:09.150 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 6.6 k 2024-09-17T07:26:09.150 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-17T07:26:09.150 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 2.4 M 2024-09-17T07:26:09.150 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 1.5 M 2024-09-17T07:26:09.150 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 291 k 2024-09-17T07:26:09.150 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 4.8 M 2024-09-17T07:26:09.150 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 17 M 2024-09-17T07:26:09.150 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T07:26:09.150 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T07:26:09.150 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T07:26:09.150 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T07:26:09.150 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T07:26:09.150 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T07:26:09.150 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T07:26:09.151 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T07:26:09.151 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T07:26:09.151 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T07:26:09.151 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections 2024-09-17T07:26:09.151 INFO:teuthology.orchestra.run.smithi045.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-17T07:26:09.151 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T07:26:09.151 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T07:26:09.151 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T07:26:09.151 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T07:26:09.151 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T07:26:09.151 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T07:26:09.151 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T07:26:09.151 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T07:26:09.152 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T07:26:09.152 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T07:26:09.152 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T07:26:09.152 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T07:26:09.152 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T07:26:09.152 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T07:26:09.152 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T07:26:09.152 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T07:26:09.152 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T07:26:09.152 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T07:26:09.152 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T07:26:09.152 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T07:26:09.152 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:09.152 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:26:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:26:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:Install 37 Packages 2024-09-17T07:26:09.153 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 31 M 2024-09-17T07:26:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 109 M 2024-09-17T07:26:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:26:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T07:26:09.243 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T07:26:09.295 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T07:26:09.342 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T07:26:09.400 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T07:26:09.456 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T07:26:09.645 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T07:26:09.696 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T07:26:09.774 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T07:26:09.812 INFO:teuthology.orchestra.run.smithi045.stdout:(1/37): ceph-19.3.0-4883.g6e8474a0.el9.x86_64.r 50 kB/s | 6.6 kB 00:00 2024-09-17T07:26:09.863 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T07:26:09.906 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T07:26:10.043 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T07:26:10.202 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9. 32/37 2024-09-17T07:26:10.230 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/37 2024-09-17T07:26:10.257 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/37 2024-09-17T07:26:10.257 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:26:10.257 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T07:26:10.257 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T07:26:10.257 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T07:26:10.257 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:10.370 INFO:teuthology.orchestra.run.smithi045.stdout:(2/37): ceph-mgr-19.3.0-4883.g6e8474a0.el9.x86_ 2.2 MB/s | 1.5 MB 00:00 2024-09-17T07:26:10.537 INFO:teuthology.orchestra.run.smithi045.stdout:(3/37): ceph-mds-19.3.0-4883.g6e8474a0.el9.x86_ 2.8 MB/s | 2.4 MB 00:00 2024-09-17T07:26:10.658 INFO:teuthology.orchestra.run.smithi045.stdout:(4/37): ceph-mon-19.3.0-4883.g6e8474a0.el9.x86_ 5.7 MB/s | 4.8 MB 00:00 2024-09-17T07:26:10.683 INFO:teuthology.orchestra.run.smithi045.stdout:(5/37): ceph-mgr-modules-core-19.3.0-4883.g6e84 1.9 MB/s | 291 kB 00:00 2024-09-17T07:26:10.809 INFO:teuthology.orchestra.run.smithi045.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-17T07:26:10.859 INFO:teuthology.orchestra.run.smithi045.stdout:(7/37): python3-cryptography-36.0.1-4.el9.x86_6 7.0 MB/s | 1.2 MB 00:00 2024-09-17T07:26:10.884 INFO:teuthology.orchestra.run.smithi045.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-17T07:26:10.910 INFO:teuthology.orchestra.run.smithi045.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-09-17T07:26:10.935 INFO:teuthology.orchestra.run.smithi045.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-09-17T07:26:10.969 INFO:teuthology.orchestra.run.smithi045.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-17T07:26:11.102 INFO:teuthology.orchestra.run.smithi045.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-09-17T07:26:11.136 INFO:teuthology.orchestra.run.smithi045.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-17T07:26:11.161 INFO:teuthology.orchestra.run.smithi045.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 593 kB/s | 35 kB 00:00 2024-09-17T07:26:11.187 INFO:teuthology.orchestra.run.smithi045.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-17T07:26:11.212 INFO:teuthology.orchestra.run.smithi045.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-17T07:26:11.237 INFO:teuthology.orchestra.run.smithi045.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 828 kB/s | 42 kB 00:00 2024-09-17T07:26:11.296 INFO:teuthology.orchestra.run.smithi045.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-17T07:26:11.321 INFO:teuthology.orchestra.run.smithi045.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 204 kB/s | 22 kB 00:00 2024-09-17T07:26:11.346 INFO:teuthology.orchestra.run.smithi045.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 863 kB/s | 43 kB 00:00 2024-09-17T07:26:11.405 INFO:teuthology.orchestra.run.smithi045.stdout:(21/37): python3-cherrypy-18.6.1-2.el9.noarch.r 7.0 MB/s | 358 kB 00:00 2024-09-17T07:26:11.430 INFO:teuthology.orchestra.run.smithi045.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 465 kB/s | 11 kB 00:00 2024-09-17T07:26:11.456 INFO:teuthology.orchestra.run.smithi045.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 705 kB/s | 18 kB 00:00 2024-09-17T07:26:11.481 INFO:teuthology.orchestra.run.smithi045.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-17T07:26:11.506 INFO:teuthology.orchestra.run.smithi045.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-17T07:26:11.532 INFO:teuthology.orchestra.run.smithi045.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-09-17T07:26:11.557 INFO:teuthology.orchestra.run.smithi045.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 388 kB/s | 20 kB 00:00 2024-09-17T07:26:11.582 INFO:teuthology.orchestra.run.smithi045.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 921 kB/s | 46 kB 00:00 2024-09-17T07:26:11.608 INFO:teuthology.orchestra.run.smithi045.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-17T07:26:11.633 INFO:teuthology.orchestra.run.smithi045.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 663 kB/s | 16 kB 00:00 2024-09-17T07:26:11.667 INFO:teuthology.orchestra.run.smithi045.stdout:(31/37): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-17T07:26:11.692 INFO:teuthology.orchestra.run.smithi045.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-17T07:26:11.723 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/37 2024-09-17T07:26:11.734 INFO:teuthology.orchestra.run.smithi045.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 534 kB/s | 36 kB 00:00 2024-09-17T07:26:11.750 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/37 2024-09-17T07:26:11.750 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:26:11.750 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T07:26:11.750 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T07:26:11.751 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T07:26:11.751 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:11.768 INFO:teuthology.orchestra.run.smithi045.stdout:(34/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-17T07:26:11.793 INFO:teuthology.orchestra.run.smithi045.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 797 kB/s | 20 kB 00:00 2024-09-17T07:26:11.827 INFO:teuthology.orchestra.run.smithi045.stdout:(36/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-09-17T07:26:12.198 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 35/37 2024-09-17T07:26:12.222 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 35/37 2024-09-17T07:26:12.222 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:26:12.222 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T07:26:12.222 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T07:26:12.223 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T07:26:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:12.335 INFO:teuthology.orchestra.run.smithi045.stdout:(37/37): ceph-osd-19.3.0-4883.g6e8474a0.el9.x86 8.7 MB/s | 17 MB 00:01 2024-09-17T07:26:12.339 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:26:12.339 INFO:teuthology.orchestra.run.smithi045.stdout:Total 9.6 MB/s | 31 MB 00:03 2024-09-17T07:26:12.458 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 36/37 2024-09-17T07:26:12.481 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 36/37 2024-09-17T07:26:12.482 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:26:12.482 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T07:26:12.482 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T07:26:12.482 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T07:26:12.482 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:12.493 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:26:12.538 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:26:12.538 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:26:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 37/37 2024-09-17T07:26:13.074 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:26:13.075 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:26:14.305 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 37/37 2024-09-17T07:26:14.305 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/37 2024-09-17T07:26:14.305 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/37 2024-09-17T07:26:14.305 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/37 2024-09-17T07:26:14.306 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/37 2024-09-17T07:26:14.306 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 5/37 2024-09-17T07:26:14.306 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9. 6/37 2024-09-17T07:26:14.306 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T07:26:14.306 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T07:26:14.306 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T07:26:14.306 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T07:26:14.306 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T07:26:14.306 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T07:26:14.306 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T07:26:14.306 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T07:26:14.306 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T07:26:14.306 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T07:26:14.307 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T07:26:14.307 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T07:26:14.307 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T07:26:14.307 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T07:26:14.307 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T07:26:14.307 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T07:26:14.307 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T07:26:14.307 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T07:26:14.307 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T07:26:14.307 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T07:26:14.307 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T07:26:14.307 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T07:26:14.308 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T07:26:14.309 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T07:26:14.309 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T07:26:14.309 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T07:26:14.309 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T07:26:14.309 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T07:26:14.309 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T07:26:14.309 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T07:26:14.594 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:26:14.655 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T07:26:14.695 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T07:26:14.724 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T07:26:14.799 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T07:26:14.849 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T07:26:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T07:26:14.978 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T07:26:15.162 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T07:26:15.210 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T07:26:15.245 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T07:26:15.274 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T07:26:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T07:26:15.359 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T07:26:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T07:26:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:15.419 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:26:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:26:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T07:26:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T07:26:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T07:26:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T07:26:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T07:26:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T07:26:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T07:26:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T07:26:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T07:26:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T07:26:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T07:26:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T07:26:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T07:26:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T07:26:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T07:26:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T07:26:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T07:26:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T07:26:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T07:26:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T07:26:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T07:26:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T07:26:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T07:26:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T07:26:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T07:26:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T07:26:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T07:26:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T07:26:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T07:26:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T07:26:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T07:26:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:15.423 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:26:15.424 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T07:26:15.467 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T07:26:15.501 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T07:26:15.551 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T07:26:15.592 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T07:26:15.666 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T07:26:15.791 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-base 2024-09-17T07:26:15.811 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T07:26:15.902 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T07:26:15.962 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T07:26:16.008 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T07:26:16.067 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T07:26:16.130 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T07:26:16.325 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T07:26:16.379 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T07:26:16.388 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:52 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:26:16.460 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T07:26:16.477 INFO:teuthology.orchestra.run.smithi192.stdout:Package ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:26:16.520 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:26:16.520 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-17T07:26:16.521 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:26:16.557 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T07:26:16.592 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install cephadm 2024-09-17T07:26:16.610 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T07:26:16.746 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T07:26:16.925 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9. 32/37 2024-09-17T07:26:16.955 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/37 2024-09-17T07:26:16.982 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 33/37 2024-09-17T07:26:16.983 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:26:16.983 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T07:26:16.983 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T07:26:16.983 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T07:26:16.983 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:17.164 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:53 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:26:17.294 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:26:17.295 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:17.295 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-17T07:26:17.295 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:17.295 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:26:17.295 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 784 k 2024-09-17T07:26:17.295 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:17.295 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:26:17.295 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:17.296 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-17T07:26:17.296 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:17.296 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 784 k 2024-09-17T07:26:17.296 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 791 k 2024-09-17T07:26:17.296 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:26:17.581 INFO:teuthology.orchestra.run.smithi192.stdout:cephadm-19.3.0-4883.g6e8474a0.el9.noarch.rpm 2.7 MB/s | 784 kB 00:00 2024-09-17T07:26:17.582 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:26:17.582 INFO:teuthology.orchestra.run.smithi192.stdout:Total 2.7 MB/s | 784 kB 00:00 2024-09-17T07:26:17.582 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:26:17.587 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:26:17.587 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:26:17.593 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:26:17.593 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:26:17.670 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:26:18.152 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-17T07:26:18.243 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-17T07:26:18.531 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/37 2024-09-17T07:26:18.561 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 34/37 2024-09-17T07:26:18.561 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:26:18.561 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T07:26:18.561 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T07:26:18.561 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T07:26:18.561 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:18.619 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-17T07:26:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-17T07:26:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:26:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:26:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:26:19.032 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 35/37 2024-09-17T07:26:19.057 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 35/37 2024-09-17T07:26:19.057 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:26:19.057 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T07:26:19.057 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T07:26:19.057 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T07:26:19.057 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:19.094 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T07:26:19.299 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 36/37 2024-09-17T07:26:19.325 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 36/37 2024-09-17T07:26:19.325 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:26:19.325 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T07:26:19.325 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T07:26:19.325 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T07:26:19.326 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:19.391 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 37/37 2024-09-17T07:26:19.693 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:26:19.831 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:26:19.832 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-09-17T07:26:19.832 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:19.832 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:26:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 145 k 2024-09-17T07:26:19.832 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:19.833 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:26:19.833 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:19.833 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-17T07:26:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:19.833 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 145 k 2024-09-17T07:26:19.833 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 432 k 2024-09-17T07:26:19.833 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:26:20.046 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-immutable-object-cache-19.3.0-4883.g6e8474 679 kB/s | 145 kB 00:00 2024-09-17T07:26:20.046 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:26:20.047 INFO:teuthology.orchestra.run.smithi192.stdout:Total 675 kB/s | 145 kB 00:00 2024-09-17T07:26:20.047 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:26:20.054 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:26:20.054 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:26:20.111 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:26:20.112 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:26:20.256 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:26:20.314 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-17T07:26:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-17T07:26:20.339 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:26:20.339 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T07:26:20.339 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 37/37 2024-09-17T07:26:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/37 2024-09-17T07:26:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/37 2024-09-17T07:26:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 3/37 2024-09-17T07:26:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 4/37 2024-09-17T07:26:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 5/37 2024-09-17T07:26:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9. 6/37 2024-09-17T07:26:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T07:26:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T07:26:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T07:26:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T07:26:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T07:26:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T07:26:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T07:26:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T07:26:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T07:26:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T07:26:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T07:26:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T07:26:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T07:26:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T07:26:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T07:26:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T07:26:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T07:26:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T07:26:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T07:26:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T07:26:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T07:26:21.121 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T07:26:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T07:26:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T07:26:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T07:26:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T07:26:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T07:26:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T07:26:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T07:26:21.123 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T07:26:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-17T07:26:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:21.322 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:26:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:21.322 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:26:21.541 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr 2024-09-17T07:26:21.971 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T07:26:21.971 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:21.971 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:26:21.971 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:21.971 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:21.971 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:21.971 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:26:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T07:26:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T07:26:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T07:26:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T07:26:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T07:26:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T07:26:21.972 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T07:26:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T07:26:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T07:26:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T07:26:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T07:26:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T07:26:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T07:26:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T07:26:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T07:26:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T07:26:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T07:26:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T07:26:21.973 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T07:26:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T07:26:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T07:26:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T07:26:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T07:26:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T07:26:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T07:26:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T07:26:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T07:26:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T07:26:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T07:26:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T07:26:21.974 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T07:26:21.975 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:21.975 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:26:22.130 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:26:22.222 INFO:teuthology.orchestra.run.smithi192.stdout:Package ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:26:22.265 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:26:22.266 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-17T07:26:22.267 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:26:22.345 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T07:26:22.348 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-base 2024-09-17T07:26:22.922 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:26:22.960 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:26:23.053 INFO:teuthology.orchestra.run.smithi045.stdout:Package ceph-base-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:26:23.054 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:26:23.055 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-17T07:26:23.056 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:23.056 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:26:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 14 M 2024-09-17T07:26:23.056 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-17T07:26:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T07:26:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 34 k 2024-09-17T07:26:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 16 k 2024-09-17T07:26:23.056 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T07:26:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T07:26:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T07:26:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T07:26:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T07:26:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T07:26:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T07:26:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T07:26:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:23.057 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:26:23.057 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:23.057 INFO:teuthology.orchestra.run.smithi192.stdout:Install 12 Packages 2024-09-17T07:26:23.057 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:23.058 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 19 M 2024-09-17T07:26:23.058 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 159 M 2024-09-17T07:26:23.058 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:26:23.097 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:26:23.098 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-17T07:26:23.098 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:26:23.171 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install cephadm 2024-09-17T07:26:23.467 INFO:teuthology.orchestra.run.smithi192.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4883.g6e 221 kB/s | 34 kB 00:00 2024-09-17T07:26:23.492 INFO:teuthology.orchestra.run.smithi192.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4883.g6e8 91 kB/s | 16 kB 00:00 2024-09-17T07:26:23.617 INFO:teuthology.orchestra.run.smithi192.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.1 MB/s | 269 kB 00:00 2024-09-17T07:26:23.667 INFO:teuthology.orchestra.run.smithi192.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-09-17T07:26:23.718 INFO:teuthology.orchestra.run.smithi192.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-17T07:26:23.743 INFO:teuthology.orchestra.run.smithi192.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 782 kB/s | 19 kB 00:00 2024-09-17T07:26:23.770 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:56 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:26:23.827 INFO:teuthology.orchestra.run.smithi192.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 24 MB/s | 2.0 MB 00:00 2024-09-17T07:26:23.852 INFO:teuthology.orchestra.run.smithi192.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-17T07:26:23.877 INFO:teuthology.orchestra.run.smithi192.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-17T07:26:23.907 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:26:23.908 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:26:23.908 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-17T07:26:23.908 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:26:23.908 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:26:23.908 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 784 k 2024-09-17T07:26:23.908 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:23.908 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:26:23.908 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:26:23.908 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-17T07:26:23.909 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:23.909 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 784 k 2024-09-17T07:26:23.909 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 791 k 2024-09-17T07:26:23.909 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:26:23.911 INFO:teuthology.orchestra.run.smithi192.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.5 MB/s | 188 kB 00:00 2024-09-17T07:26:23.978 INFO:teuthology.orchestra.run.smithi192.stdout:(11/12): protobuf-compiler-3.14.0-13.el9.x86_64 2.3 MB/s | 863 kB 00:00 2024-09-17T07:26:24.120 INFO:teuthology.orchestra.run.smithi192.stdout:(12/12): ceph-mgr-dashboard-19.3.0-4883.g6e8474 18 MB/s | 14 MB 00:00 2024-09-17T07:26:24.123 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:26:24.123 INFO:teuthology.orchestra.run.smithi192.stdout:Total 18 MB/s | 19 MB 00:01 2024-09-17T07:26:24.166 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm-19.3.0-4883.g6e8474a0.el9.noarch.rpm 3.0 MB/s | 784 kB 00:00 2024-09-17T07:26:24.167 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:26:24.167 INFO:teuthology.orchestra.run.smithi045.stdout:Total 3.0 MB/s | 784 kB 00:00 2024-09-17T07:26:24.167 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:26:24.171 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:26:24.173 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:26:24.173 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:26:24.179 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:26:24.180 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:26:24.190 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:26:24.190 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:26:24.254 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:26:24.472 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:26:24.472 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:26:24.699 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-17T07:26:24.799 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-17T07:26:25.072 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:26:25.154 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T07:26:25.165 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-17T07:26:25.189 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T07:26:25.235 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T07:26:25.274 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T07:26:25.375 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T07:26:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-17T07:26:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:25.427 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:26:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:26:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:25.428 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:26:25.508 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T07:26:25.602 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T07:26:25.654 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T07:26:25.673 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T07:26:25.691 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T07:26:25.725 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9 10/12 2024-09-17T07:26:26.281 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:00:59 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:26:26.420 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:26:26.421 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:26:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-09-17T07:26:26.421 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:26:26.421 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:26:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 145 k 2024-09-17T07:26:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:26.421 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:26:26.421 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:26:26.421 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-17T07:26:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:26.421 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 145 k 2024-09-17T07:26:26.422 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 432 k 2024-09-17T07:26:26.422 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:26:26.682 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-immutable-object-cache-19.3.0-4883.g6e8474 556 kB/s | 145 kB 00:00 2024-09-17T07:26:26.683 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:26:26.683 INFO:teuthology.orchestra.run.smithi045.stdout:Total 552 kB/s | 145 kB 00:00 2024-09-17T07:26:26.683 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:26:26.691 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:26:26.692 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:26:26.753 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:26:26.754 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:26:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:26:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-17T07:26:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-17T07:26:27.011 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:26:27.011 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T07:26:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0. 1/1 2024-09-17T07:26:28.023 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:28.024 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:26:28.024 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:26:28.024 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:28.024 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:26:28.203 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr 2024-09-17T07:26:28.809 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:01 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:26:28.900 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el 11/12 2024-09-17T07:26:28.901 INFO:teuthology.orchestra.run.smithi045.stdout:Package ceph-mgr-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:26:28.945 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:26:28.946 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-17T07:26:28.946 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:26:28.960 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 12/12 2024-09-17T07:26:29.011 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T07:26:29.605 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:02 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:26:29.742 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:26:29.743 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:26:29.743 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-17T07:26:29.743 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:26:29.743 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:26:29.744 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 14 M 2024-09-17T07:26:29.744 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-17T07:26:29.744 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T07:26:29.744 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 34 k 2024-09-17T07:26:29.744 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 16 k 2024-09-17T07:26:29.744 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T07:26:29.744 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T07:26:29.744 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T07:26:29.744 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T07:26:29.744 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T07:26:29.744 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T07:26:29.744 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T07:26:29.744 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T07:26:29.745 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:29.745 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:26:29.745 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:26:29.745 INFO:teuthology.orchestra.run.smithi045.stdout:Install 12 Packages 2024-09-17T07:26:29.745 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:29.745 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 19 M 2024-09-17T07:26:29.745 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 159 M 2024-09-17T07:26:29.746 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:26:30.146 INFO:teuthology.orchestra.run.smithi045.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4883.g6e 214 kB/s | 34 kB 00:00 2024-09-17T07:26:30.163 INFO:teuthology.orchestra.run.smithi045.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4883.g6e8 93 kB/s | 16 kB 00:00 2024-09-17T07:26:30.297 INFO:teuthology.orchestra.run.smithi045.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.0 MB/s | 269 kB 00:00 2024-09-17T07:26:30.405 INFO:teuthology.orchestra.run.smithi045.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-09-17T07:26:30.455 INFO:teuthology.orchestra.run.smithi045.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-17T07:26:30.472 INFO:teuthology.orchestra.run.smithi045.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-17T07:26:30.531 INFO:teuthology.orchestra.run.smithi045.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 35 MB/s | 2.0 MB 00:00 2024-09-17T07:26:30.573 INFO:teuthology.orchestra.run.smithi045.stdout:(8/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.1 MB/s | 863 kB 00:00 2024-09-17T07:26:30.599 INFO:teuthology.orchestra.run.smithi045.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.1 MB/s | 144 kB 00:00 2024-09-17T07:26:30.623 INFO:teuthology.orchestra.run.smithi045.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 613 kB/s | 31 kB 00:00 2024-09-17T07:26:30.666 INFO:teuthology.orchestra.run.smithi045.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-17T07:26:30.933 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 12/12 2024-09-17T07:26:30.933 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el 1/12 2024-09-17T07:26:30.933 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 2/12 2024-09-17T07:26:30.933 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9 3/12 2024-09-17T07:26:30.934 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T07:26:30.934 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T07:26:30.934 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T07:26:30.934 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T07:26:30.934 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T07:26:30.934 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T07:26:30.934 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T07:26:30.934 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T07:26:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T07:26:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:31.254 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:26:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T07:26:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:26:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:26:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:26:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T07:26:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T07:26:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T07:26:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T07:26:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T07:26:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T07:26:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T07:26:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T07:26:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:31.255 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:26:31.515 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T07:26:31.757 INFO:teuthology.orchestra.run.smithi045.stdout:(12/12): ceph-mgr-dashboard-19.3.0-4883.g6e8474 8.0 MB/s | 14 MB 00:01 2024-09-17T07:26:31.762 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:26:31.762 INFO:teuthology.orchestra.run.smithi045.stdout:Total 9.6 MB/s | 19 MB 00:02 2024-09-17T07:26:31.815 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:26:31.837 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:26:31.837 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:26:32.098 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:08 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:26:32.128 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:26:32.129 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:26:32.231 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:26:32.232 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-09-17T07:26:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-17T07:26:32.232 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-09-17T07:26:32.232 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:26:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 7.4 M 2024-09-17T07:26:32.233 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-17T07:26:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T07:26:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T07:26:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T07:26:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T07:26:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T07:26:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T07:26:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T07:26:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T07:26:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T07:26:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T07:26:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T07:26:32.234 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:32.234 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:26:32.234 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-09-17T07:26:32.234 INFO:teuthology.orchestra.run.smithi192.stdout:Install 12 Packages 2024-09-17T07:26:32.234 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:32.234 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 38 M 2024-09-17T07:26:32.234 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 212 M 2024-09-17T07:26:32.235 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:26:32.734 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:26:32.817 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T07:26:32.852 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T07:26:32.900 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T07:26:32.929 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T07:26:33.031 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T07:26:33.179 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T07:26:33.278 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T07:26:33.327 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T07:26:33.355 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T07:26:33.384 INFO:teuthology.orchestra.run.smithi192.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 232 kB/s | 191 kB 00:00 2024-09-17T07:26:33.387 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9 10/12 2024-09-17T07:26:33.442 INFO:teuthology.orchestra.run.smithi192.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 907 kB/s | 801 kB 00:00 2024-09-17T07:26:33.459 INFO:teuthology.orchestra.run.smithi192.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 436 kB/s | 33 kB 00:00 2024-09-17T07:26:33.493 INFO:teuthology.orchestra.run.smithi192.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 548 kB/s | 18 kB 00:00 2024-09-17T07:26:33.535 INFO:teuthology.orchestra.run.smithi192.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 925 kB/s | 38 kB 00:00 2024-09-17T07:26:33.669 INFO:teuthology.orchestra.run.smithi192.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-09-17T07:26:33.711 INFO:teuthology.orchestra.run.smithi192.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 5.8 MB/s | 245 kB 00:00 2024-09-17T07:26:33.786 INFO:teuthology.orchestra.run.smithi192.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 20 MB/s | 4.9 MB 00:00 2024-09-17T07:26:33.828 INFO:teuthology.orchestra.run.smithi192.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 5.9 MB/s | 250 kB 00:00 2024-09-17T07:26:33.903 INFO:teuthology.orchestra.run.smithi192.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 5.1 MB 00:00 2024-09-17T07:26:34.028 INFO:teuthology.orchestra.run.smithi192.stdout:(11/12): ceph-mgr-diskprediction-local-19.3.0-4 5.0 MB/s | 7.4 MB 00:01 2024-09-17T07:26:34.245 INFO:teuthology.orchestra.run.smithi192.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-09-17T07:26:34.246 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:26:34.246 INFO:teuthology.orchestra.run.smithi192.stdout:Total 19 MB/s | 38 MB 00:02 2024-09-17T07:26:34.600 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:26:34.628 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:26:34.628 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:26:35.067 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:26:35.068 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:26:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:26:35.961 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T07:26:36.002 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T07:26:36.101 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T07:26:36.144 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T07:26:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T07:26:36.368 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T07:26:36.445 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T07:26:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el 11/12 2024-09-17T07:26:36.816 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 12/12 2024-09-17T07:26:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T07:26:37.249 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T07:26:38.506 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T07:26:38.766 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 12/12 2024-09-17T07:26:38.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el 1/12 2024-09-17T07:26:38.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noa 2/12 2024-09-17T07:26:38.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9 3/12 2024-09-17T07:26:38.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T07:26:38.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T07:26:38.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T07:26:38.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T07:26:38.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T07:26:38.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T07:26:38.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T07:26:38.766 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T07:26:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T07:26:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:39.093 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:26:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T07:26:39.093 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:26:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:26:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:26:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T07:26:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T07:26:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T07:26:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T07:26:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T07:26:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T07:26:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T07:26:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T07:26:39.094 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:39.094 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:26:39.319 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T07:26:39.545 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T07:26:39.567 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 12/12 2024-09-17T07:26:39.929 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:12 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:26:40.073 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:26:40.074 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-09-17T07:26:40.074 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-17T07:26:40.074 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-09-17T07:26:40.074 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:26:40.074 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 7.4 M 2024-09-17T07:26:40.075 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-17T07:26:40.075 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T07:26:40.075 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T07:26:40.075 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T07:26:40.075 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T07:26:40.075 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T07:26:40.075 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T07:26:40.075 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T07:26:40.075 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T07:26:40.075 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T07:26:40.075 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T07:26:40.075 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T07:26:40.075 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:40.076 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:26:40.076 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-09-17T07:26:40.076 INFO:teuthology.orchestra.run.smithi045.stdout:Install 12 Packages 2024-09-17T07:26:40.076 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:40.076 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 38 M 2024-09-17T07:26:40.076 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 212 M 2024-09-17T07:26:40.077 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:26:40.717 INFO:teuthology.orchestra.run.smithi045.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 334 kB/s | 191 kB 00:00 2024-09-17T07:26:40.834 INFO:teuthology.orchestra.run.smithi045.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 281 kB/s | 33 kB 00:00 2024-09-17T07:26:40.976 INFO:teuthology.orchestra.run.smithi045.stdout:(3/12): ceph-mgr-diskprediction-local-19.3.0-48 8.9 MB/s | 7.4 MB 00:00 2024-09-17T07:26:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:(4/12): libgfortran-11.5.0-2.el9.x86_64.rpm 935 kB/s | 801 kB 00:00 2024-09-17T07:26:41.126 INFO:teuthology.orchestra.run.smithi045.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 308 kB/s | 38 kB 00:00 2024-09-17T07:26:41.252 INFO:teuthology.orchestra.run.smithi045.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 66 kB/s | 18 kB 00:00 2024-09-17T07:26:41.561 INFO:teuthology.orchestra.run.smithi045.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.1 MB/s | 3.0 MB 00:00 2024-09-17T07:26:41.744 INFO:teuthology.orchestra.run.smithi045.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 8.0 MB/s | 4.9 MB 00:00 2024-09-17T07:26:41.770 INFO:teuthology.orchestra.run.smithi045.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 473 kB/s | 245 kB 00:00 2024-09-17T07:26:41.903 INFO:teuthology.orchestra.run.smithi045.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-09-17T07:26:41.928 INFO:teuthology.orchestra.run.smithi045.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.3 MB/s | 250 kB 00:00 2024-09-17T07:26:42.270 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 12/12 2024-09-17T07:26:42.270 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 1/12 2024-09-17T07:26:42.270 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T07:26:42.270 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T07:26:42.270 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T07:26:42.270 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T07:26:42.271 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T07:26:42.271 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T07:26:42.271 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T07:26:42.271 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T07:26:42.271 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T07:26:42.271 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T07:26:42.696 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T07:26:42.696 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:42.696 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:26:42.696 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:26:42.696 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T07:26:42.697 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T07:26:42.697 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T07:26:42.697 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T07:26:42.697 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T07:26:42.697 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T07:26:42.697 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T07:26:42.697 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T07:26:42.697 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T07:26:42.697 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T07:26:42.697 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T07:26:42.697 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:42.697 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:26:43.054 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-rook 2024-09-17T07:26:43.644 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:19 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:26:43.776 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:26:43.777 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:43.777 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-17T07:26:43.777 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:43.777 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:26:43.777 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 49 k 2024-09-17T07:26:43.777 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-17T07:26:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T07:26:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T07:26:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T07:26:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T07:26:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T07:26:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T07:26:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T07:26:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T07:26:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T07:26:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T07:26:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T07:26:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T07:26:43.778 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:43.779 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:26:43.779 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:43.779 INFO:teuthology.orchestra.run.smithi192.stdout:Install 13 Packages 2024-09-17T07:26:43.779 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:43.779 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 2.2 M 2024-09-17T07:26:43.779 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 26 M 2024-09-17T07:26:43.779 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:26:44.478 INFO:teuthology.orchestra.run.smithi192.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 318 kB/s | 26 kB 00:00 2024-09-17T07:26:44.502 INFO:teuthology.orchestra.run.smithi192.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 181 kB/s | 19 kB 00:00 2024-09-17T07:26:44.578 INFO:teuthology.orchestra.run.smithi192.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-17T07:26:44.611 INFO:teuthology.orchestra.run.smithi192.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-17T07:26:44.637 INFO:teuthology.orchestra.run.smithi192.stdout:(5/13): ceph-mgr-rook-19.3.0-4883.g6e8474a0.el9 205 kB/s | 49 kB 00:00 2024-09-17T07:26:44.662 INFO:teuthology.orchestra.run.smithi192.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-17T07:26:44.687 INFO:teuthology.orchestra.run.smithi192.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 279 kB 00:00 2024-09-17T07:26:44.737 INFO:teuthology.orchestra.run.smithi192.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 321 kB/s | 32 kB 00:00 2024-09-17T07:26:44.763 INFO:teuthology.orchestra.run.smithi192.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-17T07:26:44.788 INFO:teuthology.orchestra.run.smithi192.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-17T07:26:44.813 INFO:teuthology.orchestra.run.smithi192.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.9 MB/s | 220 kB 00:00 2024-09-17T07:26:44.838 INFO:teuthology.orchestra.run.smithi192.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-17T07:26:44.905 INFO:teuthology.orchestra.run.smithi192.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.2 MB/s | 1.0 MB 00:00 2024-09-17T07:26:44.909 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:26:44.909 INFO:teuthology.orchestra.run.smithi192.stdout:Total 2.0 MB/s | 2.2 MB 00:01 2024-09-17T07:26:44.977 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:26:44.993 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:26:44.993 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:26:45.083 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:26:45.084 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:26:45.588 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:26:45.769 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T07:26:45.839 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T07:26:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T07:26:45.932 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T07:26:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T07:26:46.074 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T07:26:46.185 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T07:26:46.242 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T07:26:46.886 INFO:teuthology.orchestra.run.smithi045.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.2 MB/s | 16 MB 00:05 2024-09-17T07:26:46.887 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:26:46.888 INFO:teuthology.orchestra.run.smithi045.stdout:Total 5.6 MB/s | 38 MB 00:06 2024-09-17T07:26:46.952 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T07:26:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T07:26:47.040 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T07:26:47.096 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T07:26:47.118 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 13/13 2024-09-17T07:26:47.258 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:26:47.286 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:26:47.286 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:26:47.758 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:26:47.759 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:26:47.905 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 13/13 2024-09-17T07:26:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 1/13 2024-09-17T07:26:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T07:26:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T07:26:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T07:26:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T07:26:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T07:26:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T07:26:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T07:26:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T07:26:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T07:26:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T07:26:47.906 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T07:26:48.284 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T07:26:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:48.285 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:26:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:26:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T07:26:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T07:26:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T07:26:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T07:26:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T07:26:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T07:26:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T07:26:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T07:26:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T07:26:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T07:26:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T07:26:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T07:26:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:48.286 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:26:48.548 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:26:48.562 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T07:26:48.632 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T07:26:48.677 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T07:26:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T07:26:48.828 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T07:26:49.020 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T07:26:49.052 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T07:26:49.131 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T07:26:49.143 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:25 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:26:49.277 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:26:49.278 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-17T07:26:49.278 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:49.278 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:26:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 154 k 2024-09-17T07:26:49.278 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-17T07:26:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T07:26:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T07:26:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T07:26:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T07:26:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T07:26:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:49.279 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:26:49.279 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:49.279 INFO:teuthology.orchestra.run.smithi192.stdout:Install 6 Packages 2024-09-17T07:26:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:49.279 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 7.0 M 2024-09-17T07:26:49.280 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 33 M 2024-09-17T07:26:49.280 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:26:49.807 INFO:teuthology.orchestra.run.smithi192.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-17T07:26:49.829 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T07:26:49.832 INFO:teuthology.orchestra.run.smithi192.stdout:(2/6): ceph-mgr-cephadm-19.3.0-4883.g6e8474a0.e 658 kB/s | 154 kB 00:00 2024-09-17T07:26:49.891 INFO:teuthology.orchestra.run.smithi192.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-17T07:26:49.916 INFO:teuthology.orchestra.run.smithi192.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-17T07:26:49.950 INFO:teuthology.orchestra.run.smithi192.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.8 MB/s | 548 kB 00:00 2024-09-17T07:26:49.953 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T07:26:50.959 INFO:teuthology.orchestra.run.smithi192.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.4 MB/s | 6.0 MB 00:01 2024-09-17T07:26:50.961 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:26:50.961 INFO:teuthology.orchestra.run.smithi192.stdout:Total 4.2 MB/s | 7.0 MB 00:01 2024-09-17T07:26:51.031 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:26:51.042 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:26:51.043 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:26:51.145 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:26:51.145 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:26:51.289 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T07:26:51.453 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:26:51.618 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T07:26:51.669 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T07:26:52.208 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T07:26:52.306 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T07:26:52.366 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T07:26:52.373 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T07:26:52.395 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 12/12 2024-09-17T07:26:52.398 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 6/6 2024-09-17T07:26:53.161 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 6/6 2024-09-17T07:26:53.161 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/6 2024-09-17T07:26:53.161 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T07:26:53.162 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T07:26:53.162 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T07:26:53.162 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T07:26:53.492 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T07:26:53.492 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:53.492 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:26:53.492 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:26:53.492 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T07:26:53.492 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T07:26:53.492 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T07:26:53.493 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T07:26:53.493 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T07:26:53.493 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:53.493 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:26:53.764 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-fuse 2024-09-17T07:26:54.345 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:30 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:26:54.479 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:26:54.480 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:54.480 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-17T07:26:54.480 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:54.480 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:26:54.480 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 899 k 2024-09-17T07:26:54.480 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-17T07:26:54.480 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T07:26:54.480 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:54.480 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:26:54.480 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:54.480 INFO:teuthology.orchestra.run.smithi192.stdout:Install 2 Packages 2024-09-17T07:26:54.480 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:54.481 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 979 k 2024-09-17T07:26:54.481 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 2.9 M 2024-09-17T07:26:54.481 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:26:54.761 INFO:teuthology.orchestra.run.smithi192.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 439 kB/s | 80 kB 00:00 2024-09-17T07:26:54.852 INFO:teuthology.orchestra.run.smithi192.stdout:(2/2): ceph-fuse-19.3.0-4883.g6e8474a0.el9.x86_ 3.2 MB/s | 899 kB 00:00 2024-09-17T07:26:54.853 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:26:54.853 INFO:teuthology.orchestra.run.smithi192.stdout:Total 2.6 MB/s | 979 kB 00:00 2024-09-17T07:26:54.865 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:26:54.887 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:26:54.887 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:26:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 12/12 2024-09-17T07:26:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e847 1/12 2024-09-17T07:26:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T07:26:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T07:26:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T07:26:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T07:26:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T07:26:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T07:26:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T07:26:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T07:26:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T07:26:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T07:26:54.966 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:26:54.967 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:26:55.193 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:26:55.329 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T07:26:55.401 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T07:26:55.401 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:55.401 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:26:55.401 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:26:55.401 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T07:26:55.401 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T07:26:55.401 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T07:26:55.402 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T07:26:55.402 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T07:26:55.402 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T07:26:55.402 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T07:26:55.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T07:26:55.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T07:26:55.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T07:26:55.402 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T07:26:55.402 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:55.402 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:26:55.403 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/2 2024-09-17T07:26:55.774 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-rook 2024-09-17T07:26:56.082 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/2 2024-09-17T07:26:56.082 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/2 2024-09-17T07:26:56.377 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T07:26:56.377 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:56.377 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:26:56.377 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T07:26:56.377 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:56.377 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:26:56.384 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:29 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:26:56.521 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:26:56.522 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:26:56.523 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-17T07:26:56.523 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:26:56.523 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:26:56.523 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 49 k 2024-09-17T07:26:56.523 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-17T07:26:56.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T07:26:56.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T07:26:56.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T07:26:56.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T07:26:56.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T07:26:56.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T07:26:56.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T07:26:56.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T07:26:56.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T07:26:56.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T07:26:56.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T07:26:56.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T07:26:56.524 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:56.524 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:26:56.524 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:26:56.524 INFO:teuthology.orchestra.run.smithi045.stdout:Install 13 Packages 2024-09-17T07:26:56.524 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:26:56.525 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 2.2 M 2024-09-17T07:26:56.525 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 26 M 2024-09-17T07:26:56.525 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:26:56.659 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-volume 2024-09-17T07:26:56.980 INFO:teuthology.orchestra.run.smithi045.stdout:(1/13): ceph-mgr-rook-19.3.0-4883.g6e8474a0.el9 298 kB/s | 49 kB 00:00 2024-09-17T07:26:57.249 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:33 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:26:57.339 INFO:teuthology.orchestra.run.smithi045.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 37 kB/s | 19 kB 00:00 2024-09-17T07:26:57.364 INFO:teuthology.orchestra.run.smithi045.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 47 kB/s | 26 kB 00:00 2024-09-17T07:26:57.387 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:26:57.387 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-17T07:26:57.387 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:57.387 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:26:57.387 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 285 k 2024-09-17T07:26:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:57.388 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:26:57.388 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:26:57.388 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-17T07:26:57.388 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:57.388 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 285 k 2024-09-17T07:26:57.388 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 1.3 M 2024-09-17T07:26:57.389 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:26:57.624 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-volume-19.3.0-4883.g6e8474a0.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-17T07:26:57.625 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:26:57.625 INFO:teuthology.orchestra.run.smithi192.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-17T07:26:57.625 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:26:57.633 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:26:57.633 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:26:57.659 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:26:57.659 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:26:57.756 INFO:teuthology.orchestra.run.smithi045.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 287 kB/s | 222 kB 00:00 2024-09-17T07:26:57.874 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:26:57.942 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-17T07:26:57.958 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-17T07:26:57.959 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:26:57.959 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T07:26:57.959 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:58.048 INFO:teuthology.orchestra.run.smithi045.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 224 kB/s | 159 kB 00:00 2024-09-17T07:26:58.098 INFO:teuthology.orchestra.run.smithi045.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 660 kB/s | 32 kB 00:00 2024-09-17T07:26:58.123 INFO:teuthology.orchestra.run.smithi045.stdout:(7/13): python3-certifi-2023.05.07-4.el9.noarch 569 kB/s | 14 kB 00:00 2024-09-17T07:26:58.157 INFO:teuthology.orchestra.run.smithi045.stdout:(8/13): python3-google-auth-2.34.0-1.el9.noarch 6.5 MB/s | 220 kB 00:00 2024-09-17T07:26:58.215 INFO:teuthology.orchestra.run.smithi045.stdout:(9/13): python3-kubernetes-26.1.0-2.el9.noarch. 18 MB/s | 1.0 MB 00:00 2024-09-17T07:26:58.241 INFO:teuthology.orchestra.run.smithi045.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 318 kB/s | 279 kB 00:00 2024-09-17T07:26:58.266 INFO:teuthology.orchestra.run.smithi045.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-17T07:26:58.325 INFO:teuthology.orchestra.run.smithi045.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.1 MB/s | 90 kB 00:00 2024-09-17T07:26:58.434 INFO:teuthology.orchestra.run.smithi045.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 79 kB/s | 54 kB 00:00 2024-09-17T07:26:58.872 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:26:58.873 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.2 MB/s | 2.2 MB 00:01 2024-09-17T07:26:58.873 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:26:58.873 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:26:58.873 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:26:58.873 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:26:58.873 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:26:58.991 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-17T07:26:58.991 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:58.991 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:26:58.991 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:26:58.991 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:26:58.991 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:26:59.145 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:26:59.303 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librados-devel 2024-09-17T07:26:59.330 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T07:26:59.394 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T07:26:59.459 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T07:26:59.495 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T07:26:59.545 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T07:26:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T07:26:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T07:26:59.831 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T07:26:59.891 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:27:00.027 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:27:00.027 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:27:00.028 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-09-17T07:27:00.028 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:27:00.028 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:27:00.028 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 127 k 2024-09-17T07:27:00.028 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:00.028 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:27:00.028 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:27:00.028 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-17T07:27:00.028 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:00.028 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 127 k 2024-09-17T07:27:00.029 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 456 k 2024-09-17T07:27:00.029 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:27:00.237 INFO:teuthology.orchestra.run.smithi192.stdout:librados-devel-19.3.0-4883.g6e8474a0.el9.x86_64 608 kB/s | 127 kB 00:00 2024-09-17T07:27:00.237 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:27:00.237 INFO:teuthology.orchestra.run.smithi192.stdout:Total 604 kB/s | 127 kB 00:00 2024-09-17T07:27:00.238 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:27:00.243 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:27:00.243 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:27:00.309 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:27:00.309 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:27:00.460 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:27:00.559 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:00.586 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T07:27:00.644 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T07:27:00.681 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T07:27:00.727 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T07:27:00.756 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 13/13 2024-09-17T07:27:01.049 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:01.327 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:27:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:27:01.327 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:01.327 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:27:01.455 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 13/13 2024-09-17T07:27:01.455 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 1/13 2024-09-17T07:27:01.456 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T07:27:01.456 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T07:27:01.456 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T07:27:01.456 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T07:27:01.456 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T07:27:01.456 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T07:27:01.456 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T07:27:01.456 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T07:27:01.456 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T07:27:01.457 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T07:27:01.457 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T07:27:01.524 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install libcephfs2 2024-09-17T07:27:01.798 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T07:27:01.798 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:01.798 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:27:01.798 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:27:01.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T07:27:01.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T07:27:01.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T07:27:01.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T07:27:01.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T07:27:01.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T07:27:01.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T07:27:01.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T07:27:01.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T07:27:01.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T07:27:01.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T07:27:01.799 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T07:27:01.799 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:01.800 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:02.026 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T07:27:02.125 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:27:02.218 INFO:teuthology.orchestra.run.smithi192.stdout:Package libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:27:02.263 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:27:02.264 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-17T07:27:02.264 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:27:02.336 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install libcephfs-devel 2024-09-17T07:27:02.628 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:27:02.766 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:27:02.766 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:02.767 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-17T07:27:02.767 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:02.767 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:27:02.767 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 154 k 2024-09-17T07:27:02.767 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-17T07:27:02.767 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T07:27:02.767 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T07:27:02.767 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T07:27:02.767 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T07:27:02.768 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T07:27:02.768 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:02.768 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:27:02.768 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:02.768 INFO:teuthology.orchestra.run.smithi045.stdout:Install 6 Packages 2024-09-17T07:27:02.768 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:02.768 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 7.0 M 2024-09-17T07:27:02.769 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 33 M 2024-09-17T07:27:02.769 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:27:02.918 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:27:03.050 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:27:03.050 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:27:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-09-17T07:27:03.050 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:27:03.050 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:27:03.050 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 32 k 2024-09-17T07:27:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:03.051 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:27:03.051 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:27:03.051 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-17T07:27:03.051 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:03.051 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 32 k 2024-09-17T07:27:03.051 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 142 k 2024-09-17T07:27:03.051 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:27:03.206 INFO:teuthology.orchestra.run.smithi192.stdout:libcephfs-devel-19.3.0-4883.g6e8474a0.el9.x86_6 205 kB/s | 32 kB 00:00 2024-09-17T07:27:03.207 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:27:03.207 INFO:teuthology.orchestra.run.smithi192.stdout:Total 203 kB/s | 32 kB 00:00 2024-09-17T07:27:03.207 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:27:03.211 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:27:03.211 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:27:03.232 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:27:03.232 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:27:03.324 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:27:03.379 INFO:teuthology.orchestra.run.smithi045.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-09-17T07:27:03.404 INFO:teuthology.orchestra.run.smithi045.stdout:(2/6): ceph-mgr-cephadm-19.3.0-4883.g6e8474a0.e 766 kB/s | 154 kB 00:00 2024-09-17T07:27:03.429 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:03.455 INFO:teuthology.orchestra.run.smithi045.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-17T07:27:03.505 INFO:teuthology.orchestra.run.smithi045.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.3 MB/s | 548 kB 00:00 2024-09-17T07:27:03.530 INFO:teuthology.orchestra.run.smithi045.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 990 kB/s | 74 kB 00:00 2024-09-17T07:27:03.821 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:04.064 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:04.064 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:04.064 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:27:04.064 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:27:04.064 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:04.064 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:27:04.252 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librados2 2024-09-17T07:27:04.780 INFO:teuthology.orchestra.run.smithi045.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.8 MB/s | 6.0 MB 00:01 2024-09-17T07:27:04.783 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:27:04.784 INFO:teuthology.orchestra.run.smithi045.stdout:Total 3.5 MB/s | 7.0 MB 00:02 2024-09-17T07:27:04.828 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:27:04.859 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:27:04.872 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:27:04.872 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:27:04.919 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:27:04.962 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:27:04.963 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-17T07:27:04.963 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:27:04.981 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:27:04.981 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:27:05.037 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librbd1 2024-09-17T07:27:05.300 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:27:05.469 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T07:27:05.533 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T07:27:05.623 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:27:05.714 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:27:05.758 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:27:05.759 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-17T07:27:05.759 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:27:05.826 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rados 2024-09-17T07:27:06.083 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T07:27:06.172 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T07:27:06.248 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T07:27:06.279 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 6/6 2024-09-17T07:27:06.403 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:27:06.492 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:27:06.535 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:27:06.536 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-17T07:27:06.536 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:27:06.608 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rgw 2024-09-17T07:27:07.038 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 6/6 2024-09-17T07:27:07.039 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 1/6 2024-09-17T07:27:07.039 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T07:27:07.039 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T07:27:07.039 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T07:27:07.039 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T07:27:07.191 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:27:07.282 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:27:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T07:27:07.321 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:07.321 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:27:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:27:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T07:27:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T07:27:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T07:27:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T07:27:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T07:27:07.322 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:07.322 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:07.326 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:27:07.327 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-17T07:27:07.327 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:27:07.392 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-cephfs 2024-09-17T07:27:07.572 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-fuse 2024-09-17T07:27:07.969 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:27:08.057 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:27:08.099 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:27:08.101 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-17T07:27:08.101 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:27:08.175 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:27:08.177 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rbd 2024-09-17T07:27:08.318 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:27:08.319 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:08.319 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-17T07:27:08.319 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:08.319 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:27:08.319 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 899 k 2024-09-17T07:27:08.319 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-17T07:27:08.319 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T07:27:08.319 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:08.320 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:27:08.320 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:08.320 INFO:teuthology.orchestra.run.smithi045.stdout:Install 2 Packages 2024-09-17T07:27:08.320 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:08.320 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 979 k 2024-09-17T07:27:08.320 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 2.9 M 2024-09-17T07:27:08.320 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:27:08.762 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:27:08.852 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:27:08.896 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:27:08.897 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-17T07:27:08.897 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:27:08.969 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-fuse 2024-09-17T07:27:09.017 INFO:teuthology.orchestra.run.smithi045.stdout:(1/2): ceph-fuse-19.3.0-4883.g6e8474a0.el9.x86_ 1.5 MB/s | 899 kB 00:00 2024-09-17T07:27:09.050 INFO:teuthology.orchestra.run.smithi045.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 127 kB/s | 80 kB 00:00 2024-09-17T07:27:09.051 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:27:09.052 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.3 MB/s | 979 kB 00:00 2024-09-17T07:27:09.065 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:27:09.091 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:27:09.091 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:27:09.179 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:27:09.179 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:27:09.377 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:27:09.524 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T07:27:09.551 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:27:09.595 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/2 2024-09-17T07:27:09.685 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:27:09.685 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:27:09.685 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-09-17T07:27:09.686 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:27:09.686 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:27:09.686 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 86 k 2024-09-17T07:27:09.686 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:09.686 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:27:09.686 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:27:09.686 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-17T07:27:09.686 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:09.686 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 86 k 2024-09-17T07:27:09.686 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 231 k 2024-09-17T07:27:09.686 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:27:09.876 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-fuse-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 451 kB/s | 86 kB 00:00 2024-09-17T07:27:09.877 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:27:09.877 INFO:teuthology.orchestra.run.smithi192.stdout:Total 448 kB/s | 86 kB 00:00 2024-09-17T07:27:09.877 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:27:09.884 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:27:09.884 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:27:09.948 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:27:09.948 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:27:10.095 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:27:10.190 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:10.299 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2/2 2024-09-17T07:27:10.299 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/2 2024-09-17T07:27:10.581 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T07:27:10.581 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:10.581 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:27:10.581 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T07:27:10.581 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:10.581 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:10.598 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:10.862 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-volume 2024-09-17T07:27:10.874 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:10.874 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:10.875 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:27:10.875 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:27:10.875 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:10.875 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:27:11.147 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-mirror 2024-09-17T07:27:11.475 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:27:11.615 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:27:11.616 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:11.616 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-17T07:27:11.616 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:11.617 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:27:11.617 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.3.0-4883.g6e8474a0.el9 ceph-noarch 285 k 2024-09-17T07:27:11.617 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:11.617 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:27:11.617 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:11.617 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-17T07:27:11.617 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:11.617 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 285 k 2024-09-17T07:27:11.617 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 1.3 M 2024-09-17T07:27:11.617 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:27:11.752 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:27:11.856 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-volume-19.3.0-4883.g6e8474a0.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-17T07:27:11.857 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:27:11.857 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-17T07:27:11.858 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:27:11.867 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:27:11.867 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:27:11.891 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:27:11.892 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:27:11.892 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-09-17T07:27:11.892 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:27:11.892 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:27:11.892 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.2 M 2024-09-17T07:27:11.892 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:11.892 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:27:11.892 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:27:11.892 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-17T07:27:11.892 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:11.893 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 3.2 M 2024-09-17T07:27:11.893 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 14 M 2024-09-17T07:27:11.893 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:27:11.906 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:27:11.906 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:27:12.138 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:27:12.209 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-17T07:27:12.226 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-17T07:27:12.226 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:27:12.227 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T07:27:12.227 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:13.363 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 1/1 2024-09-17T07:27:13.363 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:27:13.363 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.3.0-4883.g6e8474a0.el9.noarch 2024-09-17T07:27:13.363 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:13.634 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librados-devel 2024-09-17T07:27:14.250 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:27:14.387 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:27:14.387 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:14.387 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-09-17T07:27:14.387 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:14.387 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:27:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 127 k 2024-09-17T07:27:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:14.388 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:27:14.388 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:14.388 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-17T07:27:14.388 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:14.388 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 127 k 2024-09-17T07:27:14.389 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 456 k 2024-09-17T07:27:14.389 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:27:14.605 INFO:teuthology.orchestra.run.smithi045.stdout:librados-devel-19.3.0-4883.g6e8474a0.el9.x86_64 586 kB/s | 127 kB 00:00 2024-09-17T07:27:14.606 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:27:14.606 INFO:teuthology.orchestra.run.smithi045.stdout:Total 582 kB/s | 127 kB 00:00 2024-09-17T07:27:14.606 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:27:14.612 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:27:14.612 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:27:14.680 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:27:14.681 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:27:14.839 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:27:14.940 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:15.350 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:15.620 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:15.620 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:15.621 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:27:15.621 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:27:15.621 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:15.621 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:15.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install libcephfs2 2024-09-17T07:27:16.483 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:49 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:27:16.576 INFO:teuthology.orchestra.run.smithi045.stdout:Package libcephfs2-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:27:16.623 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:27:16.624 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-17T07:27:16.624 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:16.698 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install libcephfs-devel 2024-09-17T07:27:17.302 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:50 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:27:17.440 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:27:17.441 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:17.441 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-09-17T07:27:17.441 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:17.441 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:27:17.441 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 32 k 2024-09-17T07:27:17.441 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:17.441 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:27:17.441 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:17.441 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-17T07:27:17.441 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:17.442 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 32 k 2024-09-17T07:27:17.442 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 142 k 2024-09-17T07:27:17.442 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:27:17.593 INFO:teuthology.orchestra.run.smithi045.stdout:libcephfs-devel-19.3.0-4883.g6e8474a0.el9.x86_6 211 kB/s | 32 kB 00:00 2024-09-17T07:27:17.593 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:27:17.594 INFO:teuthology.orchestra.run.smithi045.stdout:Total 209 kB/s | 32 kB 00:00 2024-09-17T07:27:17.594 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:27:17.597 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:27:17.597 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:27:17.626 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:27:17.626 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:27:17.725 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:27:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:18.244 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:18.421 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-mirror-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 501 kB/s | 3.2 MB 00:06 2024-09-17T07:27:18.422 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:27:18.422 INFO:teuthology.orchestra.run.smithi192.stdout:Total 500 kB/s | 3.2 MB 00:06 2024-09-17T07:27:18.422 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:27:18.434 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:27:18.434 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:27:18.495 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:18.496 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:18.496 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:27:18.496 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:27:18.496 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:18.496 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:18.520 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:27:18.520 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:27:18.695 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librados2 2024-09-17T07:27:18.967 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:27:19.038 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:19.065 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:19.065 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:27:19.065 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T07:27:19.065 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T07:27:19.065 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T07:27:19.065 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:19.296 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:52 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:27:19.391 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:27:19.437 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:27:19.438 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-17T07:27:19.438 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:19.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install librbd1 2024-09-17T07:27:20.136 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:53 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:27:20.154 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:20.154 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:20.154 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:27:20.154 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:27:20.154 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:20.154 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:27:20.242 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:27:20.289 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:27:20.290 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-17T07:27:20.290 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:20.362 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rados 2024-09-17T07:27:20.386 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-nbd 2024-09-17T07:27:20.970 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:53 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:27:20.982 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:56 ago on Tue 17 Sep 2024 07:25:24 AM UTC. 2024-09-17T07:27:21.062 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rados-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:27:21.108 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:27:21.109 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-17T07:27:21.110 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:21.137 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-17T07:27:21.138 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:27:21.138 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-09-17T07:27:21.138 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:27:21.138 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-17T07:27:21.138 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 170 k 2024-09-17T07:27:21.138 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:21.138 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-17T07:27:21.138 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-17T07:27:21.138 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-17T07:27:21.138 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:21.138 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 170 k 2024-09-17T07:27:21.139 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 490 k 2024-09-17T07:27:21.139 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-17T07:27:21.177 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rgw 2024-09-17T07:27:21.353 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-nbd-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 797 kB/s | 170 kB 00:00 2024-09-17T07:27:21.353 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:27:21.353 INFO:teuthology.orchestra.run.smithi192.stdout:Total 790 kB/s | 170 kB 00:00 2024-09-17T07:27:21.354 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-17T07:27:21.361 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-17T07:27:21.361 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-17T07:27:21.426 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-17T07:27:21.426 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-17T07:27:21.591 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-17T07:27:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:21.785 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:54 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:27:21.880 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rgw-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:27:21.926 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:27:21.927 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-17T07:27:21.927 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:22.003 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-cephfs 2024-09-17T07:27:22.127 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:22.388 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:22.388 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:22.389 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-17T07:27:22.389 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:27:22.389 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:22.389 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-17T07:27:22.607 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:55 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:27:22.618 DEBUG:teuthology.parallel:result is None 2024-09-17T07:27:22.699 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-cephfs-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:27:22.746 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:27:22.747 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-17T07:27:22.747 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:22.824 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install python3-rbd 2024-09-17T07:27:23.432 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:56 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:27:23.525 INFO:teuthology.orchestra.run.smithi045.stdout:Package python3-rbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 is already installed. 2024-09-17T07:27:23.572 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:27:23.573 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-09-17T07:27:23.573 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:23.644 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-fuse 2024-09-17T07:27:24.248 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:57 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:27:24.388 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-09-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 86 k 2024-09-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-17T07:27:24.389 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:24.390 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 86 k 2024-09-17T07:27:24.390 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 231 k 2024-09-17T07:27:24.390 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:27:24.588 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-fuse-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 431 kB/s | 86 kB 00:00 2024-09-17T07:27:24.589 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:27:24.589 INFO:teuthology.orchestra.run.smithi045.stdout:Total 428 kB/s | 86 kB 00:00 2024-09-17T07:27:24.589 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:27:24.597 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:27:24.597 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:27:24.665 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:27:24.665 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:27:24.817 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:27:24.915 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:25.333 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:25.593 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:25.593 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:25.593 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:27:25.593 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:27:25.594 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:25.594 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:25.903 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-mirror 2024-09-17T07:27:26.513 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:01:59 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:27:26.652 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:27:26.653 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repo Size 2024-09-17T07:27:26.653 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:26.653 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:27:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 3.2 M 2024-09-17T07:27:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:26.654 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:27:26.654 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:26.654 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-17T07:27:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:26.654 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 3.2 M 2024-09-17T07:27:26.654 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 14 M 2024-09-17T07:27:26.654 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:27:27.001 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-mirror-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 9.2 MB/s | 3.2 MB 00:00 2024-09-17T07:27:27.001 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:27:27.002 INFO:teuthology.orchestra.run.smithi045.stdout:Total 9.2 MB/s | 3.2 MB 00:00 2024-09-17T07:27:27.002 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:27:27.013 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:27:27.013 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:27:27.100 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:27:27.100 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:27:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:27:27.619 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:27.645 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:27.645 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T07:27:27.645 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T07:27:27.645 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T07:27:27.645 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T07:27:27.646 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:28.794 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:28.795 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:28.795 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:27:28.795 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:27:28.795 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:28.795 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:29.033 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install rbd-nbd 2024-09-17T07:27:29.639 INFO:teuthology.orchestra.run.smithi045.stdout:Last metadata expiration check: 0:02:02 ago on Tue 17 Sep 2024 07:25:27 AM UTC. 2024-09-17T07:27:29.779 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-17T07:27:29.780 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:29.780 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-09-17T07:27:29.780 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:29.781 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-17T07:27:29.781 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.3.0-4883.g6e8474a0.el9 ceph 170 k 2024-09-17T07:27:29.781 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:29.781 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-17T07:27:29.781 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-17T07:27:29.781 INFO:teuthology.orchestra.run.smithi045.stdout:Install 1 Package 2024-09-17T07:27:29.781 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:29.781 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 170 k 2024-09-17T07:27:29.781 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 490 k 2024-09-17T07:27:29.781 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-17T07:27:29.998 INFO:teuthology.orchestra.run.smithi045.stdout:rbd-nbd-19.3.0-4883.g6e8474a0.el9.x86_64.rpm 787 kB/s | 170 kB 00:00 2024-09-17T07:27:29.998 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-17T07:27:29.998 INFO:teuthology.orchestra.run.smithi045.stdout:Total 782 kB/s | 170 kB 00:00 2024-09-17T07:27:29.999 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-17T07:27:30.007 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-17T07:27:30.007 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-17T07:27:30.076 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-17T07:27:30.076 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-17T07:27:30.233 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-17T07:27:30.342 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:30.743 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 1/1 2024-09-17T07:27:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:31.009 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-17T07:27:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.3.0-4883.g6e8474a0.el9.x86_64 2024-09-17T07:27:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:31.009 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-17T07:27:31.245 DEBUG:teuthology.parallel:result is None 2024-09-17T07:27:31.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-17T07:27:31.390 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T07:27:31.412 INFO:teuthology.orchestra.run.smithi045.stdout:19.3.0-4883.g6e8474a0.el9 2024-09-17T07:27:31.413 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4883.g6e8474a0.el9 2024-09-17T07:27:31.413 INFO:teuthology.task.install:The correct ceph version 19.3.0-4883.g6e8474a0 is installed. 2024-09-17T07:27:31.416 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6e8474a02f2b86353394076ab80c6f50ea28e361 2024-09-17T07:27:31.552 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T07:27:31.578 INFO:teuthology.orchestra.run.smithi192.stdout:19.3.0-4883.g6e8474a0.el9 2024-09-17T07:27:31.578 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4883.g6e8474a0.el9 2024-09-17T07:27:31.579 INFO:teuthology.task.install:The correct ceph version 19.3.0-4883.g6e8474a0 is installed. 2024-09-17T07:27:31.580 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T07:27:31.581 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:27:31.581 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T07:27:31.607 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:27:31.607 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T07:27:31.648 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T07:27:31.649 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:27:31.649 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T07:27:31.674 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T07:27:31.740 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:27:31.740 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T07:27:31.768 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T07:27:31.836 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T07:27:31.836 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:27:31.836 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T07:27:31.862 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T07:27:31.928 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:27:31.929 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T07:27:31.955 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T07:27:32.021 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T07:27:32.022 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:27:32.022 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T07:27:32.047 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T07:27:32.113 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:27:32.113 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T07:27:32.139 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T07:27:32.205 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T07:27:32.287 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T07:27:32.288 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /var/log/ceph 2024-09-17T07:27:32.290 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /var/log/ceph 2024-09-17T07:27:32.316 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T07:27:32.316 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T07:27:32.356 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T07:27:32.382 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T07:27:32.382 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T07:27:32.422 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T07:27:32.457 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T07:27:32.458 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache max dirty': 0}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T07:27:32.458 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dis-2024-09-17_07:04:29-rbd-wip-dis-testing-distro-default-smithi/7908707', 'branch': 'wip-dis-testing', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writethrough}', 'email': 'idryomov@gmail.com', 'first_in_suite': False, 'job_id': '7908707', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'dis-2024-09-17_07:04:29-rbd-wip-dis-testing-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-dis-testing'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache max dirty': 0}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-dis-testing', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}}, 'owner': 'scheduled_dis@teuthology', 'priority': 70, 'repo': 'https://github.com/ceph/ceph-ci', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 2961, 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361', 'sleep_before_teardown': 0, 'subset': '66/100', 'suite': 'rbd', 'suite_branch': 'wip-dis-testing', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_6e8474a02f2b86353394076ab80c6f50ea28e361/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci', 'suite_sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361', 'targets': {'smithi045.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE8zmjdF4PdtILv+2zCeT/tUVrwhzBkkt3E2R4lNgccEXDhBLdxbyGqOroGqwXuVhymK6WH4UAzDcBbdHEel4is=', 'smithi192.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDk2D3YUPVg1wW2pfKgbPzFSIrhcVYe4tc7gPcjnfvUD2eOhWXII+Q6ZenBr8LnSilr7vqSxijAInFQe2xfgrOQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_07:04:29', 'tube': 'smithi', 'user': 'dis', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T07:27:32.458 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T07:27:32.488 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T07:27:32.504 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T07:27:32.544 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T07:27:32.569 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:27:32.569 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T07:27:32.624 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T07:27:32.624 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-09-17T07:27:32.680 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T07:27:32.680 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T07:27:32.680 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-17T07:27:32.681 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T07:27:32.681 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T07:27:32.681 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-17 07:27:27.926988429 +0000 2024-09-17T07:27:32.681 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-17 07:24:45.351988337 +0000 2024-09-17T07:27:32.681 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-17 07:24:45.351988337 +0000 2024-09-17T07:27:32.681 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-17 07:24:45.351988337 +0000 2024-09-17T07:27:32.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T07:27:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-17T07:27:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-17T07:27:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 9.961e-05 s, 5.1 MB/s 2024-09-17T07:27:32.746 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T07:27:32.804 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-09-17T07:27:32.860 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T07:27:32.860 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T07:27:32.861 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-17T07:27:32.861 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T07:27:32.861 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T07:27:32.861 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-17 07:27:27.927988415 +0000 2024-09-17T07:27:32.861 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-17 07:24:45.683983687 +0000 2024-09-17T07:27:32.861 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-17 07:24:45.683983687 +0000 2024-09-17T07:27:32.861 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-17 07:24:45.683983687 +0000 2024-09-17T07:27:32.861 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T07:27:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-17T07:27:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-17T07:27:32.925 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000103088 s, 5.0 MB/s 2024-09-17T07:27:32.926 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T07:27:32.983 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-09-17T07:27:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T07:27:33.039 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T07:27:33.039 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-17T07:27:33.039 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T07:27:33.039 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T07:27:33.040 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-17 07:27:27.928988402 +0000 2024-09-17T07:27:33.040 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-17 07:24:46.003979205 +0000 2024-09-17T07:27:33.040 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-17 07:24:46.003979205 +0000 2024-09-17T07:27:33.040 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-17 07:24:46.003979205 +0000 2024-09-17T07:27:33.040 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T07:27:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-17T07:27:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-17T07:27:33.104 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 9.5852e-05 s, 5.3 MB/s 2024-09-17T07:27:33.105 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T07:27:33.162 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-09-17T07:27:33.219 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T07:27:33.219 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T07:27:33.219 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-17T07:27:33.219 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T07:27:33.219 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T07:27:33.219 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-17 07:27:27.928988402 +0000 2024-09-17T07:27:33.219 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-17 07:24:46.329974638 +0000 2024-09-17T07:27:33.219 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-17 07:24:46.329974638 +0000 2024-09-17T07:27:33.219 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-17 07:24:46.329974638 +0000 2024-09-17T07:27:33.219 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T07:27:33.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-17T07:27:33.282 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-17T07:27:33.282 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000101334 s, 5.1 MB/s 2024-09-17T07:27:33.283 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T07:27:33.340 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T07:27:33.340 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:27:33.340 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T07:27:33.362 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T07:27:33.362 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-09-17T07:27:33.419 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T07:27:33.419 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T07:27:33.419 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-17T07:27:33.420 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T07:27:33.420 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T07:27:33.420 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-17 07:27:19.296268239 +0000 2024-09-17T07:27:33.420 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-17 07:24:45.365254831 +0000 2024-09-17T07:27:33.420 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-17 07:24:45.365254831 +0000 2024-09-17T07:27:33.420 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-17 07:24:45.365254831 +0000 2024-09-17T07:27:33.420 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T07:27:33.485 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-17T07:27:33.485 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-17T07:27:33.485 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000269266 s, 1.9 MB/s 2024-09-17T07:27:33.486 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T07:27:33.544 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-09-17T07:27:33.600 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T07:27:33.600 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T07:27:33.600 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-17T07:27:33.600 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T07:27:33.600 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T07:27:33.600 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-17 07:27:19.297268220 +0000 2024-09-17T07:27:33.600 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-17 07:24:45.785246440 +0000 2024-09-17T07:27:33.600 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-17 07:24:45.785246440 +0000 2024-09-17T07:27:33.600 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-17 07:24:45.785246440 +0000 2024-09-17T07:27:33.601 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T07:27:33.665 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-17T07:27:33.666 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-17T07:27:33.666 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000208714 s, 2.5 MB/s 2024-09-17T07:27:33.667 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T07:27:33.725 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-09-17T07:27:33.784 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T07:27:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T07:27:33.785 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-17T07:27:33.785 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T07:27:33.785 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T07:27:33.785 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-17 07:27:19.297268220 +0000 2024-09-17T07:27:33.785 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-17 07:24:46.096240227 +0000 2024-09-17T07:27:33.785 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-17 07:24:46.096240227 +0000 2024-09-17T07:27:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-17 07:24:46.096240227 +0000 2024-09-17T07:27:33.786 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T07:27:33.849 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-17T07:27:33.850 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-17T07:27:33.850 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.00022305 s, 2.3 MB/s 2024-09-17T07:27:33.851 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T07:27:33.908 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-09-17T07:27:33.964 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T07:27:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T07:27:33.965 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-17T07:27:33.965 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T07:27:33.965 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T07:27:33.965 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-17 07:27:19.298268200 +0000 2024-09-17T07:27:33.965 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-17 07:24:46.430233554 +0000 2024-09-17T07:27:33.965 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-17 07:24:46.430233554 +0000 2024-09-17T07:27:33.965 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-17 07:24:46.430233554 +0000 2024-09-17T07:27:33.965 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T07:27:34.033 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-17T07:27:34.033 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-17T07:27:34.033 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000223255 s, 2.3 MB/s 2024-09-17T07:27:34.034 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T07:27:34.092 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T07:27:34.092 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T07:27:34.092 INFO:tasks.ceph:Generating config... 2024-09-17T07:27:34.093 INFO:tasks.ceph:[client] rbd cache = True 2024-09-17T07:27:34.093 INFO:tasks.ceph:[client] rbd cache max dirty = 0 2024-09-17T07:27:34.094 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-17T07:27:34.094 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-17T07:27:34.094 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-17T07:27:34.094 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T07:27:34.094 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T07:27:34.094 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T07:27:34.094 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T07:27:34.094 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T07:27:34.094 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T07:27:34.094 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2024-09-17T07:27:34.094 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-17T07:27:34.094 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T07:27:34.094 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-17T07:27:34.095 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-17T07:27:34.095 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T07:27:34.095 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T07:27:34.095 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-17T07:27:34.095 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T07:27:34.095 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T07:27:34.095 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T07:27:34.095 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-17T07:27:34.095 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T07:27:34.095 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-17T07:27:34.095 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T07:27:34.095 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-17T07:27:34.095 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T07:27:34.096 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T07:27:34.152 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T07:27:34.154 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T07:27:34.201 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T07:27:34.228 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.45'), ('mon.c', '[v2:172.21.15.45:3301,v1:172.21.15.45:6790]'), ('mon.b', '172.21.15.192')] 2024-09-17T07:27:34.228 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.45,[v2:172.21.15.45:3301,v1:172.21.15.45:6790],172.21.15.192', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache max dirty': 0}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-17T07:27:34.229 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:27:34.229 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T07:27:34.285 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.45 --addv c '[v2:172.21.15.45:3301,v1:172.21.15.45:6790]' --add b 172.21.15.192 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T07:27:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T07:27:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: generated fsid b283e6d5-0f4a-4271-a443-15e695e35364 2024-09-17T07:27:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = quincy 2024-09-17T07:27:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 0 2024-09-17T07:27:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:fsid b283e6d5-0f4a-4271-a443-15e695e35364 2024-09-17T07:27:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:last_changed 2024-09-17T07:27:34.373043+0000 2024-09-17T07:27:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-09-17T07:27:34.373043+0000 2024-09-17T07:27:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:min_mon_release 17 (quincy) 2024-09-17T07:27:34.373 INFO:teuthology.orchestra.run.smithi045.stdout:election_strategy: 1 2024-09-17T07:27:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-17T07:27:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.b 2024-09-17T07:27:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:2: [v2:172.21.15.45:3301/0,v1:172.21.15.45:6790/0] mon.c 2024-09-17T07:27:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T07:27:34.375 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T07:27:34.391 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b283e6d5-0f4a-4271-a443-15e695e35364... 2024-09-17T07:27:34.392 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T07:27:34.434 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T07:27:34.476 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-09-17T07:27:34.477 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2024-09-17T07:27:34.477 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T07:27:34.477 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2024-09-17T07:27:34.477 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.477 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2024-09-17T07:27:34.477 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.477 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2024-09-17T07:27:34.477 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.477 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2024-09-17T07:27:34.477 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2024-09-17T07:27:34.477 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.477 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2024-09-17T07:27:34.478 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2024-09-17T07:27:34.478 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.478 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T07:27:34.478 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2024-09-17T07:27:34.478 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.478 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T07:27:34.478 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T07:27:34.478 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.478 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2024-09-17T07:27:34.478 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.478 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2024-09-17T07:27:34.478 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2024-09-17T07:27:34.478 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2024-09-17T07:27:34.478 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2024-09-17T07:27:34.479 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2024-09-17T07:27:34.479 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2024-09-17T07:27:34.479 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2024-09-17T07:27:34.479 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T07:27:34.479 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T07:27:34.479 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2024-09-17T07:27:34.479 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.479 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T07:27:34.479 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.479 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2024-09-17T07:27:34.479 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.479 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2024-09-17T07:27:34.479 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.479 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2024-09-17T07:27:34.480 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2024-09-17T07:27:34.480 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2024-09-17T07:27:34.480 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.45,[v2:172.21.15.45:3301,v1:172.21.15.45:6790],172.21.15.192" 2024-09-17T07:27:34.480 INFO:teuthology.orchestra.run.smithi045.stdout: mon client directed command retry = 5 2024-09-17T07:27:34.480 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on pool no app = False 2024-09-17T07:27:34.480 INFO:teuthology.orchestra.run.smithi045.stdout: ms inject socket failures = 5000 2024-09-17T07:27:34.480 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = b283e6d5-0f4a-4271-a443-15e695e35364 2024-09-17T07:27:34.480 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.480 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2024-09-17T07:27:34.480 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2024-09-17T07:27:34.480 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.480 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2024-09-17T07:27:34.480 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2024-09-17T07:27:34.481 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2024-09-17T07:27:34.481 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.481 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2024-09-17T07:27:34.481 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2024-09-17T07:27:34.481 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.481 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2024-09-17T07:27:34.481 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2024-09-17T07:27:34.481 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2024-09-17T07:27:34.481 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug trim objects = true 2024-09-17T07:27:34.481 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.481 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2024-09-17T07:27:34.481 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2024-09-17T07:27:34.481 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.482 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2024-09-17T07:27:34.482 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.482 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2024-09-17T07:27:34.482 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.482 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2024-09-17T07:27:34.482 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.482 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2024-09-17T07:27:34.482 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2024-09-17T07:27:34.482 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.482 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2024-09-17T07:27:34.482 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2024-09-17T07:27:34.482 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2024-09-17T07:27:34.482 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore compression algorithm = snappy 2024-09-17T07:27:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore compression mode = aggressive 2024-09-17T07:27:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2024-09-17T07:27:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 1/20 2024-09-17T07:27:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 1/20 2024-09-17T07:27:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-09-17T07:27:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2024-09-17T07:27:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 4/10 2024-09-17T07:27:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T07:27:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2024-09-17T07:27:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2024-09-17T07:27:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: osd blocked scrub grace period = 3600 2024-09-17T07:27:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2024-09-17T07:27:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: osd max markdown count = 1000 2024-09-17T07:27:34.483 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2024-09-17T07:27:34.484 INFO:teuthology.orchestra.run.smithi045.stdout: osd shutdown pgref assert = True 2024-09-17T07:27:34.484 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.484 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2024-09-17T07:27:34.484 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-09-17T07:27:34.484 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2024-09-17T07:27:34.484 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-09-17T07:27:34.484 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-09-17T07:27:34.484 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-09-17T07:27:34.484 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-09-17T07:27:34.484 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2024-09-17T07:27:34.484 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.484 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2024-09-17T07:27:34.484 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-09-17T07:27:34.484 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-09-17T07:27:34.485 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2024-09-17T07:27:34.485 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-09-17T07:27:34.485 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2024-09-17T07:27:34.485 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2024-09-17T07:27:34.485 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-09-17T07:27:34.485 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2024-09-17T07:27:34.485 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2024-09-17T07:27:34.485 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-09-17T07:27:34.485 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.485 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T07:27:34.485 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T07:27:34.485 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T07:27:34.485 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.485 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T07:27:34.486 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T07:27:34.486 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T07:27:34.486 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.486 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2024-09-17T07:27:34.486 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2024-09-17T07:27:34.486 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.486 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2024-09-17T07:27:34.486 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:27:34.486 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2024-09-17T07:27:34.486 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2024-09-17T07:27:34.486 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2024-09-17T07:27:34.486 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2024-09-17T07:27:34.486 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T07:27:34.487 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T07:27:34.487 INFO:teuthology.orchestra.run.smithi045.stdout: rbd cache = True 2024-09-17T07:27:34.487 INFO:teuthology.orchestra.run.smithi045.stdout: rbd cache max dirty = 0 2024-09-17T07:27:34.487 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2024-09-17T07:27:34.487 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2024-09-17T07:27:34.487 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2024-09-17T07:27:34.487 INFO:teuthology.orchestra.run.smithi192.stdout:[global] 2024-09-17T07:27:34.487 INFO:teuthology.orchestra.run.smithi192.stdout: chdir = "" 2024-09-17T07:27:34.487 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T07:27:34.487 INFO:teuthology.orchestra.run.smithi192.stdout: auth supported = cephx 2024-09-17T07:27:34.488 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.488 INFO:teuthology.orchestra.run.smithi192.stdout: filestore xattr use omap = true 2024-09-17T07:27:34.488 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.488 INFO:teuthology.orchestra.run.smithi192.stdout: mon clock drift allowed = 1.000 2024-09-17T07:27:34.488 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.488 INFO:teuthology.orchestra.run.smithi192.stdout: osd crush chooseleaf type = 0 2024-09-17T07:27:34.488 INFO:teuthology.orchestra.run.smithi192.stdout: auth debug = true 2024-09-17T07:27:34.488 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.488 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on old message = true 2024-09-17T07:27:34.489 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on bug = true 2024-09-17T07:27:34.489 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.489 INFO:teuthology.orchestra.run.smithi192.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T07:27:34.489 INFO:teuthology.orchestra.run.smithi192.stdout: mon pg warn max object skew = 0 2024-09-17T07:27:34.489 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.489 INFO:teuthology.orchestra.run.smithi192.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T07:27:34.489 INFO:teuthology.orchestra.run.smithi192.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T07:27:34.489 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.489 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default size = 2 2024-09-17T07:27:34.489 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.489 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow primary affinity = true 2024-09-17T07:27:34.489 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow pg remap = true 2024-09-17T07:27:34.489 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on legacy crush tunables = false 2024-09-17T07:27:34.489 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on crush straw calc version zero = false 2024-09-17T07:27:34.490 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on no sortbitwise = false 2024-09-17T07:27:34.490 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on osd down out interval zero = false 2024-09-17T07:27:34.490 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on too few osds = false 2024-09-17T07:27:34.490 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T07:27:34.490 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T07:27:34.490 INFO:teuthology.orchestra.run.smithi192.stdout: mon_allow_pool_size_one = true 2024-09-17T07:27:34.490 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.490 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T07:27:34.490 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.490 INFO:teuthology.orchestra.run.smithi192.stdout: osd default data pool replay window = 5 2024-09-17T07:27:34.490 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.490 INFO:teuthology.orchestra.run.smithi192.stdout: mon allow pool delete = true 2024-09-17T07:27:34.490 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.490 INFO:teuthology.orchestra.run.smithi192.stdout: mon cluster log file level = debug 2024-09-17T07:27:34.490 INFO:teuthology.orchestra.run.smithi192.stdout: debug asserts on shutdown = true 2024-09-17T07:27:34.491 INFO:teuthology.orchestra.run.smithi192.stdout: mon health detail to clog = false 2024-09-17T07:27:34.491 INFO:teuthology.orchestra.run.smithi192.stdout: mon host = "172.21.15.45,[v2:172.21.15.45:3301,v1:172.21.15.45:6790],172.21.15.192" 2024-09-17T07:27:34.491 INFO:teuthology.orchestra.run.smithi192.stdout: mon client directed command retry = 5 2024-09-17T07:27:34.491 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on pool no app = False 2024-09-17T07:27:34.491 INFO:teuthology.orchestra.run.smithi192.stdout: ms inject socket failures = 5000 2024-09-17T07:27:34.491 INFO:teuthology.orchestra.run.smithi192.stdout: fsid = b283e6d5-0f4a-4271-a443-15e695e35364 2024-09-17T07:27:34.491 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.491 INFO:teuthology.orchestra.run.smithi192.stdout:[osd] 2024-09-17T07:27:34.491 INFO:teuthology.orchestra.run.smithi192.stdout: osd journal size = 100 2024-09-17T07:27:34.491 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.491 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub load threshold = 5.0 2024-09-17T07:27:34.491 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub max interval = 600 2024-09-17T07:27:34.491 INFO:teuthology.orchestra.run.smithi192.stdout: osd mclock profile = high_recovery_ops 2024-09-17T07:27:34.492 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.492 INFO:teuthology.orchestra.run.smithi192.stdout: osd recover clone overlap = true 2024-09-17T07:27:34.492 INFO:teuthology.orchestra.run.smithi192.stdout: osd recovery max chunk = 1048576 2024-09-17T07:27:34.492 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.492 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug shutdown = true 2024-09-17T07:27:34.492 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug op order = true 2024-09-17T07:27:34.492 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify stray on activate = true 2024-09-17T07:27:34.492 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug trim objects = true 2024-09-17T07:27:34.492 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.492 INFO:teuthology.orchestra.run.smithi192.stdout: osd open classes on start = true 2024-09-17T07:27:34.492 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug pg log writeout = true 2024-09-17T07:27:34.492 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.492 INFO:teuthology.orchestra.run.smithi192.stdout: osd deep scrub update digest min age = 30 2024-09-17T07:27:34.492 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.493 INFO:teuthology.orchestra.run.smithi192.stdout: osd map max advance = 10 2024-09-17T07:27:34.493 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.493 INFO:teuthology.orchestra.run.smithi192.stdout: journal zero on create = true 2024-09-17T07:27:34.493 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.493 INFO:teuthology.orchestra.run.smithi192.stdout: filestore ondisk finisher threads = 3 2024-09-17T07:27:34.493 INFO:teuthology.orchestra.run.smithi192.stdout: filestore apply finisher threads = 3 2024-09-17T07:27:34.493 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.493 INFO:teuthology.orchestra.run.smithi192.stdout: bdev debug aio = true 2024-09-17T07:27:34.493 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug misdirected ops = true 2024-09-17T07:27:34.493 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore block size = 96636764160 2024-09-17T07:27:34.493 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore compression algorithm = snappy 2024-09-17T07:27:34.493 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore compression mode = aggressive 2024-09-17T07:27:34.493 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore fsck on mount = True 2024-09-17T07:27:34.493 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluefs = 1/20 2024-09-17T07:27:34.494 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluestore = 1/20 2024-09-17T07:27:34.494 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-09-17T07:27:34.494 INFO:teuthology.orchestra.run.smithi192.stdout: debug osd = 20 2024-09-17T07:27:34.494 INFO:teuthology.orchestra.run.smithi192.stdout: debug rocksdb = 4/10 2024-09-17T07:27:34.494 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T07:27:34.494 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd full ratio = 0.9 2024-09-17T07:27:34.494 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd nearfull ratio = 0.8 2024-09-17T07:27:34.494 INFO:teuthology.orchestra.run.smithi192.stdout: osd blocked scrub grace period = 3600 2024-09-17T07:27:34.494 INFO:teuthology.orchestra.run.smithi192.stdout: osd failsafe full ratio = 0.95 2024-09-17T07:27:34.494 INFO:teuthology.orchestra.run.smithi192.stdout: osd max markdown count = 1000 2024-09-17T07:27:34.494 INFO:teuthology.orchestra.run.smithi192.stdout: osd objectstore = bluestore 2024-09-17T07:27:34.494 INFO:teuthology.orchestra.run.smithi192.stdout: osd shutdown pgref assert = True 2024-09-17T07:27:34.494 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.494 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr] 2024-09-17T07:27:34.495 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-09-17T07:27:34.495 INFO:teuthology.orchestra.run.smithi192.stdout: debug mgr = 20 2024-09-17T07:27:34.495 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2024-09-17T07:27:34.495 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2024-09-17T07:27:34.495 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2024-09-17T07:27:34.495 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2024-09-17T07:27:34.495 INFO:teuthology.orchestra.run.smithi192.stdout: mgr/telemetry/nag = false 2024-09-17T07:27:34.495 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.495 INFO:teuthology.orchestra.run.smithi192.stdout:[mon] 2024-09-17T07:27:34.495 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-09-17T07:27:34.495 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2024-09-17T07:27:34.495 INFO:teuthology.orchestra.run.smithi192.stdout: debug paxos = 20 2024-09-17T07:27:34.495 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2024-09-17T07:27:34.495 INFO:teuthology.orchestra.run.smithi192.stdout: mon data avail warn = 5 2024-09-17T07:27:34.496 INFO:teuthology.orchestra.run.smithi192.stdout: mon mgr mkfs grace = 240 2024-09-17T07:27:34.496 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2024-09-17T07:27:34.496 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd reporter subtree level = osd 2024-09-17T07:27:34.496 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd prime pg temp = true 2024-09-17T07:27:34.496 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2024-09-17T07:27:34.496 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.496 INFO:teuthology.orchestra.run.smithi192.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T07:27:34.496 INFO:teuthology.orchestra.run.smithi192.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T07:27:34.496 INFO:teuthology.orchestra.run.smithi192.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T07:27:34.496 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.496 INFO:teuthology.orchestra.run.smithi192.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T07:27:34.496 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T07:27:34.496 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T07:27:34.496 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.497 INFO:teuthology.orchestra.run.smithi192.stdout: # 1m isn't quite enough 2024-09-17T07:27:34.497 INFO:teuthology.orchestra.run.smithi192.stdout: mon_down_mkfs_grace = 2m 2024-09-17T07:27:34.497 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.497 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_filestore_osds = false 2024-09-17T07:27:34.497 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-17T07:27:34.497 INFO:teuthology.orchestra.run.smithi192.stdout:[client] 2024-09-17T07:27:34.497 INFO:teuthology.orchestra.run.smithi192.stdout: rgw cache enabled = true 2024-09-17T07:27:34.497 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable ops log = true 2024-09-17T07:27:34.497 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable usage log = true 2024-09-17T07:27:34.497 INFO:teuthology.orchestra.run.smithi192.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T07:27:34.497 INFO:teuthology.orchestra.run.smithi192.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T07:27:34.497 INFO:teuthology.orchestra.run.smithi192.stdout: rbd cache = True 2024-09-17T07:27:34.497 INFO:teuthology.orchestra.run.smithi192.stdout: rbd cache max dirty = 0 2024-09-17T07:27:34.497 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.a] 2024-09-17T07:27:34.497 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.c] 2024-09-17T07:27:34.498 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.b] 2024-09-17T07:27:34.499 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T07:27:34.499 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T07:27:34.596 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T07:27:34.596 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:27:34.596 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T07:27:34.612 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:27:34.612 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T07:27:34.668 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi045.front.sepia.ceph.com 2024-09-17T07:27:34.668 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:27:34.668 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T07:27:34.668 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T07:27:34.744 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:27:34.744 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T07:27:34.800 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi192.front.sepia.ceph.com 2024-09-17T07:27:34.800 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:27:34.800 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T07:27:34.800 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T07:27:34.837 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:27:34.837 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T07:27:34.893 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T07:27:34.893 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T07:27:34.894 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T07:27:34.947 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T07:27:34.949 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T07:27:35.034 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T07:27:35.036 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T07:27:35.036 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T07:27:35.036 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T07:27:35.079 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T07:27:35.092 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T07:27:35.134 INFO:teuthology.orchestra.run.smithi192.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-17T07:27:35.146 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T07:27:35.146 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T07:27:35.146 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T07:27:35.172 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T07:27:35.172 INFO:tasks.ceph:role: osd.0 2024-09-17T07:27:35.172 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-17T07:27:35.172 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T07:27:35.238 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T07:27:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T07:27:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T07:27:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T07:27:35.238 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T07:27:35.238 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-09-17T07:27:35.239 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T07:27:35.239 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T07:27:35.239 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T07:27:35.239 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T07:27:35.271 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-09-17T07:27:35.272 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-09-17T07:27:35.273 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T07:27:35.343 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T07:27:35.414 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T07:27:35.480 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T07:27:35.480 INFO:tasks.ceph:role: osd.1 2024-09-17T07:27:35.480 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-17T07:27:35.480 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T07:27:35.545 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T07:27:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T07:27:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T07:27:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T07:27:35.545 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T07:27:35.545 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-09-17T07:27:35.546 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T07:27:35.546 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T07:27:35.546 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T07:27:35.546 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T07:27:35.578 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-09-17T07:27:35.580 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-09-17T07:27:35.580 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T07:27:35.650 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T07:27:35.720 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T07:27:35.786 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T07:27:35.786 INFO:tasks.ceph:role: osd.2 2024-09-17T07:27:35.786 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-17T07:27:35.787 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T07:27:35.852 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T07:27:35.852 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T07:27:35.853 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T07:27:35.853 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T07:27:35.854 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T07:27:35.854 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-09-17T07:27:35.854 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T07:27:35.854 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T07:27:35.854 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T07:27:35.854 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T07:27:35.884 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-09-17T07:27:35.886 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-09-17T07:27:35.887 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T07:27:35.956 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T07:27:36.026 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T07:27:36.093 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T07:27:36.094 INFO:tasks.ceph:role: osd.3 2024-09-17T07:27:36.094 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-17T07:27:36.094 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T07:27:36.159 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T07:27:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T07:27:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T07:27:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T07:27:36.159 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T07:27:36.159 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-09-17T07:27:36.159 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T07:27:36.159 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T07:27:36.160 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T07:27:36.160 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T07:27:36.192 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-09-17T07:27:36.194 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-09-17T07:27:36.194 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-17T07:27:36.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T07:27:36.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T07:27:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:36.426+0000 7fb79be0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T07:27:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:36.426+0000 7fb79be0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T07:27:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:36.426+0000 7fb79be0f640 -1 bdev(0x56245cbcb800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T07:27:36.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:36.426+0000 7fb79be0f640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T07:27:39.354 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T07:27:39.385 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T07:27:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:39.477+0000 7f37339a8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T07:27:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:39.477+0000 7f37339a8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T07:27:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:39.477+0000 7f37339a8640 -1 bdev(0x5588300b5800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T07:27:39.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:39.477+0000 7f37339a8640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T07:27:42.634 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T07:27:42.665 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T07:27:42.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:42.757+0000 7f04d8f02640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T07:27:42.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:42.757+0000 7f04d8f02640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T07:27:42.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:42.757+0000 7f04d8f02640 -1 bdev(0x558ec8939800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T07:27:42.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:42.757+0000 7f04d8f02640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T07:27:45.922 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T07:27:45.954 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T07:27:46.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:46.045+0000 7fe2e4c5e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T07:27:46.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:46.045+0000 7fe2e4c5e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T07:27:46.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:46.045+0000 7fe2e4c5e640 -1 bdev(0x55d6f95a5800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T07:27:46.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:27:46.045+0000 7fe2e4c5e640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T07:27:49.202 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T07:27:49.232 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T07:27:49.262 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T07:27:49.263 INFO:tasks.ceph:role: osd.4 2024-09-17T07:27:49.263 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-17T07:27:49.263 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T07:27:49.339 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T07:27:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T07:27:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T07:27:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T07:27:49.340 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T07:27:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-09-17T07:27:49.340 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T07:27:49.340 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T07:27:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T07:27:49.340 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T07:27:49.838 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-09-17T07:27:49.840 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-09-17T07:27:49.840 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-17T07:27:49.984 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T07:27:50.017 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T07:27:50.084 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T07:27:50.084 INFO:tasks.ceph:role: osd.5 2024-09-17T07:27:50.085 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-17T07:27:50.085 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T07:27:50.159 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T07:27:50.159 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T07:27:50.159 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T07:27:50.159 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T07:27:50.159 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T07:27:50.159 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-09-17T07:27:50.159 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T07:27:50.159 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T07:27:50.160 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T07:27:50.160 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T07:27:50.627 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-09-17T07:27:50.629 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-09-17T07:27:50.629 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-17T07:27:50.722 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T07:27:50.752 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-17T07:27:50.819 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T07:27:50.819 INFO:tasks.ceph:role: osd.6 2024-09-17T07:27:50.819 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-17T07:27:50.819 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T07:27:50.893 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T07:27:50.893 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T07:27:50.893 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T07:27:50.893 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T07:27:50.893 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T07:27:50.893 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-09-17T07:27:50.893 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T07:27:50.893 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T07:27:50.893 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T07:27:50.893 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T07:27:51.376 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-09-17T07:27:51.378 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-09-17T07:27:51.379 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-17T07:27:51.517 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-17T07:27:51.550 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-17T07:27:51.618 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T07:27:51.618 INFO:tasks.ceph:role: osd.7 2024-09-17T07:27:51.618 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-17T07:27:51.619 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T07:27:51.693 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T07:27:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T07:27:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T07:27:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T07:27:51.693 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T07:27:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-09-17T07:27:51.693 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T07:27:51.693 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T07:27:51.693 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T07:27:51.693 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T07:27:52.208 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-09-17T07:27:52.210 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-09-17T07:27:52.211 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-17T07:27:52.322 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-17T07:27:52.354 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T07:27:52.460 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:27:52.459+0000 7f93af341640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T07:27:52.460 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:27:52.459+0000 7f93af341640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T07:27:52.460 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:27:52.459+0000 7f93af341640 -1 bdev(0x55e62b455800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T07:27:52.460 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:27:52.459+0000 7f93af341640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T07:27:55.375 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T07:27:55.405 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T07:27:55.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:27:55.495+0000 7fb93ca1e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T07:27:55.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:27:55.495+0000 7fb93ca1e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T07:27:55.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:27:55.495+0000 7fb93ca1e640 -1 bdev(0x55e3c2e33800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T07:27:55.496 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:27:55.495+0000 7fb93ca1e640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T07:27:58.420 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T07:27:58.450 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T07:27:58.539 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:27:58.538+0000 7f6d44c16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-17T07:27:58.539 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:27:58.538+0000 7f6d44c16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-17T07:27:58.539 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:27:58.539+0000 7f6d44c16640 -1 bdev(0x55fcae367800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-17T07:27:58.540 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:27:58.539+0000 7f6d44c16640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-17T07:28:01.938 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T07:28:01.969 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T07:28:02.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:28:02.058+0000 7f9dcad58640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-17T07:28:02.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:28:02.058+0000 7f9dcad58640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-17T07:28:02.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:28:02.058+0000 7f9dcad58640 -1 bdev(0x560d80b4d800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-17T07:28:02.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-17T07:28:02.058+0000 7f9dcad58640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-17T07:28:04.971 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-17T07:28:04.999 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T07:28:05.000 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:28:05.000 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-17T07:28:05.031 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:28:05.031 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T07:28:05.102 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:28:05.102 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T07:28:05.173 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:28:05.173 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T07:28:05.243 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:28:05.244 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T07:28:05.313 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:28:05.313 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T07:28:05.343 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:28:05.343 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T07:28:05.412 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:28:05.412 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T07:28:05.479 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:28:05.479 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-17T07:28:05.546 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:28:05.546 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-17T07:28:05.612 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:28:05.613 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T07:28:05.632 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:28:05.632 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-17T07:28:05.668 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T07:28:05.669 DEBUG:teuthology.orchestra.run.smithi045:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T07:28:05.675 DEBUG:teuthology.orchestra.run.smithi192:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T07:28:05.712 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.y] 2024-09-17T07:28:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBmL+lmGsl4OBAAkm1z3Z4Hr5mZCnGnG2Xf3g== 2024-09-17T07:28:05.712 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.0] 2024-09-17T07:28:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBoL+lmO9SEGRAAiLJiiO3oLHa10yXyMrIdgw== 2024-09-17T07:28:05.712 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.1] 2024-09-17T07:28:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBrL+lmN2eQHBAAz4K6SiybmuOw0jr+xuj8PA== 2024-09-17T07:28:05.712 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.2] 2024-09-17T07:28:05.712 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBuL+lmH/I9LRAAddSyfm/QNf40SYXWUNudFg== 2024-09-17T07:28:05.712 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.3] 2024-09-17T07:28:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQByL+lm5w7RAhAA08qAunvgXlk0woHFbgVjMQ== 2024-09-17T07:28:05.713 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.x] 2024-09-17T07:28:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBnL+lmL2kJAhAAvJbqSIw8ZubdFFdispfm2g== 2024-09-17T07:28:05.713 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.4] 2024-09-17T07:28:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQB4L+lmZClsGxAAzKuYPR+c4w5HMY7Dnqt+tg== 2024-09-17T07:28:05.713 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.5] 2024-09-17T07:28:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQB7L+lmpQGQHRAAPSArYMfYhME8v6l+/+fNVw== 2024-09-17T07:28:05.713 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.6] 2024-09-17T07:28:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQB+L+lmo/8mIBAA1hNv3x00ctTNl4CHWTxkPA== 2024-09-17T07:28:05.713 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.7] 2024-09-17T07:28:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQCCL+lms2WCAxAA4lGXsROtgNIF7NlMMCyGlg== 2024-09-17T07:28:05.713 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-09-17T07:28:05.713 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBnL+lmnxG8BBAAiuzZYB0Ly01/LcQ4wiFhQQ== 2024-09-17T07:28:05.714 INFO:teuthology.orchestra.run.smithi045.stdout:[client.1] 2024-09-17T07:28:05.714 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBnL+lmL4f/BxAAhHj452eQorQJvNczMfz7LQ== 2024-09-17T07:28:05.732 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.y] 2024-09-17T07:28:05.733 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBmL+lmGsl4OBAAkm1z3Z4Hr5mZCnGnG2Xf3g== 2024-09-17T07:28:05.733 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.0] 2024-09-17T07:28:05.733 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBoL+lmO9SEGRAAiLJiiO3oLHa10yXyMrIdgw== 2024-09-17T07:28:05.733 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.1] 2024-09-17T07:28:05.733 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBrL+lmN2eQHBAAz4K6SiybmuOw0jr+xuj8PA== 2024-09-17T07:28:05.733 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.2] 2024-09-17T07:28:05.733 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBuL+lmH/I9LRAAddSyfm/QNf40SYXWUNudFg== 2024-09-17T07:28:05.733 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.3] 2024-09-17T07:28:05.733 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQByL+lm5w7RAhAA08qAunvgXlk0woHFbgVjMQ== 2024-09-17T07:28:05.733 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.x] 2024-09-17T07:28:05.733 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBnL+lmL2kJAhAAvJbqSIw8ZubdFFdispfm2g== 2024-09-17T07:28:05.734 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.4] 2024-09-17T07:28:05.734 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQB4L+lmZClsGxAAzKuYPR+c4w5HMY7Dnqt+tg== 2024-09-17T07:28:05.734 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.5] 2024-09-17T07:28:05.734 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQB7L+lmpQGQHRAAPSArYMfYhME8v6l+/+fNVw== 2024-09-17T07:28:05.734 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.6] 2024-09-17T07:28:05.734 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQB+L+lmo/8mIBAA1hNv3x00ctTNl4CHWTxkPA== 2024-09-17T07:28:05.734 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.7] 2024-09-17T07:28:05.734 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCCL+lms2WCAxAA4lGXsROtgNIF7NlMMCyGlg== 2024-09-17T07:28:05.734 INFO:teuthology.orchestra.run.smithi192.stdout:[client.0] 2024-09-17T07:28:05.734 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBnL+lmnxG8BBAAiuzZYB0Ly01/LcQ4wiFhQQ== 2024-09-17T07:28:05.734 INFO:teuthology.orchestra.run.smithi192.stdout:[client.1] 2024-09-17T07:28:05.734 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQBnL+lmL4f/BxAAhHj452eQorQJvNczMfz7LQ== 2024-09-17T07:28:05.735 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T07:28:05.757 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T07:28:05.826 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:05.829 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:05.882 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:05.885 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:05.936 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:05.940 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:05.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:05.994 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:06.046 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T07:28:06.048 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T07:28:06.099 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:06.101 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:06.152 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:06.155 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:06.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:06.208 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:06.260 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:06.262 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T07:28:06.313 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T07:28:06.315 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T07:28:06.369 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T07:28:06.371 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T07:28:06.426 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T07:28:06.426 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T07:28:06.451 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T07:28:06.549 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T07:28:06.575 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T07:28:06.641 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T07:28:06.738 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T07:28:06.764 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T07:28:06.790 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T07:28:06.887 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T07:28:06.912 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T07:28:06.914 DEBUG:teuthology.orchestra.run.smithi192:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T07:28:06.967 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T07:28:06.968 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T07:28:06.968 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T07:28:06.970 INFO:tasks.ceph.mon.a:Started 2024-09-17T07:28:06.971 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T07:28:06.971 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T07:28:06.973 INFO:tasks.ceph.mon.c:Started 2024-09-17T07:28:06.973 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T07:28:06.973 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T07:28:07.010 INFO:tasks.ceph.mon.b:Started 2024-09-17T07:28:07.010 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T07:28:07.010 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-17T07:28:07.010 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-17T07:28:07.013 INFO:tasks.ceph.mgr.y:Started 2024-09-17T07:28:07.013 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T07:28:07.013 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T07:28:07.015 INFO:tasks.ceph.mgr.x:Started 2024-09-17T07:28:07.015 DEBUG:tasks.ceph:set 0 configs 2024-09-17T07:28:07.015 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T07:28:07.015 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T07:28:12.447 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2024-09-17T07:28:12.458 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T07:28:12.458 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T07:28:12.459 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:28:12.459 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T07:28:12.485 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:28:12.485 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T07:28:12.552 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:28:12.552 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T07:28:12.617 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-17T07:28:12.617 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T07:28:12.683 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:28:12.683 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T07:28:12.714 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:28:12.714 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T07:28:12.782 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:28:12.782 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-17T07:28:12.848 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-17T07:28:12.848 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-17T07:28:12.914 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 7794e6a8-ebf4-4b7c-bf3d-63cd015234ef 0 2024-09-17T07:28:13.137 INFO:teuthology.orchestra.run.smithi192.stdout:0 2024-09-17T07:28:13.148 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5 1 2024-09-17T07:28:13.331 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-09-17T07:28:13.342 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 195a9db1-568d-459a-bb50-2a1bb2445f53 2 2024-09-17T07:28:13.529 INFO:teuthology.orchestra.run.smithi192.stdout:2 2024-09-17T07:28:13.540 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new b3fda98b-a10a-4354-905e-1b37a5124b98 3 2024-09-17T07:28:13.725 INFO:teuthology.orchestra.run.smithi192.stdout:3 2024-09-17T07:28:13.736 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 6ab67b48-2d83-4e68-8898-302a1adc0a52 4 2024-09-17T07:28:13.919 INFO:teuthology.orchestra.run.smithi192.stdout:4 2024-09-17T07:28:13.931 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new ae3c787c-63d2-4e88-8f99-6d4f19837a3a 5 2024-09-17T07:28:14.110 INFO:teuthology.orchestra.run.smithi192.stdout:5 2024-09-17T07:28:14.121 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new f792f18d-ad9e-49a8-b194-24522398fb25 6 2024-09-17T07:28:14.299 INFO:teuthology.orchestra.run.smithi192.stdout:6 2024-09-17T07:28:14.311 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 4a589d43-b0b9-42de-989c-46928d693c24 7 2024-09-17T07:28:14.489 INFO:teuthology.orchestra.run.smithi192.stdout:7 2024-09-17T07:28:14.501 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T07:28:14.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T07:28:14.503 INFO:tasks.ceph.osd.0:Started 2024-09-17T07:28:14.504 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T07:28:14.504 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T07:28:14.506 INFO:tasks.ceph.osd.1:Started 2024-09-17T07:28:14.506 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T07:28:14.506 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T07:28:14.508 INFO:tasks.ceph.osd.2:Started 2024-09-17T07:28:14.508 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T07:28:14.508 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T07:28:14.510 INFO:tasks.ceph.osd.3:Started 2024-09-17T07:28:14.510 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T07:28:14.510 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T07:28:14.512 INFO:tasks.ceph.osd.4:Started 2024-09-17T07:28:14.513 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T07:28:14.513 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T07:28:14.515 INFO:tasks.ceph.osd.5:Started 2024-09-17T07:28:14.515 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-17T07:28:14.515 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-17T07:28:14.517 INFO:tasks.ceph.osd.6:Started 2024-09-17T07:28:14.517 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-17T07:28:14.517 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-17T07:28:14.519 INFO:tasks.ceph.osd.7:Started 2024-09-17T07:28:14.520 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T07:28:14.715 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:28:14.715 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"fsid":"b283e6d5-0f4a-4271-a443-15e695e35364","created":"2024-09-17T07:28:12.276015+0000","modified":"2024-09-17T07:28:14.487756+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T07:28:14.487756+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7794e6a8-ebf4-4b7c-bf3d-63cd015234ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"195a9db1-568d-459a-bb50-2a1bb2445f53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b3fda98b-a10a-4354-905e-1b37a5124b98","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6ab67b48-2d83-4e68-8898-302a1adc0a52","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ae3c787c-63d2-4e88-8f99-6d4f19837a3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f792f18d-ad9e-49a8-b194-24522398fb25","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4a589d43-b0b9-42de-989c-46928d693c24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T07:28:14.726 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T07:28:14.726 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T07:28:15.533 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T07:28:15.697 INFO:teuthology.misc.health.smithi045.stdout: 2024-09-17T07:28:15.697 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":10,"fsid":"b283e6d5-0f4a-4271-a443-15e695e35364","created":"2024-09-17T07:28:12.276015+0000","modified":"2024-09-17T07:28:14.487756+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T07:28:14.487756+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7794e6a8-ebf4-4b7c-bf3d-63cd015234ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"195a9db1-568d-459a-bb50-2a1bb2445f53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b3fda98b-a10a-4354-905e-1b37a5124b98","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6ab67b48-2d83-4e68-8898-302a1adc0a52","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ae3c787c-63d2-4e88-8f99-6d4f19837a3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f792f18d-ad9e-49a8-b194-24522398fb25","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4a589d43-b0b9-42de-989c-46928d693c24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T07:28:15.707 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T07:28:15.735 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:15.733+0000 7f8ea3dc5640 -1 Falling back to public interface 2024-09-17T07:28:15.739 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:15.738+0000 7f2995a1e640 -1 Falling back to public interface 2024-09-17T07:28:15.759 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:15.758+0000 7fdb771dd640 -1 Falling back to public interface 2024-09-17T07:28:15.959 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:15.958+0000 7f9a07e1c640 -1 Falling back to public interface 2024-09-17T07:28:15.973 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:15.972+0000 7f1480154640 -1 Falling back to public interface 2024-09-17T07:28:15.981 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:15.980+0000 7fb4f82ae640 -1 Falling back to public interface 2024-09-17T07:28:16.242 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:16.242+0000 7f29ba1ae640 -1 Falling back to public interface 2024-09-17T07:28:16.495 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:16.494+0000 7fac1121e640 -1 Falling back to public interface 2024-09-17T07:28:17.756 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-17T07:28:17.755+0000 7f9bb64ca640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T07:28:17.757 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-17T07:28:17.757+0000 7f9bb64ca640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T07:28:19.493 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:19.493+0000 7f9a07e1c640 -1 osd.3 0 log_to_monitors true 2024-09-17T07:28:19.546 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:19.546+0000 7f8ea3dc5640 -1 osd.0 0 log_to_monitors true 2024-09-17T07:28:19.547 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:19.546+0000 7fb4f82ae640 -1 osd.5 0 log_to_monitors true 2024-09-17T07:28:19.760 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:19.759+0000 7f1480154640 -1 osd.6 0 log_to_monitors true 2024-09-17T07:28:19.817 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:19.816+0000 7f2995a1e640 -1 osd.1 0 log_to_monitors true 2024-09-17T07:28:19.857 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:19.857+0000 7fdb771dd640 -1 osd.7 0 log_to_monitors true 2024-09-17T07:28:20.054 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:20.053+0000 7f29ba1ae640 -1 osd.2 0 log_to_monitors true 2024-09-17T07:28:20.326 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:20.325+0000 7fac1121e640 -1 osd.4 0 log_to_monitors true 2024-09-17T07:28:22.291 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:22.291+0000 7fb4f423d640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T07:28:22.305 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:22.304+0000 7fb4ec05c640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T07:28:22.318 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:22.317+0000 7f147c0e3640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T07:28:22.327 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:22.327+0000 7f1473f02640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T07:28:22.339 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:22.338+0000 7fdb7316c640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T07:28:22.346 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:22.346+0000 7fdb6af8b640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T07:28:22.457 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:22.457+0000 7f9a037a1640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T07:28:22.465 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:22.464+0000 7f299145e640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T07:28:22.474 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:22.473+0000 7f29b613d640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T07:28:22.475 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:22.475+0000 7f99fb5c0640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T07:28:22.478 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:22.478+0000 7f298927d640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T07:28:22.490 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:22.490+0000 7f29adf5c640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T07:28:22.496 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:22.495+0000 7f8e9fd54640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T07:28:22.512 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:22.512+0000 7f8e97b73640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T07:28:22.515 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T07:28:22.805 INFO:teuthology.misc.health.smithi045.stdout: 2024-09-17T07:28:22.805 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":13,"fsid":"b283e6d5-0f4a-4271-a443-15e695e35364","created":"2024-09-17T07:28:12.276015+0000","modified":"2024-09-17T07:28:22.298884+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T07:28:14.487756+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7794e6a8-ebf4-4b7c-bf3d-63cd015234ef","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"195a9db1-568d-459a-bb50-2a1bb2445f53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b3fda98b-a10a-4354-905e-1b37a5124b98","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"6ab67b48-2d83-4e68-8898-302a1adc0a52","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ae3c787c-63d2-4e88-8f99-6d4f19837a3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f792f18d-ad9e-49a8-b194-24522398fb25","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4a589d43-b0b9-42de-989c-46928d693c24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T07:28:22.816 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T07:28:23.008 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:23.007+0000 7fac0d472640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T07:28:23.021 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:23.021+0000 7fac04a7d640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T07:28:29.624 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T07:28:29.933 INFO:teuthology.misc.health.smithi045.stdout: 2024-09-17T07:28:29.933 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":18,"fsid":"b283e6d5-0f4a-4271-a443-15e695e35364","created":"2024-09-17T07:28:12.276015+0000","modified":"2024-09-17T07:28:27.331102+0000","last_up_change":"2024-09-17T07:28:23.300710+0000","last_in_change":"2024-09-17T07:28:14.487756+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T07:28:24.767696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7794e6a8-ebf4-4b7c-bf3d-63cd015234ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6801","nonce":2039818160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6803","nonce":2039818160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6807","nonce":2039818160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6805","nonce":2039818160}]},"public_addr":"172.21.15.45:6801/2039818160","cluster_addr":"172.21.15.45:6803/2039818160","heartbeat_back_addr":"172.21.15.45:6807/2039818160","heartbeat_front_addr":"172.21.15.45:6805/2039818160","state":["exists","up"]},{"osd":1,"uuid":"f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6809","nonce":1158817756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6811","nonce":1158817756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6815","nonce":1158817756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6813","nonce":1158817756}]},"public_addr":"172.21.15.45:6809/1158817756","cluster_addr":"172.21.15.45:6811/1158817756","heartbeat_back_addr":"172.21.15.45:6815/1158817756","heartbeat_front_addr":"172.21.15.45:6813/1158817756","state":["exists","up"]},{"osd":2,"uuid":"195a9db1-568d-459a-bb50-2a1bb2445f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6825","nonce":3298886784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6827","nonce":3298886784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6831","nonce":3298886784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6829","nonce":3298886784}]},"public_addr":"172.21.15.45:6825/3298886784","cluster_addr":"172.21.15.45:6827/3298886784","heartbeat_back_addr":"172.21.15.45:6831/3298886784","heartbeat_front_addr":"172.21.15.45:6829/3298886784","state":["exists","up"]},{"osd":3,"uuid":"b3fda98b-a10a-4354-905e-1b37a5124b98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1554491604},{"type":"v1","addr":"172.21.15.45:6817","nonce":1554491604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1554491604},{"type":"v1","addr":"172.21.15.45:6819","nonce":1554491604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1554491604},{"type":"v1","addr":"172.21.15.45:6823","nonce":1554491604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1554491604},{"type":"v1","addr":"172.21.15.45:6821","nonce":1554491604}]},"public_addr":"172.21.15.45:6817/1554491604","cluster_addr":"172.21.15.45:6819/1554491604","heartbeat_back_addr":"172.21.15.45:6823/1554491604","heartbeat_front_addr":"172.21.15.45:6821/1554491604","state":["exists","up"]},{"osd":4,"uuid":"6ab67b48-2d83-4e68-8898-302a1adc0a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6825","nonce":1644784047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6827","nonce":1644784047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6831","nonce":1644784047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6829","nonce":1644784047}]},"public_addr":"172.21.15.192:6825/1644784047","cluster_addr":"172.21.15.192:6827/1644784047","heartbeat_back_addr":"172.21.15.192:6831/1644784047","heartbeat_front_addr":"172.21.15.192:6829/1644784047","state":["exists","up"]},{"osd":5,"uuid":"ae3c787c-63d2-4e88-8f99-6d4f19837a3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6817","nonce":4077426583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6819","nonce":4077426583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6823","nonce":4077426583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6821","nonce":4077426583}]},"public_addr":"172.21.15.192:6817/4077426583","cluster_addr":"172.21.15.192:6819/4077426583","heartbeat_back_addr":"172.21.15.192:6823/4077426583","heartbeat_front_addr":"172.21.15.192:6821/4077426583","state":["exists","up"]},{"osd":6,"uuid":"f792f18d-ad9e-49a8-b194-24522398fb25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6809","nonce":2922229639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6811","nonce":2922229639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6815","nonce":2922229639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6813","nonce":2922229639}]},"public_addr":"172.21.15.192:6809/2922229639","cluster_addr":"172.21.15.192:6811/2922229639","heartbeat_back_addr":"172.21.15.192:6815/2922229639","heartbeat_front_addr":"172.21.15.192:6813/2922229639","state":["exists","up"]},{"osd":7,"uuid":"4a589d43-b0b9-42de-989c-46928d693c24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6801","nonce":780298944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6803","nonce":780298944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6807","nonce":780298944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6805","nonce":780298944}]},"public_addr":"172.21.15.192:6801/780298944","cluster_addr":"172.21.15.192:6803/780298944","heartbeat_back_addr":"172.21.15.192:6807/780298944","heartbeat_front_addr":"172.21.15.192:6805/780298944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.586514+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.833505+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:21.059899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.499470+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:21.353342+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.586673+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.767847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.859615+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T07:28:29.944 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-17T07:28:29.944 INFO:tasks.ceph:Creating RBD pool 2024-09-17T07:28:29.945 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-17T07:28:30.348 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'rbd' created 2024-09-17T07:28:30.361 DEBUG:teuthology.orchestra.run.smithi045:> rbd --cluster ceph pool init rbd 2024-09-17T07:28:33.370 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T07:28:33.371 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-17T07:28:33.371 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T07:28:33.681 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:28:33.693 INFO:teuthology.orchestra.run.smithi045.stdout:[{"version":9,"timestamp":"2024-09-17T07:28:23.009365+0000","name":"","changes":[{"name":"osd.4/osd_mclock_max_capacity_iops_ssd","new_value":"7479.689669"}]}] 2024-09-17T07:28:33.694 INFO:tasks.ceph_manager:config epoch is 9 2024-09-17T07:28:33.694 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T07:28:33.694 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T07:28:33.694 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T07:28:34.047 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:28:34.057 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":660847114},{"type":"v1","addr":"172.21.15.45:6833","nonce":660847114}]},"active_addr":"172.21.15.45:6833/660847114","active_change":"2024-09-17T07:28:16.743065+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4120,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":3670813977}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":2705810036}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":3266503080}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":2685671438}]}]} 2024-09-17T07:28:34.062 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T07:28:34.062 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T07:28:34.062 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T07:28:34.363 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:28:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"b283e6d5-0f4a-4271-a443-15e695e35364","created":"2024-09-17T07:28:12.276015+0000","modified":"2024-09-17T07:28:33.355114+0000","last_up_change":"2024-09-17T07:28:23.300710+0000","last_in_change":"2024-09-17T07:28:14.487756+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T07:28:24.767696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T07:28:30.249487+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7794e6a8-ebf4-4b7c-bf3d-63cd015234ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6801","nonce":2039818160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6803","nonce":2039818160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6807","nonce":2039818160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6805","nonce":2039818160}]},"public_addr":"172.21.15.45:6801/2039818160","cluster_addr":"172.21.15.45:6803/2039818160","heartbeat_back_addr":"172.21.15.45:6807/2039818160","heartbeat_front_addr":"172.21.15.45:6805/2039818160","state":["exists","up"]},{"osd":1,"uuid":"f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6809","nonce":1158817756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6811","nonce":1158817756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6815","nonce":1158817756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6813","nonce":1158817756}]},"public_addr":"172.21.15.45:6809/1158817756","cluster_addr":"172.21.15.45:6811/1158817756","heartbeat_back_addr":"172.21.15.45:6815/1158817756","heartbeat_front_addr":"172.21.15.45:6813/1158817756","state":["exists","up"]},{"osd":2,"uuid":"195a9db1-568d-459a-bb50-2a1bb2445f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6825","nonce":3298886784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6827","nonce":3298886784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6831","nonce":3298886784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6829","nonce":3298886784}]},"public_addr":"172.21.15.45:6825/3298886784","cluster_addr":"172.21.15.45:6827/3298886784","heartbeat_back_addr":"172.21.15.45:6831/3298886784","heartbeat_front_addr":"172.21.15.45:6829/3298886784","state":["exists","up"]},{"osd":3,"uuid":"b3fda98b-a10a-4354-905e-1b37a5124b98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1554491604},{"type":"v1","addr":"172.21.15.45:6817","nonce":1554491604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1554491604},{"type":"v1","addr":"172.21.15.45:6819","nonce":1554491604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1554491604},{"type":"v1","addr":"172.21.15.45:6823","nonce":1554491604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1554491604},{"type":"v1","addr":"172.21.15.45:6821","nonce":1554491604}]},"public_addr":"172.21.15.45:6817/1554491604","cluster_addr":"172.21.15.45:6819/1554491604","heartbeat_back_addr":"172.21.15.45:6823/1554491604","heartbeat_front_addr":"172.21.15.45:6821/1554491604","state":["exists","up"]},{"osd":4,"uuid":"6ab67b48-2d83-4e68-8898-302a1adc0a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6825","nonce":1644784047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6827","nonce":1644784047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6831","nonce":1644784047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6829","nonce":1644784047}]},"public_addr":"172.21.15.192:6825/1644784047","cluster_addr":"172.21.15.192:6827/1644784047","heartbeat_back_addr":"172.21.15.192:6831/1644784047","heartbeat_front_addr":"172.21.15.192:6829/1644784047","state":["exists","up"]},{"osd":5,"uuid":"ae3c787c-63d2-4e88-8f99-6d4f19837a3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6817","nonce":4077426583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6819","nonce":4077426583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6823","nonce":4077426583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6821","nonce":4077426583}]},"public_addr":"172.21.15.192:6817/4077426583","cluster_addr":"172.21.15.192:6819/4077426583","heartbeat_back_addr":"172.21.15.192:6823/4077426583","heartbeat_front_addr":"172.21.15.192:6821/4077426583","state":["exists","up"]},{"osd":6,"uuid":"f792f18d-ad9e-49a8-b194-24522398fb25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6809","nonce":2922229639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6811","nonce":2922229639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6815","nonce":2922229639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6813","nonce":2922229639}]},"public_addr":"172.21.15.192:6809/2922229639","cluster_addr":"172.21.15.192:6811/2922229639","heartbeat_back_addr":"172.21.15.192:6815/2922229639","heartbeat_front_addr":"172.21.15.192:6813/2922229639","state":["exists","up"]},{"osd":7,"uuid":"4a589d43-b0b9-42de-989c-46928d693c24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6801","nonce":780298944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6803","nonce":780298944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6807","nonce":780298944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6805","nonce":780298944}]},"public_addr":"172.21.15.192:6801/780298944","cluster_addr":"172.21.15.192:6803/780298944","heartbeat_back_addr":"172.21.15.192:6807/780298944","heartbeat_front_addr":"172.21.15.192:6805/780298944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.586514+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.833505+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:21.059899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.499470+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:21.353342+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.586673+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.767847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.859615+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T07:28:34.376 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T07:28:34.376 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T07:28:34.674 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:28:34.674 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":22,"fsid":"b283e6d5-0f4a-4271-a443-15e695e35364","created":"2024-09-17T07:28:12.276015+0000","modified":"2024-09-17T07:28:33.355114+0000","last_up_change":"2024-09-17T07:28:23.300710+0000","last_in_change":"2024-09-17T07:28:14.487756+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T07:28:24.767696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T07:28:30.249487+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7794e6a8-ebf4-4b7c-bf3d-63cd015234ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6801","nonce":2039818160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6803","nonce":2039818160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6807","nonce":2039818160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6805","nonce":2039818160}]},"public_addr":"172.21.15.45:6801/2039818160","cluster_addr":"172.21.15.45:6803/2039818160","heartbeat_back_addr":"172.21.15.45:6807/2039818160","heartbeat_front_addr":"172.21.15.45:6805/2039818160","state":["exists","up"]},{"osd":1,"uuid":"f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6809","nonce":1158817756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6811","nonce":1158817756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6815","nonce":1158817756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6813","nonce":1158817756}]},"public_addr":"172.21.15.45:6809/1158817756","cluster_addr":"172.21.15.45:6811/1158817756","heartbeat_back_addr":"172.21.15.45:6815/1158817756","heartbeat_front_addr":"172.21.15.45:6813/1158817756","state":["exists","up"]},{"osd":2,"uuid":"195a9db1-568d-459a-bb50-2a1bb2445f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6825","nonce":3298886784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6827","nonce":3298886784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6831","nonce":3298886784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6829","nonce":3298886784}]},"public_addr":"172.21.15.45:6825/3298886784","cluster_addr":"172.21.15.45:6827/3298886784","heartbeat_back_addr":"172.21.15.45:6831/3298886784","heartbeat_front_addr":"172.21.15.45:6829/3298886784","state":["exists","up"]},{"osd":3,"uuid":"b3fda98b-a10a-4354-905e-1b37a5124b98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1554491604},{"type":"v1","addr":"172.21.15.45:6817","nonce":1554491604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1554491604},{"type":"v1","addr":"172.21.15.45:6819","nonce":1554491604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1554491604},{"type":"v1","addr":"172.21.15.45:6823","nonce":1554491604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1554491604},{"type":"v1","addr":"172.21.15.45:6821","nonce":1554491604}]},"public_addr":"172.21.15.45:6817/1554491604","cluster_addr":"172.21.15.45:6819/1554491604","heartbeat_back_addr":"172.21.15.45:6823/1554491604","heartbeat_front_addr":"172.21.15.45:6821/1554491604","state":["exists","up"]},{"osd":4,"uuid":"6ab67b48-2d83-4e68-8898-302a1adc0a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6825","nonce":1644784047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6827","nonce":1644784047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6831","nonce":1644784047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6829","nonce":1644784047}]},"public_addr":"172.21.15.192:6825/1644784047","cluster_addr":"172.21.15.192:6827/1644784047","heartbeat_back_addr":"172.21.15.192:6831/1644784047","heartbeat_front_addr":"172.21.15.192:6829/1644784047","state":["exists","up"]},{"osd":5,"uuid":"ae3c787c-63d2-4e88-8f99-6d4f19837a3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6817","nonce":4077426583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6819","nonce":4077426583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6823","nonce":4077426583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6821","nonce":4077426583}]},"public_addr":"172.21.15.192:6817/4077426583","cluster_addr":"172.21.15.192:6819/4077426583","heartbeat_back_addr":"172.21.15.192:6823/4077426583","heartbeat_front_addr":"172.21.15.192:6821/4077426583","state":["exists","up"]},{"osd":6,"uuid":"f792f18d-ad9e-49a8-b194-24522398fb25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6809","nonce":2922229639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6811","nonce":2922229639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6815","nonce":2922229639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6813","nonce":2922229639}]},"public_addr":"172.21.15.192:6809/2922229639","cluster_addr":"172.21.15.192:6811/2922229639","heartbeat_back_addr":"172.21.15.192:6815/2922229639","heartbeat_front_addr":"172.21.15.192:6813/2922229639","state":["exists","up"]},{"osd":7,"uuid":"4a589d43-b0b9-42de-989c-46928d693c24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6801","nonce":780298944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6803","nonce":780298944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6807","nonce":780298944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6805","nonce":780298944}]},"public_addr":"172.21.15.192:6801/780298944","cluster_addr":"172.21.15.192:6803/780298944","heartbeat_back_addr":"172.21.15.192:6807/780298944","heartbeat_front_addr":"172.21.15.192:6805/780298944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.586514+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.833505+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:21.059899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.499470+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:21.353342+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.586673+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.767847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.859615+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T07:28:34.687 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T07:28:34.688 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T07:28:34.688 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T07:28:34.688 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T07:28:34.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T07:28:34.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T07:28:34.690 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T07:28:34.690 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T07:28:34.936 INFO:teuthology.orchestra.run.smithi045.stdout:60129542148 2024-09-17T07:28:34.936 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T07:28:34.945 INFO:teuthology.orchestra.run.smithi045.stdout:60129542148 2024-09-17T07:28:34.945 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T07:28:34.946 INFO:teuthology.orchestra.run.smithi045.stdout:60129542148 2024-09-17T07:28:34.946 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T07:28:34.947 INFO:teuthology.orchestra.run.smithi045.stdout:60129542148 2024-09-17T07:28:34.947 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T07:28:34.949 INFO:teuthology.orchestra.run.smithi045.stdout:60129542148 2024-09-17T07:28:34.949 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T07:28:34.950 INFO:teuthology.orchestra.run.smithi045.stdout:60129542148 2024-09-17T07:28:34.950 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T07:28:34.951 INFO:teuthology.orchestra.run.smithi045.stdout:60129542149 2024-09-17T07:28:34.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T07:28:34.952 INFO:teuthology.orchestra.run.smithi045.stdout:60129542148 2024-09-17T07:28:34.953 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T07:28:35.501 INFO:teuthology.orchestra.run.smithi045.stdout:60129542147 2024-09-17T07:28:35.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.3 2024-09-17T07:28:35.562 INFO:teuthology.orchestra.run.smithi045.stdout:60129542147 2024-09-17T07:28:35.575 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.1 2024-09-17T07:28:35.618 INFO:teuthology.orchestra.run.smithi045.stdout:60129542147 2024-09-17T07:28:35.627 INFO:teuthology.orchestra.run.smithi045.stdout:60129542147 2024-09-17T07:28:35.628 INFO:teuthology.orchestra.run.smithi045.stdout:60129542147 2024-09-17T07:28:35.631 INFO:teuthology.orchestra.run.smithi045.stdout:60129542147 2024-09-17T07:28:35.632 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.4 2024-09-17T07:28:35.636 INFO:teuthology.orchestra.run.smithi045.stdout:60129542147 2024-09-17T07:28:35.640 INFO:teuthology.orchestra.run.smithi045.stdout:60129542148 2024-09-17T07:28:35.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.5 2024-09-17T07:28:35.649 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.2 2024-09-17T07:28:35.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.7 2024-09-17T07:28:35.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.0 2024-09-17T07:28:35.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2024-09-17T07:28:36.522 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T07:28:36.634 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T07:28:36.635 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T07:28:36.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T07:28:36.650 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T07:28:36.655 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T07:28:36.657 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T07:28:36.657 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T07:28:37.236 INFO:teuthology.orchestra.run.smithi045.stdout:60129542147 2024-09-17T07:28:37.256 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.4 2024-09-17T07:28:37.260 INFO:teuthology.orchestra.run.smithi045.stdout:60129542147 2024-09-17T07:28:37.266 INFO:teuthology.orchestra.run.smithi045.stdout:60129542147 2024-09-17T07:28:37.281 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.3 2024-09-17T07:28:37.285 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.1 2024-09-17T07:28:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:60129542148 2024-09-17T07:28:37.304 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2024-09-17T07:28:37.322 INFO:teuthology.orchestra.run.smithi045.stdout:60129542147 2024-09-17T07:28:37.323 INFO:teuthology.orchestra.run.smithi045.stdout:60129542147 2024-09-17T07:28:37.333 INFO:teuthology.orchestra.run.smithi045.stdout:60129542147 2024-09-17T07:28:37.336 INFO:teuthology.orchestra.run.smithi045.stdout:60129542147 2024-09-17T07:28:37.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.2 2024-09-17T07:28:37.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.0 2024-09-17T07:28:37.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.7 2024-09-17T07:28:37.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.5 2024-09-17T07:28:38.257 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T07:28:38.282 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T07:28:38.286 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T07:28:38.304 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T07:28:38.342 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T07:28:38.342 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T07:28:38.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T07:28:38.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T07:28:38.808 INFO:teuthology.orchestra.run.smithi045.stdout:60129542148 2024-09-17T07:28:38.833 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.4 2024-09-17T07:28:38.833 DEBUG:teuthology.parallel:result is None 2024-09-17T07:28:38.866 INFO:teuthology.orchestra.run.smithi045.stdout:60129542148 2024-09-17T07:28:38.881 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.1 2024-09-17T07:28:38.881 DEBUG:teuthology.parallel:result is None 2024-09-17T07:28:38.896 INFO:teuthology.orchestra.run.smithi045.stdout:60129542149 2024-09-17T07:28:38.910 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.3 2024-09-17T07:28:38.910 DEBUG:teuthology.parallel:result is None 2024-09-17T07:28:38.933 INFO:teuthology.orchestra.run.smithi045.stdout:60129542149 2024-09-17T07:28:38.946 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.6 2024-09-17T07:28:38.946 DEBUG:teuthology.parallel:result is None 2024-09-17T07:28:38.954 INFO:teuthology.orchestra.run.smithi045.stdout:60129542149 2024-09-17T07:28:38.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.0 2024-09-17T07:28:38.969 DEBUG:teuthology.parallel:result is None 2024-09-17T07:28:38.974 INFO:teuthology.orchestra.run.smithi045.stdout:60129542149 2024-09-17T07:28:38.987 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.2 2024-09-17T07:28:38.987 DEBUG:teuthology.parallel:result is None 2024-09-17T07:28:38.988 INFO:teuthology.orchestra.run.smithi045.stdout:60129542148 2024-09-17T07:28:38.990 INFO:teuthology.orchestra.run.smithi045.stdout:60129542148 2024-09-17T07:28:39.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.7 2024-09-17T07:28:39.001 DEBUG:teuthology.parallel:result is None 2024-09-17T07:28:39.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.5 2024-09-17T07:28:39.003 DEBUG:teuthology.parallel:result is None 2024-09-17T07:28:39.003 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T07:28:39.004 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:28:39.338 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:28:39.339 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:28:39.350 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T07:28:38.752402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234796,"kb_used_data":1884,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739924,"statfs":{"total":773094113280,"available":772853682176,"internally_reserved":0,"allocated":1929216,"data_stored":1789902,"data_compressed":65196,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.402237"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.665025+0000","last_change":"2024-09-17T07:28:33.665025+0000","last_active":"2024-09-17T07:28:33.665025+0000","last_peered":"2024-09-17T07:28:33.665025+0000","last_clean":"2024-09-17T07:28:33.665025+0000","last_became_active":"2024-09-17T07:28:31.364445+0000","last_became_peered":"2024-09-17T07:28:31.364445+0000","last_unstale":"2024-09-17T07:28:33.665025+0000","last_undegraded":"2024-09-17T07:28:33.665025+0000","last_fullsized":"2024-09-17T07:28:33.665025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:23:49.197824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.361412+0000","last_change":"2024-09-17T07:28:33.361412+0000","last_active":"2024-09-17T07:28:33.361412+0000","last_peered":"2024-09-17T07:28:33.361412+0000","last_clean":"2024-09-17T07:28:33.361412+0000","last_became_active":"2024-09-17T07:28:31.362639+0000","last_became_peered":"2024-09-17T07:28:31.362639+0000","last_unstale":"2024-09-17T07:28:33.361412+0000","last_undegraded":"2024-09-17T07:28:33.361412+0000","last_fullsized":"2024-09-17T07:28:33.361412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:42.297390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049448899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.951359+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:28:33.951359+0000","last_peered":"2024-09-17T07:28:33.951359+0000","last_clean":"2024-09-17T07:28:33.951359+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:28:33.951359+0000","last_undegraded":"2024-09-17T07:28:33.951359+0000","last_fullsized":"2024-09-17T07:28:33.951359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.361447+0000","last_change":"2024-09-17T07:28:33.361447+0000","last_active":"2024-09-17T07:28:33.361447+0000","last_peered":"2024-09-17T07:28:33.361447+0000","last_clean":"2024-09-17T07:28:33.361447+0000","last_became_active":"2024-09-17T07:28:31.567277+0000","last_became_peered":"2024-09-17T07:28:31.567277+0000","last_unstale":"2024-09-17T07:28:33.361447+0000","last_undegraded":"2024-09-17T07:28:33.361447+0000","last_fullsized":"2024-09-17T07:28:33.361447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:09.089091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056035799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.364473+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:28:33.364473+0000","last_peered":"2024-09-17T07:28:33.364473+0000","last_clean":"2024-09-17T07:28:33.364473+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:28:33.364473+0000","last_undegraded":"2024-09-17T07:28:33.364473+0000","last_fullsized":"2024-09-17T07:28:33.364473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.952450+0000","last_change":"2024-09-17T07:28:33.952596+0000","last_active":"2024-09-17T07:28:33.952450+0000","last_peered":"2024-09-17T07:28:33.952450+0000","last_clean":"2024-09-17T07:28:33.952450+0000","last_became_active":"2024-09-17T07:28:31.354367+0000","last_became_peered":"2024-09-17T07:28:31.354367+0000","last_unstale":"2024-09-17T07:28:33.952450+0000","last_undegraded":"2024-09-17T07:28:33.952450+0000","last_fullsized":"2024-09-17T07:28:33.952450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:37.907368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.951319+0000","last_change":"2024-09-17T07:28:33.951319+0000","last_active":"2024-09-17T07:28:33.951319+0000","last_peered":"2024-09-17T07:28:33.951319+0000","last_clean":"2024-09-17T07:28:33.951319+0000","last_became_active":"2024-09-17T07:28:31.364065+0000","last_became_peered":"2024-09-17T07:28:31.364065+0000","last_unstale":"2024-09-17T07:28:33.951319+0000","last_undegraded":"2024-09-17T07:28:33.951319+0000","last_fullsized":"2024-09-17T07:28:33.951319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:40.816064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.361820+0000","last_change":"2024-09-17T07:28:33.361820+0000","last_active":"2024-09-17T07:28:33.361820+0000","last_peered":"2024-09-17T07:28:33.361820+0000","last_clean":"2024-09-17T07:28:33.361820+0000","last_became_active":"2024-09-17T07:28:31.359703+0000","last_became_peered":"2024-09-17T07:28:31.359703+0000","last_unstale":"2024-09-17T07:28:33.361820+0000","last_undegraded":"2024-09-17T07:28:33.361820+0000","last_fullsized":"2024-09-17T07:28:33.361820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:59.468099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000487639,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.950496+0000","last_change":"2024-09-17T07:28:26.547435+0000","last_active":"2024-09-17T07:28:33.950496+0000","last_peered":"2024-09-17T07:28:33.950496+0000","last_clean":"2024-09-17T07:28:33.950496+0000","last_became_active":"2024-09-17T07:28:26.546300+0000","last_became_peered":"2024-09-17T07:28:26.546300+0000","last_unstale":"2024-09-17T07:28:33.950496+0000","last_undegraded":"2024-09-17T07:28:33.950496+0000","last_fullsized":"2024-09-17T07:28:33.950496+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:02:23.225041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65196,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":661406,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":290816,"data_stored":667238,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:28:39.352 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:28:39.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:28:39.647 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:28:39.659 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T07:28:38.752402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234796,"kb_used_data":1884,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739924,"statfs":{"total":773094113280,"available":772853682176,"internally_reserved":0,"allocated":1929216,"data_stored":1789902,"data_compressed":65196,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.402237"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.665025+0000","last_change":"2024-09-17T07:28:33.665025+0000","last_active":"2024-09-17T07:28:33.665025+0000","last_peered":"2024-09-17T07:28:33.665025+0000","last_clean":"2024-09-17T07:28:33.665025+0000","last_became_active":"2024-09-17T07:28:31.364445+0000","last_became_peered":"2024-09-17T07:28:31.364445+0000","last_unstale":"2024-09-17T07:28:33.665025+0000","last_undegraded":"2024-09-17T07:28:33.665025+0000","last_fullsized":"2024-09-17T07:28:33.665025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:23:49.197824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.361412+0000","last_change":"2024-09-17T07:28:33.361412+0000","last_active":"2024-09-17T07:28:33.361412+0000","last_peered":"2024-09-17T07:28:33.361412+0000","last_clean":"2024-09-17T07:28:33.361412+0000","last_became_active":"2024-09-17T07:28:31.362639+0000","last_became_peered":"2024-09-17T07:28:31.362639+0000","last_unstale":"2024-09-17T07:28:33.361412+0000","last_undegraded":"2024-09-17T07:28:33.361412+0000","last_fullsized":"2024-09-17T07:28:33.361412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:42.297390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049448899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.951359+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:28:33.951359+0000","last_peered":"2024-09-17T07:28:33.951359+0000","last_clean":"2024-09-17T07:28:33.951359+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:28:33.951359+0000","last_undegraded":"2024-09-17T07:28:33.951359+0000","last_fullsized":"2024-09-17T07:28:33.951359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.361447+0000","last_change":"2024-09-17T07:28:33.361447+0000","last_active":"2024-09-17T07:28:33.361447+0000","last_peered":"2024-09-17T07:28:33.361447+0000","last_clean":"2024-09-17T07:28:33.361447+0000","last_became_active":"2024-09-17T07:28:31.567277+0000","last_became_peered":"2024-09-17T07:28:31.567277+0000","last_unstale":"2024-09-17T07:28:33.361447+0000","last_undegraded":"2024-09-17T07:28:33.361447+0000","last_fullsized":"2024-09-17T07:28:33.361447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:09.089091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056035799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.364473+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:28:33.364473+0000","last_peered":"2024-09-17T07:28:33.364473+0000","last_clean":"2024-09-17T07:28:33.364473+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:28:33.364473+0000","last_undegraded":"2024-09-17T07:28:33.364473+0000","last_fullsized":"2024-09-17T07:28:33.364473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.952450+0000","last_change":"2024-09-17T07:28:33.952596+0000","last_active":"2024-09-17T07:28:33.952450+0000","last_peered":"2024-09-17T07:28:33.952450+0000","last_clean":"2024-09-17T07:28:33.952450+0000","last_became_active":"2024-09-17T07:28:31.354367+0000","last_became_peered":"2024-09-17T07:28:31.354367+0000","last_unstale":"2024-09-17T07:28:33.952450+0000","last_undegraded":"2024-09-17T07:28:33.952450+0000","last_fullsized":"2024-09-17T07:28:33.952450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:37.907368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.951319+0000","last_change":"2024-09-17T07:28:33.951319+0000","last_active":"2024-09-17T07:28:33.951319+0000","last_peered":"2024-09-17T07:28:33.951319+0000","last_clean":"2024-09-17T07:28:33.951319+0000","last_became_active":"2024-09-17T07:28:31.364065+0000","last_became_peered":"2024-09-17T07:28:31.364065+0000","last_unstale":"2024-09-17T07:28:33.951319+0000","last_undegraded":"2024-09-17T07:28:33.951319+0000","last_fullsized":"2024-09-17T07:28:33.951319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:40.816064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.361820+0000","last_change":"2024-09-17T07:28:33.361820+0000","last_active":"2024-09-17T07:28:33.361820+0000","last_peered":"2024-09-17T07:28:33.361820+0000","last_clean":"2024-09-17T07:28:33.361820+0000","last_became_active":"2024-09-17T07:28:31.359703+0000","last_became_peered":"2024-09-17T07:28:31.359703+0000","last_unstale":"2024-09-17T07:28:33.361820+0000","last_undegraded":"2024-09-17T07:28:33.361820+0000","last_fullsized":"2024-09-17T07:28:33.361820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:59.468099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000487639,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.950496+0000","last_change":"2024-09-17T07:28:26.547435+0000","last_active":"2024-09-17T07:28:33.950496+0000","last_peered":"2024-09-17T07:28:33.950496+0000","last_clean":"2024-09-17T07:28:33.950496+0000","last_became_active":"2024-09-17T07:28:26.546300+0000","last_became_peered":"2024-09-17T07:28:26.546300+0000","last_unstale":"2024-09-17T07:28:33.950496+0000","last_undegraded":"2024-09-17T07:28:33.950496+0000","last_fullsized":"2024-09-17T07:28:33.950496+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:02:23.225041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65196,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":661406,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":290816,"data_stored":667238,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:28:39.660 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T07:28:39.660 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T07:28:39.660 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T07:28:39.660 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T07:28:39.995 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:28:39.995 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T07:28:40.008 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T07:28:40.009 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-17T07:28:40.035 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-17T07:28:40.035 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-dis-testing'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache max dirty': 0}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-dis-testing', 'sha1': '6e8474a02f2b86353394076ab80c6f50ea28e361'}} 2024-09-17T07:28:40.035 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-17T07:28:40.036 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-17T07:28:40.353 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-17T07:28:40.354 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T07:28:40.354 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:28:40.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:28:40.647 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:28:40.659 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T07:28:38.752402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234796,"kb_used_data":1884,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739924,"statfs":{"total":773094113280,"available":772853682176,"internally_reserved":0,"allocated":1929216,"data_stored":1789902,"data_compressed":65196,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.402237"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.665025+0000","last_change":"2024-09-17T07:28:33.665025+0000","last_active":"2024-09-17T07:28:33.665025+0000","last_peered":"2024-09-17T07:28:33.665025+0000","last_clean":"2024-09-17T07:28:33.665025+0000","last_became_active":"2024-09-17T07:28:31.364445+0000","last_became_peered":"2024-09-17T07:28:31.364445+0000","last_unstale":"2024-09-17T07:28:33.665025+0000","last_undegraded":"2024-09-17T07:28:33.665025+0000","last_fullsized":"2024-09-17T07:28:33.665025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:23:49.197824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.361412+0000","last_change":"2024-09-17T07:28:33.361412+0000","last_active":"2024-09-17T07:28:33.361412+0000","last_peered":"2024-09-17T07:28:33.361412+0000","last_clean":"2024-09-17T07:28:33.361412+0000","last_became_active":"2024-09-17T07:28:31.362639+0000","last_became_peered":"2024-09-17T07:28:31.362639+0000","last_unstale":"2024-09-17T07:28:33.361412+0000","last_undegraded":"2024-09-17T07:28:33.361412+0000","last_fullsized":"2024-09-17T07:28:33.361412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:42.297390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049448899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.951359+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:28:33.951359+0000","last_peered":"2024-09-17T07:28:33.951359+0000","last_clean":"2024-09-17T07:28:33.951359+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:28:33.951359+0000","last_undegraded":"2024-09-17T07:28:33.951359+0000","last_fullsized":"2024-09-17T07:28:33.951359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.361447+0000","last_change":"2024-09-17T07:28:33.361447+0000","last_active":"2024-09-17T07:28:33.361447+0000","last_peered":"2024-09-17T07:28:33.361447+0000","last_clean":"2024-09-17T07:28:33.361447+0000","last_became_active":"2024-09-17T07:28:31.567277+0000","last_became_peered":"2024-09-17T07:28:31.567277+0000","last_unstale":"2024-09-17T07:28:33.361447+0000","last_undegraded":"2024-09-17T07:28:33.361447+0000","last_fullsized":"2024-09-17T07:28:33.361447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:09.089091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056035799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.364473+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:28:33.364473+0000","last_peered":"2024-09-17T07:28:33.364473+0000","last_clean":"2024-09-17T07:28:33.364473+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:28:33.364473+0000","last_undegraded":"2024-09-17T07:28:33.364473+0000","last_fullsized":"2024-09-17T07:28:33.364473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.952450+0000","last_change":"2024-09-17T07:28:33.952596+0000","last_active":"2024-09-17T07:28:33.952450+0000","last_peered":"2024-09-17T07:28:33.952450+0000","last_clean":"2024-09-17T07:28:33.952450+0000","last_became_active":"2024-09-17T07:28:31.354367+0000","last_became_peered":"2024-09-17T07:28:31.354367+0000","last_unstale":"2024-09-17T07:28:33.952450+0000","last_undegraded":"2024-09-17T07:28:33.952450+0000","last_fullsized":"2024-09-17T07:28:33.952450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:37.907368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.951319+0000","last_change":"2024-09-17T07:28:33.951319+0000","last_active":"2024-09-17T07:28:33.951319+0000","last_peered":"2024-09-17T07:28:33.951319+0000","last_clean":"2024-09-17T07:28:33.951319+0000","last_became_active":"2024-09-17T07:28:31.364065+0000","last_became_peered":"2024-09-17T07:28:31.364065+0000","last_unstale":"2024-09-17T07:28:33.951319+0000","last_undegraded":"2024-09-17T07:28:33.951319+0000","last_fullsized":"2024-09-17T07:28:33.951319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:40.816064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.361820+0000","last_change":"2024-09-17T07:28:33.361820+0000","last_active":"2024-09-17T07:28:33.361820+0000","last_peered":"2024-09-17T07:28:33.361820+0000","last_clean":"2024-09-17T07:28:33.361820+0000","last_became_active":"2024-09-17T07:28:31.359703+0000","last_became_peered":"2024-09-17T07:28:31.359703+0000","last_unstale":"2024-09-17T07:28:33.361820+0000","last_undegraded":"2024-09-17T07:28:33.361820+0000","last_fullsized":"2024-09-17T07:28:33.361820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:59.468099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000487639,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.950496+0000","last_change":"2024-09-17T07:28:26.547435+0000","last_active":"2024-09-17T07:28:33.950496+0000","last_peered":"2024-09-17T07:28:33.950496+0000","last_clean":"2024-09-17T07:28:33.950496+0000","last_became_active":"2024-09-17T07:28:26.546300+0000","last_became_peered":"2024-09-17T07:28:26.546300+0000","last_unstale":"2024-09-17T07:28:33.950496+0000","last_undegraded":"2024-09-17T07:28:33.950496+0000","last_fullsized":"2024-09-17T07:28:33.950496+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:02:23.225041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65196,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":661406,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":290816,"data_stored":667238,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:28:40.660 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:28:40.955 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:28:40.956 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:28:40.967 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-17T07:28:40.752746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234804,"kb_used_data":1896,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739916,"statfs":{"total":773094113280,"available":772853673984,"internally_reserved":0,"allocated":1941504,"data_stored":1795734,"data_compressed":65196,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.397663"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.665025+0000","last_change":"2024-09-17T07:28:33.665025+0000","last_active":"2024-09-17T07:28:33.665025+0000","last_peered":"2024-09-17T07:28:33.665025+0000","last_clean":"2024-09-17T07:28:33.665025+0000","last_became_active":"2024-09-17T07:28:31.364445+0000","last_became_peered":"2024-09-17T07:28:31.364445+0000","last_unstale":"2024-09-17T07:28:33.665025+0000","last_undegraded":"2024-09-17T07:28:33.665025+0000","last_fullsized":"2024-09-17T07:28:33.665025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:23:49.197824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.361412+0000","last_change":"2024-09-17T07:28:33.361412+0000","last_active":"2024-09-17T07:28:33.361412+0000","last_peered":"2024-09-17T07:28:33.361412+0000","last_clean":"2024-09-17T07:28:33.361412+0000","last_became_active":"2024-09-17T07:28:31.362639+0000","last_became_peered":"2024-09-17T07:28:31.362639+0000","last_unstale":"2024-09-17T07:28:33.361412+0000","last_undegraded":"2024-09-17T07:28:33.361412+0000","last_fullsized":"2024-09-17T07:28:33.361412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:42.297390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049448899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.951359+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:28:33.951359+0000","last_peered":"2024-09-17T07:28:33.951359+0000","last_clean":"2024-09-17T07:28:33.951359+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:28:33.951359+0000","last_undegraded":"2024-09-17T07:28:33.951359+0000","last_fullsized":"2024-09-17T07:28:33.951359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.361447+0000","last_change":"2024-09-17T07:28:33.361447+0000","last_active":"2024-09-17T07:28:33.361447+0000","last_peered":"2024-09-17T07:28:33.361447+0000","last_clean":"2024-09-17T07:28:33.361447+0000","last_became_active":"2024-09-17T07:28:31.567277+0000","last_became_peered":"2024-09-17T07:28:31.567277+0000","last_unstale":"2024-09-17T07:28:33.361447+0000","last_undegraded":"2024-09-17T07:28:33.361447+0000","last_fullsized":"2024-09-17T07:28:33.361447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:09.089091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056035799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.364473+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:28:33.364473+0000","last_peered":"2024-09-17T07:28:33.364473+0000","last_clean":"2024-09-17T07:28:33.364473+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:28:33.364473+0000","last_undegraded":"2024-09-17T07:28:33.364473+0000","last_fullsized":"2024-09-17T07:28:33.364473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.952450+0000","last_change":"2024-09-17T07:28:33.952596+0000","last_active":"2024-09-17T07:28:33.952450+0000","last_peered":"2024-09-17T07:28:33.952450+0000","last_clean":"2024-09-17T07:28:33.952450+0000","last_became_active":"2024-09-17T07:28:31.354367+0000","last_became_peered":"2024-09-17T07:28:31.354367+0000","last_unstale":"2024-09-17T07:28:33.952450+0000","last_undegraded":"2024-09-17T07:28:33.952450+0000","last_fullsized":"2024-09-17T07:28:33.952450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:37.907368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.951319+0000","last_change":"2024-09-17T07:28:33.951319+0000","last_active":"2024-09-17T07:28:33.951319+0000","last_peered":"2024-09-17T07:28:33.951319+0000","last_clean":"2024-09-17T07:28:33.951319+0000","last_became_active":"2024-09-17T07:28:31.364065+0000","last_became_peered":"2024-09-17T07:28:31.364065+0000","last_unstale":"2024-09-17T07:28:33.951319+0000","last_undegraded":"2024-09-17T07:28:33.951319+0000","last_fullsized":"2024-09-17T07:28:33.951319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:40.816064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.361820+0000","last_change":"2024-09-17T07:28:33.361820+0000","last_active":"2024-09-17T07:28:33.361820+0000","last_peered":"2024-09-17T07:28:33.361820+0000","last_clean":"2024-09-17T07:28:33.361820+0000","last_became_active":"2024-09-17T07:28:31.359703+0000","last_became_peered":"2024-09-17T07:28:31.359703+0000","last_unstale":"2024-09-17T07:28:33.361820+0000","last_undegraded":"2024-09-17T07:28:33.361820+0000","last_fullsized":"2024-09-17T07:28:33.361820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:59.468099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000487639,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.950496+0000","last_change":"2024-09-17T07:28:26.547435+0000","last_active":"2024-09-17T07:28:33.950496+0000","last_peered":"2024-09-17T07:28:33.950496+0000","last_clean":"2024-09-17T07:28:33.950496+0000","last_became_active":"2024-09-17T07:28:26.546300+0000","last_became_peered":"2024-09-17T07:28:26.546300+0000","last_unstale":"2024-09-17T07:28:33.950496+0000","last_undegraded":"2024-09-17T07:28:33.950496+0000","last_fullsized":"2024-09-17T07:28:33.950496+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:02:23.225041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65196,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":667238,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":290816,"data_stored":667238,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:28:40.968 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T07:28:40.968 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-17T07:28:41.268 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 22 2024-09-17T07:28:41.269 INFO:teuthology.orchestra.run.smithi045.stdout:fsid b283e6d5-0f4a-4271-a443-15e695e35364 2024-09-17T07:28:41.269 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-09-17T07:28:12.276015+0000 2024-09-17T07:28:41.269 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-09-17T07:28:33.355114+0000 2024-09-17T07:28:41.269 INFO:teuthology.orchestra.run.smithi045.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-17T07:28:41.269 INFO:teuthology.orchestra.run.smithi045.stdout:crush_version 5 2024-09-17T07:28:41.269 INFO:teuthology.orchestra.run.smithi045.stdout:full_ratio 0.95 2024-09-17T07:28:41.269 INFO:teuthology.orchestra.run.smithi045.stdout:backfillfull_ratio 0.9 2024-09-17T07:28:41.269 INFO:teuthology.orchestra.run.smithi045.stdout:nearfull_ratio 0.85 2024-09-17T07:28:41.269 INFO:teuthology.orchestra.run.smithi045.stdout:require_min_compat_client luminous 2024-09-17T07:28:41.269 INFO:teuthology.orchestra.run.smithi045.stdout:min_compat_client jewel 2024-09-17T07:28:41.269 INFO:teuthology.orchestra.run.smithi045.stdout:require_osd_release squid 2024-09-17T07:28:41.269 INFO:teuthology.orchestra.run.smithi045.stdout:stretch_mode_enabled false 2024-09-17T07:28:41.269 INFO:teuthology.orchestra.run.smithi045.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-17T07:28:41.270 INFO:teuthology.orchestra.run.smithi045.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 22 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-17T07:28:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: removed_snaps_queue [2~1] 2024-09-17T07:28:41.270 INFO:teuthology.orchestra.run.smithi045.stdout:max_osd 8 2024-09-17T07:28:41.270 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6800/2039818160,v1:172.21.15.45:6801/2039818160] [v2:172.21.15.45:6802/2039818160,v1:172.21.15.45:6803/2039818160] exists,up 7794e6a8-ebf4-4b7c-bf3d-63cd015234ef 2024-09-17T07:28:41.270 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6808/1158817756,v1:172.21.15.45:6809/1158817756] [v2:172.21.15.45:6810/1158817756,v1:172.21.15.45:6811/1158817756] exists,up f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5 2024-09-17T07:28:41.270 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6824/3298886784,v1:172.21.15.45:6825/3298886784] [v2:172.21.15.45:6826/3298886784,v1:172.21.15.45:6827/3298886784] exists,up 195a9db1-568d-459a-bb50-2a1bb2445f53 2024-09-17T07:28:41.270 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6816/1554491604,v1:172.21.15.45:6817/1554491604] [v2:172.21.15.45:6818/1554491604,v1:172.21.15.45:6819/1554491604] exists,up b3fda98b-a10a-4354-905e-1b37a5124b98 2024-09-17T07:28:41.270 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6824/1644784047,v1:172.21.15.192:6825/1644784047] [v2:172.21.15.192:6826/1644784047,v1:172.21.15.192:6827/1644784047] exists,up 6ab67b48-2d83-4e68-8898-302a1adc0a52 2024-09-17T07:28:41.270 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/4077426583,v1:172.21.15.192:6817/4077426583] [v2:172.21.15.192:6818/4077426583,v1:172.21.15.192:6819/4077426583] exists,up ae3c787c-63d2-4e88-8f99-6d4f19837a3a 2024-09-17T07:28:41.270 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6808/2922229639,v1:172.21.15.192:6809/2922229639] [v2:172.21.15.192:6810/2922229639,v1:172.21.15.192:6811/2922229639] exists,up f792f18d-ad9e-49a8-b194-24522398fb25 2024-09-17T07:28:41.270 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/780298944,v1:172.21.15.192:6801/780298944] [v2:172.21.15.192:6802/780298944,v1:172.21.15.192:6803/780298944] exists,up 4a589d43-b0b9-42de-989c-46928d693c24 2024-09-17T07:28:41.281 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6800/2039818160,v1:172.21.15.45:6801/2039818160] [v2:172.21.15.45:6802/2039818160,v1:172.21.15.45:6803/2039818160] exists,up 7794e6a8-ebf4-4b7c-bf3d-63cd015234ef', 'osd.1 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6808/1158817756,v1:172.21.15.45:6809/1158817756] [v2:172.21.15.45:6810/1158817756,v1:172.21.15.45:6811/1158817756] exists,up f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5', 'osd.2 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6824/3298886784,v1:172.21.15.45:6825/3298886784] [v2:172.21.15.45:6826/3298886784,v1:172.21.15.45:6827/3298886784] exists,up 195a9db1-568d-459a-bb50-2a1bb2445f53', 'osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6816/1554491604,v1:172.21.15.45:6817/1554491604] [v2:172.21.15.45:6818/1554491604,v1:172.21.15.45:6819/1554491604] exists,up b3fda98b-a10a-4354-905e-1b37a5124b98', 'osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6824/1644784047,v1:172.21.15.192:6825/1644784047] [v2:172.21.15.192:6826/1644784047,v1:172.21.15.192:6827/1644784047] exists,up 6ab67b48-2d83-4e68-8898-302a1adc0a52', 'osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/4077426583,v1:172.21.15.192:6817/4077426583] [v2:172.21.15.192:6818/4077426583,v1:172.21.15.192:6819/4077426583] exists,up ae3c787c-63d2-4e88-8f99-6d4f19837a3a', 'osd.6 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6808/2922229639,v1:172.21.15.192:6809/2922229639] [v2:172.21.15.192:6810/2922229639,v1:172.21.15.192:6811/2922229639] exists,up f792f18d-ad9e-49a8-b194-24522398fb25', 'osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/780298944,v1:172.21.15.192:6801/780298944] [v2:172.21.15.192:6802/780298944,v1:172.21.15.192:6803/780298944] exists,up 4a589d43-b0b9-42de-989c-46928d693c24'] 2024-09-17T07:28:41.282 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-17T07:28:41.385 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mon.a", 2024-09-17T07:28:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster": "ceph", 2024-09-17T07:28:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-17T07:28:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "admin_socket_mode": "", 2024-09-17T07:28:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-17T07:28:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_client_required": "cephx, none", 2024-09-17T07:28:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_cluster_required": "cephx", 2024-09-17T07:28:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_debug": "true", 2024-09-17T07:28:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-17T07:28:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-17T07:28:41.386 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_service_required": "cephx", 2024-09-17T07:28:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-17T07:28:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_supported": "cephx", 2024-09-17T07:28:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_aio": "true", 2024-09-17T07:28:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-17T07:28:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_aio_poll_ms": "250", 2024-09-17T07:28:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_aio_reap_max": "16", 2024-09-17T07:28:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_async_discard_threads": "0", 2024-09-17T07:28:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_block_size": "4096", 2024-09-17T07:28:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_debug_aio": "false", 2024-09-17T07:28:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-17T07:28:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-17T07:28:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_debug_inflight_ios": "false", 2024-09-17T07:28:41.387 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_enable_discard": "false", 2024-09-17T07:28:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_flock_retry": "3", 2024-09-17T07:28:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-17T07:28:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_inject_crash": "0", 2024-09-17T07:28:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-17T07:28:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_ioring": "false", 2024-09-17T07:28:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_ioring_hipri": "false", 2024-09-17T07:28:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-17T07:28:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-17T07:28:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-17T07:28:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-17T07:28:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-17T07:28:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-17T07:28:41.388 INFO:teuthology.orchestra.run.smithi045.stdout: "bdev_type": "", 2024-09-17T07:28:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_alloc_size": "1048576", 2024-09-17T07:28:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_allocator": "hybrid", 2024-09-17T07:28:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_buffered_io": "true", 2024-09-17T07:28:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_check_for_zeros": "false", 2024-09-17T07:28:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-17T07:28:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-17T07:28:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_compact_log_sync": "false", 2024-09-17T07:28:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-17T07:28:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-17T07:28:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-17T07:28:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-17T07:28:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_max_log_runway": "4194304", 2024-09-17T07:28:41.389 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_max_prefetch": "1048576", 2024-09-17T07:28:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_min_flush_size": "524288", 2024-09-17T07:28:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_min_log_runway": "1048576", 2024-09-17T07:28:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_replay_recovery": "false", 2024-09-17T07:28:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-17T07:28:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-17T07:28:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_sync_write": "false", 2024-09-17T07:28:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-17T07:28:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-17T07:28:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-17T07:28:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_allocation_from_file": "true", 2024-09-17T07:28:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_allocator": "hybrid", 2024-09-17T07:28:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_async_db_compaction": "true", 2024-09-17T07:28:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-17T07:28:41.390 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-17T07:28:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-17T07:28:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-17T07:28:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bdev_label_multi": "true", 2024-09-17T07:28:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-17T07:28:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-17T07:28:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-17T07:28:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-17T07:28:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-17T07:28:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_create": "true", 2024-09-17T07:28:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_db_create": "false", 2024-09-17T07:28:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_db_path": "", 2024-09-17T07:28:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_db_size": "0", 2024-09-17T07:28:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_path": "", 2024-09-17T07:28:41.391 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_preallocate_file": "false", 2024-09-17T07:28:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_size": "107374182400", 2024-09-17T07:28:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_wal_create": "false", 2024-09-17T07:28:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_wal_path": "", 2024-09-17T07:28:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_block_wal_size": "100663296", 2024-09-17T07:28:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bluefs": "true", 2024-09-17T07:28:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-17T07:28:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-17T07:28:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-17T07:28:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-17T07:28:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-17T07:28:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T07:28:41.392 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T07:28:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-17T07:28:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T07:28:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_autotune": "true", 2024-09-17T07:28:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-17T07:28:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-17T07:28:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-17T07:28:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-17T07:28:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_size": "0", 2024-09-17T07:28:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-17T07:28:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-17T07:28:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-17T07:28:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-17T07:28:41.393 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_type": "2q", 2024-09-17T07:28:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-17T07:28:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_clone_cow": "true", 2024-09-17T07:28:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-17T07:28:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-17T07:28:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-17T07:28:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-17T07:28:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-17T07:28:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-17T07:28:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-17T07:28:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_mode": "none", 2024-09-17T07:28:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-17T07:28:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_csum_type": "crc32c", 2024-09-17T07:28:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-17T07:28:41.394 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-17T07:28:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_freelist": "false", 2024-09-17T07:28:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-17T07:28:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-17T07:28:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-17T07:28:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-17T07:28:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-17T07:28:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-17T07:28:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-17T07:28:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-17T07:28:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-17T07:28:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-17T07:28:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-17T07:28:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-17T07:28:41.395 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_small_allocations": "0", 2024-09-17T07:28:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-17T07:28:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_default_buffered_read": "true", 2024-09-17T07:28:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_default_buffered_write": "false", 2024-09-17T07:28:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-17T07:28:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-17T07:28:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-17T07:28:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-17T07:28:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-17T07:28:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-17T07:28:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-17T07:28:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-17T07:28:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-17T07:28:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fail_eio": "false", 2024-09-17T07:28:41.396 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-17T07:28:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-17T07:28:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-17T07:28:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-17T07:28:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-17T07:28:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-17T07:28:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_on_mount": "false", 2024-09-17T07:28:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-17T07:28:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_on_umount": "false", 2024-09-17T07:28:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-17T07:28:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-17T07:28:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-17T07:28:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-17T07:28:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-17T07:28:41.397 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-17T07:28:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-17T07:28:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-17T07:28:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_ignore_data_csum": "false", 2024-09-17T07:28:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-17T07:28:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-17T07:28:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-17T07:28:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-17T07:28:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_log_op_age": "5.000000", 2024-09-17T07:28:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_max_alloc_size": "0", 2024-09-17T07:28:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_max_blob_size": "0", 2024-09-17T07:28:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-17T07:28:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-17T07:28:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-17T07:28:41.398 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_max_deferred_txc": "32", 2024-09-17T07:28:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_min_alloc_size": "0", 2024-09-17T07:28:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-17T07:28:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-17T07:28:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_nid_prealloc": "1024", 2024-09-17T07:28:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-17T07:28:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-17T07:28:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-17T07:28:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-17T07:28:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_retry_disk_reads": "3", 2024-09-17T07:28:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_rocksdb_cf": "true", 2024-09-17T07:28:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-17T07:28:41.399 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-17T07:28:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-17T07:28:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-17T07:28:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-17T07:28:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-17T07:28:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-17T07:28:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-17T07:28:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_spdk_mem": "512", 2024-09-17T07:28:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-17T07:28:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-17T07:28:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-17T07:28:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-17T07:28:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-17T07:28:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-17T07:28:41.400 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-17T07:28:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_tracing": "false", 2024-09-17T07:28:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-17T07:28:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-17T07:28:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-17T07:28:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-17T07:28:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-17T07:28:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-17T07:28:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-17T07:28:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-17T07:28:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-17T07:28:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_write_v2": "false", 2024-09-17T07:28:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_write_v2_random": "false", 2024-09-17T07:28:41.401 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_zero_block_detection": "false", 2024-09-17T07:28:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-17T07:28:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-17T07:28:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-17T07:28:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-17T07:28:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-17T07:28:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-17T07:28:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-17T07:28:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-17T07:28:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-17T07:28:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-17T07:28:41.403 INFO:teuthology.orchestra.run.smithi045.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-17T07:28:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-17T07:28:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-17T07:28:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-17T07:28:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "cephx_cluster_require_signatures": "false", 2024-09-17T07:28:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "cephx_cluster_require_version": "2", 2024-09-17T07:28:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "cephx_require_signatures": "false", 2024-09-17T07:28:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "cephx_require_version": "2", 2024-09-17T07:28:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "cephx_service_require_signatures": "false", 2024-09-17T07:28:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "cephx_service_require_version": "2", 2024-09-17T07:28:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "cephx_sign_messages": "true", 2024-09-17T07:28:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "chdir": "", 2024-09-17T07:28:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "client_acl_type": "", 2024-09-17T07:28:41.404 INFO:teuthology.orchestra.run.smithi045.stdout: "client_asio_thread_count": "2", 2024-09-17T07:28:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "client_cache_mid": "0.750000", 2024-09-17T07:28:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "client_cache_size": "16384", 2024-09-17T07:28:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "client_caps_release_delay": "5", 2024-09-17T07:28:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "client_check_pool_perm": "true", 2024-09-17T07:28:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-17T07:28:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "client_debug_force_sync_read": "false", 2024-09-17T07:28:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "client_debug_getattr_caps": "false", 2024-09-17T07:28:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "client_debug_inject_features": "", 2024-09-17T07:28:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "client_debug_inject_tick_delay": "0", 2024-09-17T07:28:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-17T07:28:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "client_die_on_failed_remount": "false", 2024-09-17T07:28:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "client_dirsize_rbytes": "true", 2024-09-17T07:28:41.405 INFO:teuthology.orchestra.run.smithi045.stdout: "client_force_lazyio": "false", 2024-09-17T07:28:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "client_fs": "", 2024-09-17T07:28:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-17T07:28:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "client_inject_release_failure": "false", 2024-09-17T07:28:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "client_max_inline_size": "4096", 2024-09-17T07:28:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-17T07:28:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "client_mds_namespace": "", 2024-09-17T07:28:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "client_metadata": "", 2024-09-17T07:28:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "client_mount_gid": "-1", 2024-09-17T07:28:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "client_mount_timeout": "300", 2024-09-17T07:28:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "client_mount_uid": "-1", 2024-09-17T07:28:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "client_mountpoint": "/", 2024-09-17T07:28:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "client_notify_timeout": "10", 2024-09-17T07:28:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "client_oc": "true", 2024-09-17T07:28:41.406 INFO:teuthology.orchestra.run.smithi045.stdout: "client_oc_max_dirty": "104857600", 2024-09-17T07:28:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-17T07:28:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_oc_max_objects": "1000", 2024-09-17T07:28:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_oc_size": "209715200", 2024-09-17T07:28:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_oc_target_dirty": "8388608", 2024-09-17T07:28:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_permissions": "true", 2024-09-17T07:28:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_quota": "true", 2024-09-17T07:28:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_quota_df": "true", 2024-09-17T07:28:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_readahead_max_bytes": "0", 2024-09-17T07:28:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_readahead_max_periods": "4", 2024-09-17T07:28:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_readahead_min": "131072", 2024-09-17T07:28:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_reconnect_stale": "false", 2024-09-17T07:28:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_shutdown_timeout": "30", 2024-09-17T07:28:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_snapdir": ".snap", 2024-09-17T07:28:41.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_tick_interval": "1", 2024-09-17T07:28:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "client_trace": "", 2024-09-17T07:28:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "client_try_dentry_invalidate": "false", 2024-09-17T07:28:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "client_use_faked_inos": "false", 2024-09-17T07:28:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "client_use_random_mds": "false", 2024-09-17T07:28:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "clog_to_graylog": "false", 2024-09-17T07:28:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-17T07:28:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "clog_to_graylog_port": "12201", 2024-09-17T07:28:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "clog_to_monitors": "default=true", 2024-09-17T07:28:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "clog_to_syslog": "false", 2024-09-17T07:28:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-17T07:28:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "clog_to_syslog_level": "info", 2024-09-17T07:28:41.408 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "-", 2024-09-17T07:28:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_network": "", 2024-09-17T07:28:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_network_interface": "", 2024-09-17T07:28:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "compressor_zlib_isal": "false", 2024-09-17T07:28:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "compressor_zlib_level": "5", 2024-09-17T07:28:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "compressor_zlib_winsize": "-15", 2024-09-17T07:28:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "compressor_zstd_level": "1", 2024-09-17T07:28:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-17T07:28:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-17T07:28:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "crimson_alien_op_num_threads": "6", 2024-09-17T07:28:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-17T07:28:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-17T07:28:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-17T07:28:41.409 INFO:teuthology.orchestra.run.smithi045.stdout: "crimson_osd_stat_interval": "0", 2024-09-17T07:28:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "crimson_seastar_cpu_cores": "", 2024-09-17T07:28:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "crimson_seastar_num_threads": "0", 2024-09-17T07:28:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location": "", 2024-09-17T07:28:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location_hook": "", 2024-09-17T07:28:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_location_hook_timeout": "10", 2024-09-17T07:28:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "daemonize": "false", 2024-09-17T07:28:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "daos_pool": "tank", 2024-09-17T07:28:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-17T07:28:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-17T07:28:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-17T07:28:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_alienstore": "0/5", 2024-09-17T07:28:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_allow_any_pool_priority": "false", 2024-09-17T07:28:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_asok": "1/5", 2024-09-17T07:28:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_asok_assert_abort": "false", 2024-09-17T07:28:41.410 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_asserts_on_shutdown": "true", 2024-09-17T07:28:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_auth": "20/20", 2024-09-17T07:28:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_bdev": "1/3", 2024-09-17T07:28:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_bluefs": "1/5", 2024-09-17T07:28:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_bluestore": "1/5", 2024-09-17T07:28:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_buffer": "0/1", 2024-09-17T07:28:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_ceph_dedup": "0/5", 2024-09-17T07:28:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_ceph_exporter": "1/5", 2024-09-17T07:28:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_cephfs_mirror": "0/5", 2024-09-17T07:28:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_cephsqlite": "0/5", 2024-09-17T07:28:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_client": "0/5", 2024-09-17T07:28:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_compressor": "1/5", 2024-09-17T07:28:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_context": "0/1", 2024-09-17T07:28:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_crimson_interrupt": "0/5", 2024-09-17T07:28:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_crush": "1/1", 2024-09-17T07:28:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_crypto": "1/5", 2024-09-17T07:28:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_cyanstore": "0/5", 2024-09-17T07:28:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_deliberately_leak_memory": "false", 2024-09-17T07:28:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_disable_randomized_ping": "false", 2024-09-17T07:28:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_dpdk": "1/5", 2024-09-17T07:28:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_eventtrace": "1/5", 2024-09-17T07:28:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_filer": "0/1", 2024-09-17T07:28:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_filestore": "1/3", 2024-09-17T07:28:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_finisher": "1/1", 2024-09-17T07:28:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_fuse": "1/5", 2024-09-17T07:28:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_heartbeat_testing_span": "0", 2024-09-17T07:28:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_heartbeatmap": "1/5", 2024-09-17T07:28:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-17T07:28:41.412 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_javaclient": "1/5", 2024-09-17T07:28:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_journal": "1/3", 2024-09-17T07:28:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_journaler": "0/5", 2024-09-17T07:28:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_kstore": "1/5", 2024-09-17T07:28:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_lockdep": "0/1", 2024-09-17T07:28:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mclock": "1/5", 2024-09-17T07:28:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mds": "1/5", 2024-09-17T07:28:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mds_balancer": "1/5", 2024-09-17T07:28:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mds_locker": "1/5", 2024-09-17T07:28:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mds_log": "1/5", 2024-09-17T07:28:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mds_log_expire": "1/5", 2024-09-17T07:28:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mds_migrator": "1/5", 2024-09-17T07:28:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mds_quiesce": "3/5", 2024-09-17T07:28:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_memstore": "1/5", 2024-09-17T07:28:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mgr": "2/5", 2024-09-17T07:28:41.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mgrc": "1/5", 2024-09-17T07:28:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_mon": "20/20", 2024-09-17T07:28:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_monc": "0/10", 2024-09-17T07:28:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_ms": "1/1", 2024-09-17T07:28:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_none": "0/5", 2024-09-17T07:28:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_objclass": "0/5", 2024-09-17T07:28:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_objectcacher": "0/5", 2024-09-17T07:28:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_objecter": "0/1", 2024-09-17T07:28:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_optracker": "0/5", 2024-09-17T07:28:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_osd": "1/5", 2024-09-17T07:28:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_paxos": "20/20", 2024-09-17T07:28:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_perfcounter": "1/5", 2024-09-17T07:28:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_prioritycache": "1/5", 2024-09-17T07:28:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rados": "0/5", 2024-09-17T07:28:41.414 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rbd": "0/5", 2024-09-17T07:28:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rbd_mirror": "0/5", 2024-09-17T07:28:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rbd_pwl": "0/5", 2024-09-17T07:28:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rbd_replay": "0/5", 2024-09-17T07:28:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_refs": "0/0", 2024-09-17T07:28:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_reserver": "1/1", 2024-09-17T07:28:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rgw": "1/5", 2024-09-17T07:28:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rgw_access": "1/5", 2024-09-17T07:28:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rgw_datacache": "1/5", 2024-09-17T07:28:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rgw_dbstore": "1/5", 2024-09-17T07:28:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rgw_flight": "1/5", 2024-09-17T07:28:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-17T07:28:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rgw_notification": "1/5", 2024-09-17T07:28:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rgw_sync": "1/5", 2024-09-17T07:28:41.415 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_rocksdb": "4/5", 2024-09-17T07:28:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore": "0/5", 2024-09-17T07:28:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_backref": "0/5", 2024-09-17T07:28:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_cache": "0/5", 2024-09-17T07:28:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_cleaner": "0/5", 2024-09-17T07:28:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_device": "0/5", 2024-09-17T07:28:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_epm": "0/5", 2024-09-17T07:28:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-17T07:28:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_journal": "0/5", 2024-09-17T07:28:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_lba": "0/5", 2024-09-17T07:28:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_odata": "0/5", 2024-09-17T07:28:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_omap": "0/5", 2024-09-17T07:28:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_onode": "0/5", 2024-09-17T07:28:41.416 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_t": "0/5", 2024-09-17T07:28:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_seastore_tm": "0/5", 2024-09-17T07:28:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_striper": "0/1", 2024-09-17T07:28:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_test": "0/5", 2024-09-17T07:28:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_throttle": "1/1", 2024-09-17T07:28:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_timer": "0/1", 2024-09-17T07:28:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_tp": "0/5", 2024-09-17T07:28:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "debug_trace": "1/5", 2024-09-17T07:28:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-17T07:28:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "device_failure_prediction_mode": "none", 2024-09-17T07:28:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-17T07:28:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-17T07:28:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "err_to_graylog": "false", 2024-09-17T07:28:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "err_to_journald": "false", 2024-09-17T07:28:41.417 INFO:teuthology.orchestra.run.smithi045.stdout: "err_to_stderr": "true", 2024-09-17T07:28:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "err_to_syslog": "false", 2024-09-17T07:28:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event_tracing": "false", 2024-09-17T07:28:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "exporter_addr": "0.0.0.0", 2024-09-17T07:28:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "exporter_cert_file": "", 2024-09-17T07:28:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "exporter_http_port": "9926", 2024-09-17T07:28:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "exporter_key_file": "", 2024-09-17T07:28:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "exporter_prio_limit": "5", 2024-09-17T07:28:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-17T07:28:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "exporter_sort_metrics": "true", 2024-09-17T07:28:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "exporter_stats_period": "5", 2024-09-17T07:28:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "fake_statfs_for_testing": "0", 2024-09-17T07:28:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "fatal_signal_handlers": "true", 2024-09-17T07:28:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "filer_max_purge_ops": "10", 2024-09-17T07:28:41.418 INFO:teuthology.orchestra.run.smithi045.stdout: "filer_max_truncate_ops": "128", 2024-09-17T07:28:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_apply_finisher_threads": "1", 2024-09-17T07:28:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_blackhole": "false", 2024-09-17T07:28:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_btrfs_clone_range": "true", 2024-09-17T07:28:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_btrfs_snap": "true", 2024-09-17T07:28:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_caller_concurrency": "10", 2024-09-17T07:28:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_collect_device_partition_information": "true", 2024-09-17T07:28:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_commit_timeout": "600.000000", 2024-09-17T07:28:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_debug_inject_read_err": "false", 2024-09-17T07:28:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_debug_omap_check": "false", 2024-09-17T07:28:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_debug_verify_split": "false", 2024-09-17T07:28:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_dump_file": "", 2024-09-17T07:28:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-17T07:28:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-17T07:28:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_fadvise": "true", 2024-09-17T07:28:41.419 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_fail_eio": "true", 2024-09-17T07:28:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_fd_cache_shards": "16", 2024-09-17T07:28:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_fd_cache_size": "128", 2024-09-17T07:28:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_fiemap": "false", 2024-09-17T07:28:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_fiemap_threshold": "4096", 2024-09-17T07:28:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-17T07:28:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-17T07:28:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_inject_stall": "0", 2024-09-17T07:28:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_journal_parallel": "false", 2024-09-17T07:28:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_journal_trailing": "false", 2024-09-17T07:28:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_journal_writeahead": "false", 2024-09-17T07:28:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_kill_at": "0", 2024-09-17T07:28:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-17T07:28:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-17T07:28:41.420 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-17T07:28:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-17T07:28:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-17T07:28:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattrs": "0", 2024-09-17T07:28:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-17T07:28:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-17T07:28:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-17T07:28:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-17T07:28:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_xattr_value_size": "0", 2024-09-17T07:28:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-17T07:28:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-17T07:28:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-17T07:28:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_merge_threshold": "-10", 2024-09-17T07:28:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-17T07:28:41.421 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_odsync_write": "false", 2024-09-17T07:28:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_omap_backend": "rocksdb", 2024-09-17T07:28:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_omap_backend_path": "", 2024-09-17T07:28:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-17T07:28:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-17T07:28:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-17T07:28:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_op_thread_timeout": "60", 2024-09-17T07:28:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_op_threads": "2", 2024-09-17T07:28:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_punch_hole": "false", 2024-09-17T07:28:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-17T07:28:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-17T07:28:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-17T07:28:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-17T07:28:41.422 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-17T07:28:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-17T07:28:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-17T07:28:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-17T07:28:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-17T07:28:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_queue_max_ops": "50", 2024-09-17T07:28:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-17T07:28:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_seek_data_hole": "false", 2024-09-17T07:28:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_sloppy_crc": "false", 2024-09-17T07:28:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-17T07:28:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_splice": "false", 2024-09-17T07:28:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_split_multiple": "2", 2024-09-17T07:28:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_split_rand_factor": "20", 2024-09-17T07:28:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_update_to": "1000", 2024-09-17T07:28:41.423 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-17T07:28:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-17T07:28:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-17T07:28:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-17T07:28:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-17T07:28:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-17T07:28:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_enable": "true", 2024-09-17T07:28:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-17T07:28:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-17T07:28:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-17T07:28:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-17T07:28:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-17T07:28:41.424 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-17T07:28:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_xfs_extsize": "false", 2024-09-17T07:28:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "filestore_zfs_snap": "false", 2024-09-17T07:28:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "fio_dir": "/tmp/fio", 2024-09-17T07:28:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "b283e6d5-0f4a-4271-a443-15e695e35364", 2024-09-17T07:28:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_allow_other": "true", 2024-09-17T07:28:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_atomic_o_trunc": "true", 2024-09-17T07:28:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_big_writes": "true", 2024-09-17T07:28:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_debug": "false", 2024-09-17T07:28:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_default_permissions": "false", 2024-09-17T07:28:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_disable_pagecache": "false", 2024-09-17T07:28:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_max_write": "0", 2024-09-17T07:28:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_multithreaded": "true", 2024-09-17T07:28:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_require_active_mds": "true", 2024-09-17T07:28:41.425 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_set_user_groups": "true", 2024-09-17T07:28:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_splice_move": "true", 2024-09-17T07:28:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_splice_read": "true", 2024-09-17T07:28:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_splice_write": "true", 2024-09-17T07:28:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-17T07:28:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "fuse_use_invalidate_cb": "true", 2024-09-17T07:28:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-17T07:28:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "gss_target_name": "ceph", 2024-09-17T07:28:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_file": "", 2024-09-17T07:28:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_inject_failure": "0", 2024-09-17T07:28:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_interval": "5", 2024-09-17T07:28:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "host": "smithi045", 2024-09-17T07:28:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-17T07:28:41.426 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-17T07:28:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-17T07:28:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-17T07:28:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-17T07:28:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-17T07:28:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-17T07:28:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-17T07:28:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-17T07:28:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-17T07:28:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-17T07:28:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-17T07:28:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-17T07:28:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "inject_early_sigterm": "false", 2024-09-17T07:28:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "jaeger_agent_port": "6799", 2024-09-17T07:28:41.427 INFO:teuthology.orchestra.run.smithi045.stdout: "jaeger_tracing_enable": "false", 2024-09-17T07:28:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_aio": "true", 2024-09-17T07:28:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_align_min_size": "65536", 2024-09-17T07:28:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_block_align": "true", 2024-09-17T07:28:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_block_size": "4096", 2024-09-17T07:28:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_dio": "true", 2024-09-17T07:28:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_discard": "false", 2024-09-17T07:28:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_force_aio": "false", 2024-09-17T07:28:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_ignore_corruption": "false", 2024-09-17T07:28:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_max_write_bytes": "10485760", 2024-09-17T07:28:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_max_write_entries": "100", 2024-09-17T07:28:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_replay_from": "0", 2024-09-17T07:28:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-17T07:28:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-17T07:28:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-17T07:28:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-17T07:28:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_write_header_frequency": "0", 2024-09-17T07:28:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "journal_zero_on_create": "false", 2024-09-17T07:28:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "journaler_prefetch_periods": "10", 2024-09-17T07:28:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "journaler_prezero_periods": "5", 2024-09-17T07:28:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "journaler_write_head_interval": "15", 2024-09-17T07:28:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "", 2024-09-17T07:28:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "keyfile": "", 2024-09-17T07:28:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-17T07:28:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_backend": "rocksdb", 2024-09-17T07:28:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_default_stripe_size": "65536", 2024-09-17T07:28:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_fsck_on_mount": "false", 2024-09-17T07:28:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-17T07:28:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_max_bytes": "67108864", 2024-09-17T07:28:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_max_ops": "512", 2024-09-17T07:28:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_nid_prealloc": "1024", 2024-09-17T07:28:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_onode_map_size": "1024", 2024-09-17T07:28:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-17T07:28:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_sync_submit_transaction": "false", 2024-09-17T07:28:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "kstore_sync_transaction": "false", 2024-09-17T07:28:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "librados_thread_count": "2", 2024-09-17T07:28:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "lockdep": "false", 2024-09-17T07:28:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "lockdep_force_backtrace": "false", 2024-09-17T07:28:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "log_coarse_timestamps": "true", 2024-09-17T07:28:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-17T07:28:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "log_flush_on_exit": "false", 2024-09-17T07:28:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_graylog_host": "127.0.0.1", 2024-09-17T07:28:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_graylog_port": "12201", 2024-09-17T07:28:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_max_new": "1000", 2024-09-17T07:28:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_max_recent": "10000", 2024-09-17T07:28:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_stderr_prefix": "", 2024-09-17T07:28:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_stop_at_utilization": "0.970000", 2024-09-17T07:28:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": "true", 2024-09-17T07:28:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_graylog": "false", 2024-09-17T07:28:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_journald": "false", 2024-09-17T07:28:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_stderr": "false", 2024-09-17T07:28:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_syslog": "false", 2024-09-17T07:28:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "max_rotating_auth_attempts": "10", 2024-09-17T07:28:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-17T07:28:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_action_on_write_error": "1", 2024-09-17T07:28:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_alternate_name_max": "8192", 2024-09-17T07:28:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_asio_thread_count": "2", 2024-09-17T07:28:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_export_pin": "true", 2024-09-17T07:28:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_fragment_dirs": "true", 2024-09-17T07:28:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-17T07:28:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_fragment_interval": "5", 2024-09-17T07:28:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-17T07:28:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-17T07:28:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_interval": "10", 2024-09-17T07:28:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_max": "-1", 2024-09-17T07:28:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_max_until": "-1", 2024-09-17T07:28:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_merge_size": "50", 2024-09-17T07:28:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_midchunk": "0.300000", 2024-09-17T07:28:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-17T07:28:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_min_start": "0.200000", 2024-09-17T07:28:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_minchunk": "0.001000", 2024-09-17T07:28:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_mode": "0", 2024-09-17T07:28:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_need_max": "1.200000", 2024-09-17T07:28:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_need_min": "0.800000", 2024-09-17T07:28:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_overload_epochs": "2", 2024-09-17T07:28:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-17T07:28:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-17T07:28:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_split_bits": "3", 2024-09-17T07:28:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-17T07:28:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_split_size": "10000", 2024-09-17T07:28:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-17T07:28:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_target_decay": "10.000000", 2024-09-17T07:28:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-17T07:28:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_beacon_grace": "15.000000", 2024-09-17T07:28:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_beacon_interval": "4.000000", 2024-09-17T07:28:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-17T07:28:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-17T07:28:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_mid": "0.700000", 2024-09-17T07:28:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-17T07:28:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_quiesce_delay": "0", 2024-09-17T07:28:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-17T07:28:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-17T07:28:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-17T07:28:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_release_free_interval": "10", 2024-09-17T07:28:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_reservation": "0.050000", 2024-09-17T07:28:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-17T07:28:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_trim_interval": "1", 2024-09-17T07:28:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache_trim_threshold": "262144", 2024-09-17T07:28:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-17T07:28:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-17T07:28:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-17T07:28:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_client_prealloc_inos": "1000", 2024-09-17T07:28:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-17T07:28:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_connect_bootstrapping": "false", 2024-09-17T07:28:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_damage_table_max_entries": "10000", 2024-09-17T07:28:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-17T07:28:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_debug_auth_pins": "false", 2024-09-17T07:28:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_debug_frag": "false", 2024-09-17T07:28:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_debug_scatterstat": "false", 2024-09-17T07:28:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_debug_subtrees": "false", 2024-09-17T07:28:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_decay_halflife": "5.000000", 2024-09-17T07:28:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_default_dir_hash": "2", 2024-09-17T07:28:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_defer_session_stale": "true", 2024-09-17T07:28:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_deny_all_reconnect": "false", 2024-09-17T07:28:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dir_keys_per_op": "16384", 2024-09-17T07:28:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dir_max_commit_size": "10", 2024-09-17T07:28:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dir_max_entries": "0", 2024-09-17T07:28:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dir_prefetch": "true", 2024-09-17T07:28:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-17T07:28:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-17T07:28:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dump_cache_on_map": "false", 2024-09-17T07:28:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-17T07:28:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-17T07:28:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_early_reply": "true", 2024-09-17T07:28:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_enable_op_tracker": "true", 2024-09-17T07:28:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_enforce_unique_name": "true", 2024-09-17T07:28:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-17T07:28:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-17T07:28:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_export_ephemeral_random": "true", 2024-09-17T07:28:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-17T07:28:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-17T07:28:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-17T07:28:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-17T07:28:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-17T07:28:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-17T07:28:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-17T07:28:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-17T07:28:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_health_summarize_threshold": "10", 2024-09-17T07:28:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-17T07:28:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-17T07:28:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_inject_health_dummy": "false", 2024-09-17T07:28:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-17T07:28:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_inject_migrator_session_race": "false", 2024-09-17T07:28:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-17T07:28:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-17T07:28:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-17T07:28:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_join_fs": "", 2024-09-17T07:28:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_journal_format": "1", 2024-09-17T07:28:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-17T07:28:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_create_at": "0", 2024-09-17T07:28:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_dirfrag_at": "0", 2024-09-17T07:28:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_export_at": "0", 2024-09-17T07:28:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_import_at": "0", 2024-09-17T07:28:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_journal_at": "0", 2024-09-17T07:28:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_journal_expire_at": "0", 2024-09-17T07:28:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_journal_replay_at": "0", 2024-09-17T07:28:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_link_at": "0", 2024-09-17T07:28:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_mdstable_at": "0", 2024-09-17T07:28:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_openc_at": "0", 2024-09-17T07:28:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_rename_at": "0", 2024-09-17T07:28:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_kill_shutdown_at": "0", 2024-09-17T07:28:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_event_large_threshold": "524288", 2024-09-17T07:28:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_events_per_segment": "1024", 2024-09-17T07:28:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-17T07:28:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_max_events": "-1", 2024-09-17T07:28:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_max_segments": "128", 2024-09-17T07:28:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_pause": "false", 2024-09-17T07:28:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_segment_size": "0", 2024-09-17T07:28:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-17T07:28:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-17T07:28:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-17T07:28:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_trim_threshold": "128", 2024-09-17T07:28:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-17T07:28:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log_warn_factor": "2.000000", 2024-09-17T07:28:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_caps_per_client": "1048576", 2024-09-17T07:28:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_completed_flushes": "100000", 2024-09-17T07:28:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_completed_requests": "100000", 2024-09-17T07:28:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_export_size": "20971520", 2024-09-17T07:28:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_file_recover": "32", 2024-09-17T07:28:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_purge_files": "64", 2024-09-17T07:28:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_purge_ops": "8192", 2024-09-17T07:28:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-17T07:28:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-17T07:28:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_max_snaps_per_dir": "100", 2024-09-17T07:28:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_metrics_update_interval": "2", 2024-09-17T07:28:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_min_caps_per_client": "100", 2024-09-17T07:28:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_min_caps_working_set": "10000", 2024-09-17T07:28:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-17T07:28:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_numa_node": "-1", 2024-09-17T07:28:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-17T07:28:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_op_complaint_time": "30.000000", 2024-09-17T07:28:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_op_history_duration": "600", 2024-09-17T07:28:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_op_history_size": "20", 2024-09-17T07:28:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_op_history_slow_op_size": "20", 2024-09-17T07:28:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-17T07:28:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_op_log_threshold": "5", 2024-09-17T07:28:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_ping_grace": "15", 2024-09-17T07:28:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_ping_interval": "5", 2024-09-17T07:28:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-17T07:28:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-17T07:28:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_recall_max_caps": "30000", 2024-09-17T07:28:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-17T07:28:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-17T07:28:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-17T07:28:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_recall_warning_threshold": "262144", 2024-09-17T07:28:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-17T07:28:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_replay_interval": "1.000000", 2024-09-17T07:28:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-17T07:28:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-17T07:28:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_root_ino_gid": "0", 2024-09-17T07:28:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_root_ino_uid": "0", 2024-09-17T07:28:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-17T07:28:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_scrub_stats_review_period": "1", 2024-09-17T07:28:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-17T07:28:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-17T07:28:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-17T07:28:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-17T07:28:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-17T07:28:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-17T07:28:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-17T07:28:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-17T07:28:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-17T07:28:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-17T07:28:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-17T07:28:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_shutdown_check": "0", 2024-09-17T07:28:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_skip_ino": "0", 2024-09-17T07:28:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-17T07:28:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_snap_max_uid": "4294967294", 2024-09-17T07:28:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_snap_min_uid": "0", 2024-09-17T07:28:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_snap_rstat": "false", 2024-09-17T07:28:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_standby_replay_damaged": "false", 2024-09-17T07:28:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_symlink_recovery": "true", 2024-09-17T07:28:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-17T07:28:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_thrash_exports": "0", 2024-09-17T07:28:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_thrash_fragments": "0", 2024-09-17T07:28:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_tick_interval": "5.000000", 2024-09-17T07:28:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_valgrind_exit": "false", 2024-09-17T07:28:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_verify_backtrace": "1", 2024-09-17T07:28:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_verify_scatter": "false", 2024-09-17T07:28:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-17T07:28:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_wipe_sessions": "false", 2024-09-17T07:28:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool_debug": "false", 2024-09-17T07:28:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-17T07:28:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "memstore_device_bytes": "1073741824", 2024-09-17T07:28:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "memstore_page_set": "false", 2024-09-17T07:28:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "memstore_page_size": "65536", 2024-09-17T07:28:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_client_bytes": "134217728", 2024-09-17T07:28:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_client_messages": "512", 2024-09-17T07:28:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-17T07:28:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-17T07:28:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-17T07:28:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-17T07:28:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_disabled_modules": "", 2024-09-17T07:28:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_enable_op_tracker": "true", 2024-09-17T07:28:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-17T07:28:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_max_pg_creating": "1024", 2024-09-17T07:28:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_max_pg_num_change": "128", 2024-09-17T07:28:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_mds_bytes": "134217728", 2024-09-17T07:28:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_mds_messages": "128", 2024-09-17T07:28:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-17T07:28:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_mon_bytes": "134217728", 2024-09-17T07:28:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_mon_messages": "128", 2024-09-17T07:28:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-17T07:28:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-17T07:28:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_op_history_duration": "600", 2024-09-17T07:28:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_op_history_size": "20", 2024-09-17T07:28:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-17T07:28:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-17T07:28:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_op_log_threshold": "5", 2024-09-17T07:28:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_osd_bytes": "536870912", 2024-09-17T07:28:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_osd_messages": "8192", 2024-09-17T07:28:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_pool": "true", 2024-09-17T07:28:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-17T07:28:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_standby_modules": "true", 2024-09-17T07:28:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_stats_period": "5", 2024-09-17T07:28:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_stats_threshold": "5", 2024-09-17T07:28:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_tick_period": "2", 2024-09-17T07:28:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-17T07:28:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-17T07:28:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_allow_pool_delete": "true", 2024-09-17T07:28:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_allow_pool_size_one": "true", 2024-09-17T07:28:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_auth_validate_all_caps": "true", 2024-09-17T07:28:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-17T07:28:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-17T07:28:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_bytes": "104857600", 2024-09-17T07:28:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_directed_command_retry": "5", 2024-09-17T07:28:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-17T07:28:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-17T07:28:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-17T07:28:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-17T07:28:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_hunt_parallel": "3", 2024-09-17T07:28:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_log_interval": "1.000000", 2024-09-17T07:28:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-17T07:28:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_ping_interval": "10.000000", 2024-09-17T07:28:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-17T07:28:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_client_target_rank": "-1", 2024-09-17T07:28:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-17T07:28:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-17T07:28:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-17T07:28:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_level": "debug", 2024-09-17T07:28:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_file": "true", 2024-09-17T07:28:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-17T07:28:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-17T07:28:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-17T07:28:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_journald": "false", 2024-09-17T07:28:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-17T07:28:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-17T07:28:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-17T07:28:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_compact_on_bootstrap": "false", 2024-09-17T07:28:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_compact_on_start": "false", 2024-09-17T07:28:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_compact_on_trim": "true", 2024-09-17T07:28:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-17T07:28:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-17T07:28:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-17T07:28:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_cpu_threads": "4", 2024-09-17T07:28:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_crush_min_required_version": "hammer", 2024-09-17T07:28:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_daemon_bytes": "419430400", 2024-09-17T07:28:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-17T07:28:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_data_avail_crit": "5", 2024-09-17T07:28:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_data_avail_warn": "5", 2024-09-17T07:28:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_data_size_warn": "16106127360", 2024-09-17T07:28:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-17T07:28:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-17T07:28:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_dump_json": "false", 2024-09-17T07:28:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-17T07:28:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_dump_transactions": "false", 2024-09-17T07:28:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_extra_checks": "false", 2024-09-17T07:28:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-17T07:28:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-17T07:28:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_no_require_reef": "false", 2024-09-17T07:28:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_no_require_squid": "false", 2024-09-17T07:28:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-17T07:28:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-17T07:28:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-17T07:28:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_down_mkfs_grace": "120", 2024-09-17T07:28:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_down_uptime_grace": "60", 2024-09-17T07:28:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_election_default_strategy": "1", 2024-09-17T07:28:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_election_timeout": "5.000000", 2024-09-17T07:28:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-17T07:28:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_elector_ping_divisor": "2", 2024-09-17T07:28:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-17T07:28:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_enable_op_tracker": "true", 2024-09-17T07:28:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_fake_pool_delete": "false", 2024-09-17T07:28:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_force_quorum_join": "false", 2024-09-17T07:28:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-17T07:28:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_globalid_prealloc": "10000", 2024-09-17T07:28:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_health_detail_to_clog": "false", 2024-09-17T07:28:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_health_log_update_period": "5", 2024-09-17T07:28:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_health_max_detail": "50", 2024-09-17T07:28:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_health_to_clog": "true", 2024-09-17T07:28:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_health_to_clog_interval": "600", 2024-09-17T07:28:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-17T07:28:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_host": "172.21.15.45,[v2:172.21.15.45:3301,v1:172.21.15.45:6790],172.21.15.192", 2024-09-17T07:28:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_host_override": "", 2024-09-17T07:28:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_initial_members": "", 2024-09-17T07:28:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-17T07:28:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-17T07:28:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-17T07:28:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-17T07:28:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-17T07:28:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_lease": "5.000000", 2024-09-17T07:28:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-17T07:28:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-17T07:28:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_log_full_interval": "50", 2024-09-17T07:28:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_log_max": "10000", 2024-09-17T07:28:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_log_max_summary": "50", 2024-09-17T07:28:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-17T07:28:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_log_epochs": "500", 2024-09-17T07:28:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-17T07:28:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-17T07:28:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-17T07:28:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_osd": "10000", 2024-09-17T07:28:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_pg_per_osd": "10000", 2024-09-17T07:28:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_pool_pg_num": "65536", 2024-09-17T07:28:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-17T07:28:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-17T07:28:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mds_force_trim_to": "0", 2024-09-17T07:28:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mds_skip_sanity": "false", 2024-09-17T07:28:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_memory_autotune": "true", 2024-09-17T07:28:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_memory_target": "2147483648", 2024-09-17T07:28:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mgr_beacon_grace": "30", 2024-09-17T07:28:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-17T07:28:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mgr_digest_period": "5", 2024-09-17T07:28:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mgr_inactive_grace": "60", 2024-09-17T07:28:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-17T07:28:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-17T07:28:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_min_osdmap_epochs": "500", 2024-09-17T07:28:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-17T07:28:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-17T07:28:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_op_complaint_time": "30", 2024-09-17T07:28:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_op_history_duration": "600", 2024-09-17T07:28:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_op_history_size": "20", 2024-09-17T07:28:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_op_history_slow_op_size": "20", 2024-09-17T07:28:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-17T07:28:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_op_log_threshold": "5", 2024-09-17T07:28:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-17T07:28:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-17T07:28:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-17T07:28:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_auto_mark_in": "false", 2024-09-17T07:28:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-17T07:28:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-17T07:28:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-17T07:28:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_cache_size": "500", 2024-09-17T07:28:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-17T07:28:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-17T07:28:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-17T07:28:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_down_out_interval": "600", 2024-09-17T07:28:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-17T07:28:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-17T07:28:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_force_trim_to": "0", 2024-09-17T07:28:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-17T07:28:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-17T07:28:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-17T07:28:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-17T07:28:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-17T07:28:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-17T07:28:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-17T07:28:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_min_down_reporters": "2", 2024-09-17T07:28:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-17T07:28:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-17T07:28:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-17T07:28:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-17T07:28:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-17T07:28:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-17T07:28:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_report_timeout": "900", 2024-09-17T07:28:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-17T07:28:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-17T07:28:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-17T07:28:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-17T07:28:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-17T07:28:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-17T07:28:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-17T07:28:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-17T07:28:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-17T07:28:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pg_stuck_threshold": "60", 2024-09-17T07:28:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-17T07:28:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-17T07:28:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-17T07:28:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-17T07:28:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-17T07:28:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-17T07:28:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_probe_timeout": "2.000000", 2024-09-17T07:28:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_reweight_max_change": "0.050000", 2024-09-17T07:28:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_reweight_max_osds": "4", 2024-09-17T07:28:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-17T07:28:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-17T07:28:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-17T07:28:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-17T07:28:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-17T07:28:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_scrub_interval": "86400", 2024-09-17T07:28:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_scrub_max_keys": "100", 2024-09-17T07:28:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_scrub_timeout": "300", 2024-09-17T07:28:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_session_timeout": "300", 2024-09-17T07:28:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_smart_report_timeout": "5", 2024-09-17T07:28:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_stat_smooth_intervals": "6", 2024-09-17T07:28:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-17T07:28:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_stretch_pool_min_size": "2", 2024-09-17T07:28:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_stretch_pool_size": "4", 2024-09-17T07:28:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-17T07:28:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-17T07:28:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_sync_debug": "false", 2024-09-17T07:28:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-17T07:28:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-17T07:28:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_sync_provider_kill_at": "0", 2024-09-17T07:28:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_sync_requester_kill_at": "0", 2024-09-17T07:28:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_sync_timeout": "60.000000", 2024-09-17T07:28:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_target_pg_per_osd": "100", 2024-09-17T07:28:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_tick_interval": "5", 2024-09-17T07:28:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_timecheck_interval": "300.000000", 2024-09-17T07:28:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-17T07:28:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_use_min_delay_socket": "false", 2024-09-17T07:28:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_older_version_delay": "604800", 2024-09-17T07:28:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-17T07:28:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-17T07:28:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-17T07:28:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-17T07:28:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-17T07:28:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-17T07:28:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-17T07:28:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_misplaced": "false", 2024-09-17T07:28:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-17T07:28:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_older_version": "true", 2024-09-17T07:28:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-17T07:28:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-17T07:28:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-17T07:28:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-17T07:28:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-17T07:28:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-17T07:28:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-17T07:28:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-17T07:28:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-17T07:28:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "monmap": "", 2024-09-17T07:28:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T07:28:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-17T07:28:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-17T07:28:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T07:28:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-17T07:28:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-17T07:28:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "motr_tracing_enabled": "false", 2024-09-17T07:28:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_op_threads": "3", 2024-09-17T07:28:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-17T07:28:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_cm": "false", 2024-09-17T07:28:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_device_name": "", 2024-09-17T07:28:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_dscp": "96", 2024-09-17T07:28:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-17T07:28:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-17T07:28:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_local_gid": "", 2024-09-17T07:28:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-17T07:28:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_port_num": "1", 2024-09-17T07:28:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-17T07:28:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-17T07:28:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-17T07:28:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-17T07:28:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_sl": "3", 2024-09-17T07:28:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_support_srq": "true", 2024-09-17T07:28:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_rdma_type": "ib", 2024-09-17T07:28:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_async_reap_threshold": "5", 2024-09-17T07:28:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_before_connect": "false", 2024-09-17T07:28:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_ipv4": "true", 2024-09-17T07:28:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_ipv6": "false", 2024-09-17T07:28:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_msgr1": "true", 2024-09-17T07:28:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_msgr2": "true", 2024-09-17T07:28:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_port_max": "7568", 2024-09-17T07:28:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_port_min": "6800", 2024-09-17T07:28:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-17T07:28:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_retry_count": "3", 2024-09-17T07:28:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_bind_retry_delay": "5", 2024-09-17T07:28:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_blackhole_client": "false", 2024-09-17T07:28:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_blackhole_mds": "false", 2024-09-17T07:28:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_blackhole_mgr": "false", 2024-09-17T07:28:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_blackhole_mon": "false", 2024-09-17T07:28:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_blackhole_osd": "false", 2024-09-17T07:28:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_client_mode": "crc secure", 2024-09-17T07:28:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-17T07:28:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_cluster_mode": "crc secure", 2024-09-17T07:28:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_cluster_type": "", 2024-09-17T07:28:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_compress_secure": "false", 2024-09-17T07:28:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_connection_idle_timeout": "900", 2024-09-17T07:28:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_connection_ready_timeout": "10", 2024-09-17T07:28:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_crc_data": "true", 2024-09-17T07:28:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_crc_header": "true", 2024-09-17T07:28:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_die_on_bad_msg": "false", 2024-09-17T07:28:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_die_on_bug": "true", 2024-09-17T07:28:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_die_on_old_message": "true", 2024-09-17T07:28:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_die_on_skipped_message": "false", 2024-09-17T07:28:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-17T07:28:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-17T07:28:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_coremask": "0xF", 2024-09-17T07:28:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-17T07:28:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-17T07:28:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_enable_tso": "true", 2024-09-17T07:28:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-17T07:28:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-17T07:28:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_hugepages": "", 2024-09-17T07:28:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-17T07:28:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-17T07:28:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_lro": "true", 2024-09-17T07:28:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_memory_channel": "4", 2024-09-17T07:28:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-17T07:28:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_pmd": "", 2024-09-17T07:28:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_port_id": "0", 2024-09-17T07:28:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-17T07:28:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-17T07:28:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_dump_on_send": "false", 2024-09-17T07:28:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_initial_backoff": "0.200000", 2024-09-17T07:28:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_inject_delay_max": "1.000000", 2024-09-17T07:28:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-17T07:28:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_inject_delay_type": "", 2024-09-17T07:28:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-17T07:28:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_inject_network_congestion": "0", 2024-09-17T07:28:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_inject_socket_failures": "5000", 2024-09-17T07:28:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_learn_addr_from_peer": "true", 2024-09-17T07:28:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_max_accept_failures": "4", 2024-09-17T07:28:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_max_backoff": "15.000000", 2024-09-17T07:28:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_mon_client_mode": "secure crc", 2024-09-17T07:28:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-17T07:28:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_mon_service_mode": "secure crc", 2024-09-17T07:28:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_osd_compress_min_size": "1024", 2024-09-17T07:28:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_osd_compress_mode": "none", 2024-09-17T07:28:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-17T07:28:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-17T07:28:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_pq_min_cost": "65536", 2024-09-17T07:28:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_public_type": "", 2024-09-17T07:28:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_service_mode": "crc secure", 2024-09-17T07:28:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_tcp_listen_backlog": "512", 2024-09-17T07:28:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_tcp_nodelay": "true", 2024-09-17T07:28:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-17T07:28:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_tcp_rcvbuf": "0", 2024-09-17T07:28:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-17T07:28:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "ms_type": "async+posix", 2024-09-17T07:28:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "no_config_file": "false", 2024-09-17T07:28:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-17T07:28:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-17T07:28:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_completion_locks_per_session": "32", 2024-09-17T07:28:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-17T07:28:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-17T07:28:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_inflight_ops": "1024", 2024-09-17T07:28:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-17T07:28:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-17T07:28:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_tick_interval": "5.000000", 2024-09-17T07:28:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter_timeout": "10.000000", 2024-09-17T07:28:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "objectstore_blackhole": "false", 2024-09-17T07:28:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-17T07:28:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "openssl_engine_opts": "", 2024-09-17T07:28:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_agent_delay_time": "5.000000", 2024-09-17T07:28:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_agent_hist_halflife": "1000", 2024-09-17T07:28:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_agent_max_low_ops": "2", 2024-09-17T07:28:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_agent_max_ops": "4", 2024-09-17T07:28:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-17T07:28:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-17T07:28:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_agent_slop": "0.020000", 2024-09-17T07:28:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-17T07:28:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-17T07:28:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_asio_thread_count": "2", 2024-09-17T07:28:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_async_recovery_min_cost": "100", 2024-09-17T07:28:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-17T07:28:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-17T07:28:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_backfill_scan_max": "512", 2024-09-17T07:28:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_backfill_scan_min": "64", 2024-09-17T07:28:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_backoff_on_degraded": "false", 2024-09-17T07:28:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_backoff_on_peering": "false", 2024-09-17T07:28:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_backoff_on_unfound": "true", 2024-09-17T07:28:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_beacon_report_interval": "300", 2024-09-17T07:28:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bench_duration": "30", 2024-09-17T07:28:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-17T07:28:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bench_max_block_size": "67108864", 2024-09-17T07:28:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-17T07:28:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_blkin_trace_all": "false", 2024-09-17T07:28:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-17T07:28:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-17T07:28:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-17T07:28:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-17T07:28:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_check_for_log_corruption": "false", 2024-09-17T07:28:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-17T07:28:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T07:28:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-17T07:28:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T07:28:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_class_update_on_start": "true", 2024-09-17T07:28:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_client_message_cap": "256", 2024-09-17T07:28:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_client_message_size_cap": "524288000", 2024-09-17T07:28:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_client_op_priority": "63", 2024-09-17T07:28:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_client_watch_timeout": "30", 2024-09-17T07:28:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_command_max_records": "256", 2024-09-17T07:28:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_compact_on_start": "false", 2024-09-17T07:28:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-17T07:28:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-17T07:28:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-17T07:28:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_crush_update_on_start": "true", 2024-09-17T07:28:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_crush_update_weight_set": "true", 2024-09-17T07:28:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-17T07:28:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-17T07:28:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-17T07:28:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-17T07:28:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-17T07:28:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_feed_pullee": "-1", 2024-09-17T07:28:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-17T07:28:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-17T07:28:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-17T07:28:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_misdirected_ops": "false", 2024-09-17T07:28:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_no_acting_change": "false", 2024-09-17T07:28:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_no_purge_strays": "false", 2024-09-17T07:28:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_op_order": "false", 2024-09-17T07:28:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-17T07:28:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-17T07:28:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-17T07:28:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-17T07:28:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_shutdown": "false", 2024-09-17T07:28:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-17T07:28:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-17T07:28:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_trim_objects": "false", 2024-09-17T07:28:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-17T07:28:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-17T07:28:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_verify_snaps": "false", 2024-09-17T07:28:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-17T07:28:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-17T07:28:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_deep_scrub_keys": "1024", 2024-09-17T07:28:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-17T07:28:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-17T07:28:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-17T07:28:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_deep_scrub_stride": "524288", 2024-09-17T07:28:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-17T07:28:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-17T07:28:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_default_notify_timeout": "30", 2024-09-17T07:28:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_delete_sleep": "0.000000", 2024-09-17T07:28:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-17T07:28:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-17T07:28:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-17T07:28:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_discard_disconnected_ops": "true", 2024-09-17T07:28:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_ec_partial_reads": "true", 2024-09-17T07:28:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_enable_op_tracker": "true", 2024-09-17T07:28:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-17T07:28:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-17T07:28:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-17T07:28:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-17T07:28:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_fast_info": "true", 2024-09-17T07:28:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_fast_shutdown": "true", 2024-09-17T07:28:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-17T07:28:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-17T07:28:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-17T07:28:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-17T07:28:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-17T07:28:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_function_tracing": "false", 2024-09-17T07:28:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_heartbeat_grace": "20", 2024-09-17T07:28:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_heartbeat_interval": "6", 2024-09-17T07:28:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-17T07:28:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_heartbeat_min_peers": "10", 2024-09-17T07:28:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_heartbeat_min_size": "2000", 2024-09-17T07:28:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_heartbeat_stale": "600", 2024-09-17T07:28:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-17T07:28:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_hit_set_max_size": "100000", 2024-09-17T07:28:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_hit_set_min_size": "1000", 2024-09-17T07:28:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-17T07:28:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-17T07:28:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-17T07:28:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-17T07:28:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-17T07:28:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-17T07:28:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_journal_size": "5120", 2024-09-17T07:28:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_kill_backfill_at": "0", 2024-09-17T07:28:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-17T07:28:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_map_cache_size": "50", 2024-09-17T07:28:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_map_dedup": "true", 2024-09-17T07:28:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_map_message_max": "40", 2024-09-17T07:28:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-17T07:28:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_map_share_max_epochs": "40", 2024-09-17T07:28:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_attr_name_len": "100", 2024-09-17T07:28:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_attr_size": "0", 2024-09-17T07:28:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_backfills": "1", 2024-09-17T07:28:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_markdown_count": "5", 2024-09-17T07:28:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_markdown_period": "600", 2024-09-17T07:28:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_object_name_len": "2048", 2024-09-17T07:28:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_object_namespace_len": "256", 2024-09-17T07:28:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_object_size": "134217728", 2024-09-17T07:28:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-17T07:28:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-17T07:28:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_pg_blocked_by": "16", 2024-09-17T07:28:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_pg_log_entries": "10000", 2024-09-17T07:28:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-17T07:28:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_pgls": "1024", 2024-09-17T07:28:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_push_cost": "8388608", 2024-09-17T07:28:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_push_objects": "10", 2024-09-17T07:28:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_scrubs": "3", 2024-09-17T07:28:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-17T07:28:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_trimming_pgs": "2", 2024-09-17T07:28:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_write_op_reply_len": "64", 2024-09-17T07:28:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_max_write_size": "90", 2024-09-17T07:28:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-17T07:28:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-17T07:28:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-17T07:28:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-17T07:28:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-17T07:28:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-17T07:28:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-17T07:28:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-17T07:28:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_profile": "balanced", 2024-09-17T07:28:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-17T07:28:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-17T07:28:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-17T07:28:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-17T07:28:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-17T07:28:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-17T07:28:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-17T07:28:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-17T07:28:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-17T07:28:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-17T07:28:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-17T07:28:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_memory_base": "805306368", 2024-09-17T07:28:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_memory_cache_min": "134217728", 2024-09-17T07:28:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-17T07:28:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-17T07:28:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_memory_target": "4294967296", 2024-09-17T07:28:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_memory_target_autotune": "false", 2024-09-17T07:28:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-17T07:28:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_min_pg_log_entries": "250", 2024-09-17T07:28:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_min_recovery_priority": "0", 2024-09-17T07:28:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-17T07:28:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-17T07:28:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mon_report_interval": "5", 2024-09-17T07:28:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-17T07:28:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-17T07:28:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_num_cache_shards": "32", 2024-09-17T07:28:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_num_op_tracker_shard": "32", 2024-09-17T07:28:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_numa_auto_affinity": "true", 2024-09-17T07:28:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_numa_node": "-1", 2024-09-17T07:28:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_numa_prefer_iface": "true", 2024-09-17T07:28:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-17T07:28:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_objecter_finishers": "1", 2024-09-17T07:28:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_objectstore": "bluestore", 2024-09-17T07:28:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_objectstore_fuse": "false", 2024-09-17T07:28:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-17T07:28:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_objectstore_tracing": "false", 2024-09-17T07:28:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_complaint_time": "30.000000", 2024-09-17T07:28:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_history_duration": "600", 2024-09-17T07:28:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_history_size": "20", 2024-09-17T07:28:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_history_slow_op_size": "20", 2024-09-17T07:28:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-17T07:28:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_log_threshold": "5", 2024-09-17T07:28:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_num_shards": "0", 2024-09-17T07:28:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_num_shards_hdd": "5", 2024-09-17T07:28:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_num_shards_ssd": "8", 2024-09-17T07:28:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-17T07:28:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-17T07:28:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-17T07:28:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-17T07:28:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_pq_min_cost": "65536", 2024-09-17T07:28:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-17T07:28:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_queue_cut_off": "high", 2024-09-17T07:28:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-17T07:28:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_op_thread_timeout": "15", 2024-09-17T07:28:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_open_classes_on_start": "true", 2024-09-17T07:28:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_os_flags": "0", 2024-09-17T07:28:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_peering_op_priority": "255", 2024-09-17T07:28:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_delete_cost": "1048576", 2024-09-17T07:28:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_delete_priority": "5", 2024-09-17T07:28:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-17T07:28:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-17T07:28:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-17T07:28:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_log_trim_max": "10000", 2024-09-17T07:28:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_log_trim_min": "100", 2024-09-17T07:28:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-17T07:28:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-17T07:28:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-17T07:28:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-17T07:28:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-17T07:28:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-17T07:28:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-17T07:28:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-17T07:28:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-17T07:28:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-17T07:28:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_crimson": "false", 2024-09-17T07:28:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-17T07:28:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-17T07:28:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-17T07:28:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-17T07:28:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-17T07:28:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-17T07:28:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-17T07:28:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-17T07:28:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_flags": "0", 2024-09-17T07:28:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-17T07:28:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_min_size": "0", 2024-09-17T07:28:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-17T07:28:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_pg_num": "32", 2024-09-17T07:28:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_pgp_num": "0", 2024-09-17T07:28:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-17T07:28:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_read_ratio": "70", 2024-09-17T07:28:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_size": "2", 2024-09-17T07:28:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_default_type": "replicated", 2024-09-17T07:28:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-17T07:28:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-17T07:28:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_push_per_object_cost": "1000", 2024-09-17T07:28:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-17T07:28:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recover_clone_overlap": "true", 2024-09-17T07:28:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-17T07:28:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_cost": "20971520", 2024-09-17T07:28:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-17T07:28:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_max_active": "0", 2024-09-17T07:28:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-17T07:28:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-17T07:28:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-17T07:28:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-17T07:28:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_max_single_start": "1", 2024-09-17T07:28:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_op_priority": "3", 2024-09-17T07:28:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-17T07:28:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_priority": "5", 2024-09-17T07:28:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-17T07:28:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_sleep": "0.000000", 2024-09-17T07:28:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-17T07:28:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-17T07:28:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-17T07:28:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_repair_during_recovery": "false", 2024-09-17T07:28:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_requested_scrub_priority": "120", 2024-09-17T07:28:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-17T07:28:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-17T07:28:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_auto_repair": "false", 2024-09-17T07:28:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-17T07:28:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-17T07:28:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_begin_hour": "0", 2024-09-17T07:28:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_begin_week_day": "0", 2024-09-17T07:28:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_chunk_max": "25", 2024-09-17T07:28:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_chunk_min": "5", 2024-09-17T07:28:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_cost": "52428800", 2024-09-17T07:28:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-17T07:28:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_during_recovery": "false", 2024-09-17T07:28:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_end_hour": "0", 2024-09-17T07:28:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_end_week_day": "0", 2024-09-17T07:28:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_event_cost": "4096", 2024-09-17T07:28:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-17T07:28:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-17T07:28:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_invalid_stats": "true", 2024-09-17T07:28:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-17T07:28:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-17T07:28:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_max_preemptions": "5", 2024-09-17T07:28:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-17T07:28:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_priority": "5", 2024-09-17T07:28:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_retry_after_noscrub": "60", 2024-09-17T07:28:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_retry_delay": "30", 2024-09-17T07:28:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_retry_new_interval": "10", 2024-09-17T07:28:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_retry_pg_state": "60", 2024-09-17T07:28:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_retry_trimming": "10", 2024-09-17T07:28:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_scrub_sleep": "0.000000", 2024-09-17T07:28:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-17T07:28:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-17T07:28:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-17T07:28:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-17T07:28:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_skip_data_digest": "false", 2024-09-17T07:28:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_smart_report_timeout": "5", 2024-09-17T07:28:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_snap_trim_cost": "1048576", 2024-09-17T07:28:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_snap_trim_priority": "5", 2024-09-17T07:28:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-17T07:28:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-17T07:28:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-17T07:28:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-17T07:28:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-17T07:28:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-17T07:28:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-17T07:28:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_target_transaction_size": "30", 2024-09-17T07:28:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-17T07:28:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-17T07:28:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-17T07:28:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-17T07:28:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-17T07:28:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-17T07:28:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-17T07:28:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-17T07:28:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-17T07:28:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-17T07:28:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_tracing": "false", 2024-09-17T07:28:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_use_stale_snap": "false", 2024-09-17T07:28:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-17T07:28:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "osdc_blkin_trace_all": "false", 2024-09-17T07:28:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_kill_at": "0", 2024-09-17T07:28:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_max_join_drift": "10", 2024-09-17T07:28:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_min": "500", 2024-09-17T07:28:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_min_wait": "0.050000", 2024-09-17T07:28:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_propose_interval": "1.000000", 2024-09-17T07:28:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_service_trim_max": "500", 2024-09-17T07:28:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-17T07:28:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_service_trim_min": "250", 2024-09-17T07:28:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_stash_full_interval": "25", 2024-09-17T07:28:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_trim_max": "500", 2024-09-17T07:28:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "paxos_trim_min": "250", 2024-09-17T07:28:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "perf": "true", 2024-09-17T07:28:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-17T07:28:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-17T07:28:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-17T07:28:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "-", 2024-09-17T07:28:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrv": "", 2024-09-17T07:28:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "public_bind_addr": "-", 2024-09-17T07:28:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "public_network": "", 2024-09-17T07:28:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "public_network_interface": "", 2024-09-17T07:28:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "qat_compressor_busy_polling": "false", 2024-09-17T07:28:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "qat_compressor_enabled": "false", 2024-09-17T07:28:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "qat_compressor_session_max_number": "256", 2024-09-17T07:28:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "rados_mon_op_timeout": "0", 2024-09-17T07:28:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "rados_osd_op_timeout": "0", 2024-09-17T07:28:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "rados_tracing": "false", 2024-09-17T07:28:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_atime_update_interval": "60", 2024-09-17T07:28:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-17T07:28:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_balance_parent_reads": "false", 2024-09-17T07:28:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_balance_snap_reads": "false", 2024-09-17T07:28:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_blkin_trace_all": "false", 2024-09-17T07:28:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-17T07:28:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-17T07:28:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache": "true", 2024-09-17T07:28:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-17T07:28:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-17T07:28:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-17T07:28:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-17T07:28:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_policy": "writearound", 2024-09-17T07:28:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_size": "33554432", 2024-09-17T07:28:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-17T07:28:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-17T07:28:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_clone_copy_on_read": "false", 2024-09-17T07:28:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_compression_hint": "none", 2024-09-17T07:28:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_concurrent_management_ops": "10", 2024-09-17T07:28:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-17T07:28:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_clone_format": "auto", 2024-09-17T07:28:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_data_pool": "", 2024-09-17T07:28:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_features": "61", 2024-09-17T07:28:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_format": "2", 2024-09-17T07:28:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_map_options": "", 2024-09-17T07:28:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_order": "22", 2024-09-17T07:28:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_pool": "rbd", 2024-09-17T07:28:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-17T07:28:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_stripe_count": "0", 2024-09-17T07:28:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_default_stripe_unit": "0", 2024-09-17T07:28:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-17T07:28:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-17T07:28:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-17T07:28:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_enable_alloc_hint": "true", 2024-09-17T07:28:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-17T07:28:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_io_scheduler": "simple", 2024-09-17T07:28:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-17T07:28:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-17T07:28:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-17T07:28:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-17T07:28:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-17T07:28:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-17T07:28:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-17T07:28:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-17T07:28:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-17T07:28:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_order": "24", 2024-09-17T07:28:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_pool": "", 2024-09-17T07:28:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_journal_splay_width": "4", 2024-09-17T07:28:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_localize_parent_reads": "false", 2024-09-17T07:28:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_localize_snap_reads": "false", 2024-09-17T07:28:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-17T07:28:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-17T07:28:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-17T07:28:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-17T07:28:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-17T07:28:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-17T07:28:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-17T07:28:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-17T07:28:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-17T07:28:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-17T07:28:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-17T07:28:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-17T07:28:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-17T07:28:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-17T07:28:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-17T07:28:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-17T07:28:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-17T07:28:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-17T07:28:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-17T07:28:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-17T07:28:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-17T07:28:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-17T07:28:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-17T07:28:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-17T07:28:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-17T07:28:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-17T07:28:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-17T07:28:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-17T07:28:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-17T07:28:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-17T07:28:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-17T07:28:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_mtime_update_interval": "60", 2024-09-17T07:28:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_non_blocking_aio": "true", 2024-09-17T07:28:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_op_thread_timeout": "60", 2024-09-17T07:28:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_op_threads": "1", 2024-09-17T07:28:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_parent_cache_enabled": "false", 2024-09-17T07:28:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-17T07:28:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-17T07:28:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-17T07:28:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_plugins": "", 2024-09-17T07:28:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_bps_burst": "0", 2024-09-17T07:28:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-17T07:28:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_bps_limit": "0", 2024-09-17T07:28:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_exclude_ops": "0", 2024-09-17T07:28:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_iops_burst": "0", 2024-09-17T07:28:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-17T07:28:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_iops_limit": "0", 2024-09-17T07:28:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-17T07:28:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-17T07:28:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-17T07:28:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-17T07:28:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-17T07:28:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-17T07:28:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-17T07:28:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-17T07:28:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-17T07:28:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-17T07:28:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-17T07:28:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-17T07:28:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-17T07:28:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-17T07:28:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_read_from_replica_policy": "default", 2024-09-17T07:28:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-17T07:28:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-17T07:28:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-17T07:28:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-17T07:28:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_skip_partial_discard": "true", 2024-09-17T07:28:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-17T07:28:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_tracing": "false", 2024-09-17T07:28:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_validate_names": "true", 2024-09-17T07:28:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_validate_pool": "true", 2024-09-17T07:28:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "restapi_base_url": "", 2024-09-17T07:28:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "restapi_log_level": "", 2024-09-17T07:28:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-17T07:28:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-17T07:28:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_acl_grants_max_num": "100", 2024-09-17T07:28:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_admin_entry": "admin", 2024-09-17T07:28:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-17T07:28:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_asio_assert_yielding": "false", 2024-09-17T07:28:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_backend_store": "rados", 2024-09-17T07:28:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_barbican_url": "", 2024-09-17T07:28:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_beast_enable_async": "true", 2024-09-17T07:28:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_counters_cache": "false", 2024-09-17T07:28:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-17T07:28:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-17T07:28:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-17T07:28:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-17T07:28:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-17T07:28:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-17T07:28:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-17T07:28:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-17T07:28:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_cache_enabled": "true", 2024-09-17T07:28:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_cache_expiry_interval": "900", 2024-09-17T07:28:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_cache_lru_size": "25000", 2024-09-17T07:28:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_config_store": "rados", 2024-09-17T07:28:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_content_length_compat": "false", 2024-09-17T07:28:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_copy_obj_progress": "true", 2024-09-17T07:28:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-17T07:28:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_cors_rules_max_num": "100", 2024-09-17T07:28:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_cross_domain_policy": "", 2024-09-17T07:28:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-17T07:28:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_addr": "", 2024-09-17T07:28:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-17T07:28:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-17T07:28:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-17T07:28:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-17T07:28:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_password": "", 2024-09-17T07:28:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-17T07:28:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_kmip_username": "", 2024-09-17T07:28:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_require_ssl": "true", 2024-09-17T07:28:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-17T07:28:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-17T07:28:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-17T07:28:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-17T07:28:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-17T07:28:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-17T07:28:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-17T07:28:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-17T07:28:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-17T07:28:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-17T07:28:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-17T07:28:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-17T07:28:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-17T07:28:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-17T07:28:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-17T07:28:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_addr": "", 2024-09-17T07:28:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_auth": "token", 2024-09-17T07:28:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_namespace": "", 2024-09-17T07:28:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_prefix": "", 2024-09-17T07:28:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-17T07:28:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-17T07:28:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-17T07:28:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-17T07:28:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_token_file": "", 2024-09-17T07:28:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-17T07:28:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_curl_buffersize": "524288", 2024-09-17T07:28:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-17T07:28:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_curl_low_speed_time": "300", 2024-09-17T07:28:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-17T07:28:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-17T07:28:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-17T07:28:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-17T07:28:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-17T07:28:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-17T07:28:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-17T07:28:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-17T07:28:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-17T07:28:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-17T07:28:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-17T07:28:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-17T07:28:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-17T07:28:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-17T07:28:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-17T07:28:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-17T07:28:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-17T07:28:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-17T07:28:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-17T07:28:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data_log_changes_size": "1000", 2024-09-17T07:28:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data_log_num_shards": "128", 2024-09-17T07:28:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-17T07:28:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data_log_window": "30", 2024-09-17T07:28:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-17T07:28:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-17T07:28:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-17T07:28:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-17T07:28:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-17T07:28:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-17T07:28:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-17T07:28:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-17T07:28:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-17T07:28:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-17T07:28:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-17T07:28:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-17T07:28:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-17T07:28:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_disable_s3select": "false", 2024-09-17T07:28:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-17T07:28:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-17T07:28:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-17T07:28:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-17T07:28:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-17T07:28:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-17T07:28:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-17T07:28:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-17T07:28:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-17T07:28:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-17T07:28:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-17T07:28:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-17T07:28:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dns_name": "", 2024-09-17T07:28:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dns_s3website_name": "", 2024-09-17T07:28:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dynamic_resharding": "true", 2024-09-17T07:28:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-17T07:28:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-17T07:28:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-17T07:28:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enable_gc_threads": "true", 2024-09-17T07:28:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enable_lc_threads": "true", 2024-09-17T07:28:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enable_mdsearch": "true", 2024-09-17T07:28:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enable_ops_log": "false", 2024-09-17T07:28:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enable_quota_threads": "true", 2024-09-17T07:28:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enable_static_website": "false", 2024-09-17T07:28:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enable_usage_log": "false", 2024-09-17T07:28:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_enforce_swift_acls": "true", 2024-09-17T07:28:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_exit_timeout_secs": "120", 2024-09-17T07:28:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_expose_bucket": "false", 2024-09-17T07:28:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_extended_http_attrs": "", 2024-09-17T07:28:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_filter": "none", 2024-09-17T07:28:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-17T07:28:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_frontends": "beast port=7480", 2024-09-17T07:28:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-17T07:28:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_max_deferred": "50", 2024-09-17T07:28:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-17T07:28:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_max_objs": "32", 2024-09-17T07:28:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-17T07:28:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-17T07:28:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-17T07:28:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-17T07:28:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_gc_processor_period": "3600", 2024-09-17T07:28:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-17T07:28:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-17T07:28:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-17T07:28:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-17T07:28:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_init_timeout": "300", 2024-09-17T07:28:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-17T07:28:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-17T07:28:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_kafka_connection_idle": "300", 2024-09-17T07:28:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-17T07:28:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-17T07:28:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-17T07:28:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-17T07:28:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-17T07:28:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_domain": "", 2024-09-17T07:28:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_password": "", 2024-09-17T07:28:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_password_path": "", 2024-09-17T07:28:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_project": "", 2024-09-17T07:28:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_tenant": "", 2024-09-17T07:28:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_token": "", 2024-09-17T07:28:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_token_path": "", 2024-09-17T07:28:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_admin_user": "", 2024-09-17T07:28:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_barbican_domain": "", 2024-09-17T07:28:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_barbican_password": "", 2024-09-17T07:28:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_barbican_project": "", 2024-09-17T07:28:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-17T07:28:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_barbican_user": "", 2024-09-17T07:28:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-17T07:28:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-17T07:28:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-17T07:28:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-17T07:28:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-17T07:28:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_url": "", 2024-09-17T07:28:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-17T07:28:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lc_debug_interval": "-1", 2024-09-17T07:28:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lc_lock_max_time": "90", 2024-09-17T07:28:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lc_max_objs": "32", 2024-09-17T07:28:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lc_max_rules": "1000", 2024-09-17T07:28:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lc_max_worker": "3", 2024-09-17T07:28:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-17T07:28:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lc_thread_delay": "0", 2024-09-17T07:28:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-17T07:28:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ldap_dnattr": "uid", 2024-09-17T07:28:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-17T07:28:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ldap_searchfilter": "", 2024-09-17T07:28:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-17T07:28:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ldap_uri": "", 2024-09-17T07:28:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-17T07:28:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-17T07:28:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-17T07:28:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-17T07:28:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_log_http_headers": "", 2024-09-17T07:28:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-17T07:28:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-17T07:28:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_log_object_name_utc": "false", 2024-09-17T07:28:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-17T07:28:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-17T07:28:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_attr_name_len": "0", 2024-09-17T07:28:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_attr_size": "0", 2024-09-17T07:28:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-17T07:28:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_chunk_size": "4194304", 2024-09-17T07:28:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-17T07:28:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-17T07:28:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-17T07:28:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_listing_results": "5000", 2024-09-17T07:28:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_notify_retries": "10", 2024-09-17T07:28:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-17T07:28:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_put_param_size": "1048576", 2024-09-17T07:28:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_put_size": "5368709120", 2024-09-17T07:28:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_max_slo_entries": "1000", 2024-09-17T07:28:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_md_log_max_shards": "64", 2024-09-17T07:28:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-17T07:28:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-17T07:28:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-17T07:28:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-17T07:28:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_mp_lock_max_time": "600", 2024-09-17T07:28:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-17T07:28:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-17T07:28:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-17T07:28:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-17T07:28:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-17T07:28:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-17T07:28:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-17T07:28:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-17T07:28:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_max_gc": "300", 2024-09-17T07:28:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-17T07:28:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-17T07:28:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-17T07:28:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-17T07:28:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-17T07:28:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-17T07:28:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-17T07:28:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_num_async_rados_threads": "32", 2024-09-17T07:28:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_num_control_oids": "8", 2024-09-17T07:28:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_numa_node": "-1", 2024-09-17T07:28:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-17T07:28:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-17T07:28:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_objexp_chunk_size": "100", 2024-09-17T07:28:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_objexp_gc_interval": "600", 2024-09-17T07:28:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-17T07:28:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-17T07:28:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-17T07:28:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_op_thread_timeout": "600", 2024-09-17T07:28:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_op_tracing": "false", 2024-09-17T07:28:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_opa_token": "", 2024-09-17T07:28:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_opa_url": "", 2024-09-17T07:28:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_opa_verify_ssl": "true", 2024-09-17T07:28:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-17T07:28:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-17T07:28:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ops_log_rados": "false", 2024-09-17T07:28:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_ops_log_socket_path": "", 2024-09-17T07:28:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-17T07:28:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-17T07:28:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-17T07:28:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_period_push_interval": "2.000000", 2024-09-17T07:28:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-17T07:28:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-17T07:28:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-17T07:28:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-17T07:28:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_posix_cache_lanes": "3", 2024-09-17T07:28:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-17T07:28:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-17T07:28:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_posix_cache_partitions": "3", 2024-09-17T07:28:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-17T07:28:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_print_continue": "true", 2024-09-17T07:28:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-17T07:28:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-17T07:28:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-17T07:28:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-17T07:28:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-17T07:28:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_rados_tracing": "false", 2024-09-17T07:28:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_realm": "", 2024-09-17T07:28:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_realm_id": "", 2024-09-17T07:28:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-17T07:28:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_region": "", 2024-09-17T07:28:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-17T07:28:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-17T07:28:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-17T07:28:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-17T07:28:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_request_uri": "", 2024-09-17T07:28:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_reshard_batch_size": "64", 2024-09-17T07:28:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-17T07:28:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-17T07:28:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_reshard_max_aio": "128", 2024-09-17T07:28:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_reshard_num_logs": "16", 2024-09-17T07:28:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-17T07:28:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-17T07:28:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_reshard_thread_interval": "600", 2024-09-17T07:28:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-17T07:28:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_resolve_cname": "false", 2024-09-17T07:28:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-17T07:28:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_run_sync_thread": "true", 2024-09-17T07:28:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-17T07:28:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-17T07:28:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-17T07:28:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-17T07:28:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-17T07:28:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-17T07:28:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-17T07:28:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-17T07:28:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-17T07:28:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_scheduler_type": "throttler", 2024-09-17T07:28:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_script_uri": "", 2024-09-17T07:28:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_service_provider_name": "", 2024-09-17T07:28:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-17T07:28:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sts_client_id": "", 2024-09-17T07:28:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sts_client_secret": "", 2024-09-17T07:28:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sts_entry": "sts", 2024-09-17T07:28:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sts_key": "sts", 2024-09-17T07:28:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-17T07:28:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sts_min_session_duration": "900", 2024-09-17T07:28:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sts_token_introspection_url": "", 2024-09-17T07:28:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_account_in_url": "false", 2024-09-17T07:28:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_auth_entry": "auth", 2024-09-17T07:28:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_auth_url": "", 2024-09-17T07:28:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_custom_header": "", 2024-09-17T07:28:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-17T07:28:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_need_stats": "true", 2024-09-17T07:28:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_tenant_name": "", 2024-09-17T07:28:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_token_expiration": "86400", 2024-09-17T07:28:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_url": "", 2024-09-17T07:28:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_url_prefix": "swift", 2024-09-17T07:28:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-17T07:28:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-17T07:28:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-17T07:28:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_lease_period": "120", 2024-09-17T07:28:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-17T07:28:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-17T07:28:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-17T07:28:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-17T07:28:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-17T07:28:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-17T07:28:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-17T07:28:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-17T07:28:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-17T07:28:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_thread_pool_size": "512", 2024-09-17T07:28:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-17T07:28:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-17T07:28:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-17T07:28:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-17T07:28:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_comment": "", 2024-09-17T07:28:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_createby": "", 2024-09-17T07:28:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_encoding": "", 2024-09-17T07:28:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_flag": "false", 2024-09-17T07:28:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-17T07:28:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_origin": "", 2024-09-17T07:28:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-17T07:28:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_torrent_tracker": "", 2024-09-17T07:28:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_trust_forwarded_https": "false", 2024-09-17T07:28:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-17T07:28:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-17T07:28:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_usage_max_shards": "32", 2024-09-17T07:28:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_usage_max_user_shards": "1", 2024-09-17T07:28:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_use_opa_authz": "false", 2024-09-17T07:28:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_counters_cache": "false", 2024-09-17T07:28:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-17T07:28:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-17T07:28:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-17T07:28:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_max_buckets": "1000", 2024-09-17T07:28:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_policies_max_num": "100", 2024-09-17T07:28:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-17T07:28:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-17T07:28:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-17T07:28:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-17T07:28:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_user_unique_email": "true", 2024-09-17T07:28:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_verify_ssl": "true", 2024-09-17T07:28:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-17T07:28:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_zone": "", 2024-09-17T07:28:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_zone_id": "", 2024-09-17T07:28:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-17T07:28:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_zonegroup": "", 2024-09-17T07:28:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_zonegroup_id": "", 2024-09-17T07:28:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-17T07:28:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rgwlc_auto_session_clear": "true", 2024-09-17T07:28:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-17T07:28:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_block_size": "4096", 2024-09-17T07:28:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-17T07:28:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-17T07:28:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-17T07:28:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-17T07:28:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-17T07:28:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cache_size": "536870912", 2024-09-17T07:28:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-17T07:28:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-17T07:28:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-17T07:28:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-17T07:28:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-17T07:28:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-17T07:28:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-17T07:28:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-17T07:28:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_index_type": "binary_search", 2024-09-17T07:28:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-17T07:28:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-17T07:28:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_partition_filters": "false", 2024-09-17T07:28:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_perf": "false", 2024-09-17T07:28:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-17T07:28:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-17T07:28:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-17T07:28:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "run_dir": "/var/run/ceph", 2024-09-17T07:28:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_block_create": "true", 2024-09-17T07:28:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_cache_lru_size": "67108864", 2024-09-17T07:28:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-17T07:28:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-17T07:28:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_default_max_object_size": "16777216", 2024-09-17T07:28:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-17T07:28:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_device_size": "53687091200", 2024-09-17T07:28:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-17T07:28:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_full_integrity_check": "false", 2024-09-17T07:28:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_journal_batch_capacity": "16", 2024-09-17T07:28:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-17T07:28:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-17T07:28:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-17T07:28:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_main_device_type": "SSD", 2024-09-17T07:28:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-17T07:28:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-17T07:28:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-17T07:28:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-17T07:28:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-17T07:28:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-17T07:28:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "seastore_segment_size": "67108864", 2024-09-17T07:28:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "set_keepcaps": "false", 2024-09-17T07:28:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "setgroup": "", 2024-09-17T07:28:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "setuser": "", 2024-09-17T07:28:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "setuser_match_path": "", 2024-09-17T07:28:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-17T07:28:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "thp": "false", 2024-09-17T07:28:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "threadpool_default_timeout": "60", 2024-09-17T07:28:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-17T07:28:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "throttler_perf_counter": "true", 2024-09-17T07:28:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "tmp_dir": "/tmp", 2024-09-17T07:28:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-17T07:28:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "uadk_compressor_enabled": "false", 2024-09-17T07:28:41.538 INFO:teuthology.orchestra.run.smithi045.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-17T07:28:41.538 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:41.541 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-17T07:28:41.657 INFO:teuthology.orchestra.run.smithi045.stdout:mon.a: {} 2024-09-17T07:28:41.658 INFO:teuthology.orchestra.run.smithi045.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T07:28:41.667 INFO:teuthology.orchestra.run.smithi045.stdout:mon.b: {} 2024-09-17T07:28:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T07:28:41.676 INFO:teuthology.orchestra.run.smithi045.stdout:mon.c: {} 2024-09-17T07:28:41.676 INFO:teuthology.orchestra.run.smithi045.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T07:28:41.687 DEBUG:teuthology.orchestra.run.smithi045:> type ceph-objectstore-tool 2024-09-17T07:28:41.702 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T07:28:41.703 DEBUG:teuthology.orchestra.run.smithi192:> type ceph-objectstore-tool 2024-09-17T07:28:41.722 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T07:28:41.722 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-17T07:28:41.758 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-17T07:28:41.758 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-17T07:28:41.759 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T07:28:41.759 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:28:41.759 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:28:41.759 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-17T07:28:41.760 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-17T07:28:41.760 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:28:41.760 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-17T07:28:41.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:28:41.761 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-17T07:28:41.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T07:28:41.761 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-17T07:28:41.764 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:41.764+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:41.806 INFO:teuthology.orchestra.run.smithi045.stdout:mapped writes DISABLED 2024-09-17T07:28:41.806 INFO:teuthology.orchestra.run.smithi045.stdout:mapped reads DISABLED 2024-09-17T07:28:41.806 INFO:teuthology.orchestra.run.smithi045.stdout:Seed set to 7536 2024-09-17T07:28:41.861 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:41.860+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:41.896 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:41.896 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:41.896 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:28:41.896 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:41.903 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:28:41.903 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:28:41.905 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:28:41.921 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:28:41.922 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:28:41.938 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:28:41.938 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:28:41.955 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:28:41.955 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:28:41.963 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:41.962+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:41.973 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:28:41.973 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:28:41.990 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:28:41.990 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:28:42.008 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:28:42.008 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:28:42.026 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:28:42.026 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:28:42.032 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:42.032 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:42.042 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:28:42.084 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:42.082+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:42.153 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:28:42.153 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:28:42.164 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:42.164+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:42.166 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-17T07:28:42.062210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234804,"kb_used_data":1896,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739916,"statfs":{"total":773094113280,"available":772853673984,"internally_reserved":0,"allocated":1941504,"data_stored":1795734,"data_compressed":65196,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.311011"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.665025+0000","last_change":"2024-09-17T07:28:33.665025+0000","last_active":"2024-09-17T07:28:33.665025+0000","last_peered":"2024-09-17T07:28:33.665025+0000","last_clean":"2024-09-17T07:28:33.665025+0000","last_became_active":"2024-09-17T07:28:31.364445+0000","last_became_peered":"2024-09-17T07:28:31.364445+0000","last_unstale":"2024-09-17T07:28:33.665025+0000","last_undegraded":"2024-09-17T07:28:33.665025+0000","last_fullsized":"2024-09-17T07:28:33.665025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:23:49.197824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.361412+0000","last_change":"2024-09-17T07:28:33.361412+0000","last_active":"2024-09-17T07:28:33.361412+0000","last_peered":"2024-09-17T07:28:33.361412+0000","last_clean":"2024-09-17T07:28:33.361412+0000","last_became_active":"2024-09-17T07:28:31.362639+0000","last_became_peered":"2024-09-17T07:28:31.362639+0000","last_unstale":"2024-09-17T07:28:33.361412+0000","last_undegraded":"2024-09-17T07:28:33.361412+0000","last_fullsized":"2024-09-17T07:28:33.361412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:42.297390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049448899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.951359+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:28:33.951359+0000","last_peered":"2024-09-17T07:28:33.951359+0000","last_clean":"2024-09-17T07:28:33.951359+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:28:33.951359+0000","last_undegraded":"2024-09-17T07:28:33.951359+0000","last_fullsized":"2024-09-17T07:28:33.951359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.361447+0000","last_change":"2024-09-17T07:28:33.361447+0000","last_active":"2024-09-17T07:28:33.361447+0000","last_peered":"2024-09-17T07:28:33.361447+0000","last_clean":"2024-09-17T07:28:33.361447+0000","last_became_active":"2024-09-17T07:28:31.567277+0000","last_became_peered":"2024-09-17T07:28:31.567277+0000","last_unstale":"2024-09-17T07:28:33.361447+0000","last_undegraded":"2024-09-17T07:28:33.361447+0000","last_fullsized":"2024-09-17T07:28:33.361447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:09.089091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056035799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.364473+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:28:33.364473+0000","last_peered":"2024-09-17T07:28:33.364473+0000","last_clean":"2024-09-17T07:28:33.364473+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:28:33.364473+0000","last_undegraded":"2024-09-17T07:28:33.364473+0000","last_fullsized":"2024-09-17T07:28:33.364473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.952450+0000","last_change":"2024-09-17T07:28:33.952596+0000","last_active":"2024-09-17T07:28:33.952450+0000","last_peered":"2024-09-17T07:28:33.952450+0000","last_clean":"2024-09-17T07:28:33.952450+0000","last_became_active":"2024-09-17T07:28:31.354367+0000","last_became_peered":"2024-09-17T07:28:31.354367+0000","last_unstale":"2024-09-17T07:28:33.952450+0000","last_undegraded":"2024-09-17T07:28:33.952450+0000","last_fullsized":"2024-09-17T07:28:33.952450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:37.907368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.951319+0000","last_change":"2024-09-17T07:28:33.951319+0000","last_active":"2024-09-17T07:28:33.951319+0000","last_peered":"2024-09-17T07:28:33.951319+0000","last_clean":"2024-09-17T07:28:33.951319+0000","last_became_active":"2024-09-17T07:28:31.364065+0000","last_became_peered":"2024-09-17T07:28:31.364065+0000","last_unstale":"2024-09-17T07:28:33.951319+0000","last_undegraded":"2024-09-17T07:28:33.951319+0000","last_fullsized":"2024-09-17T07:28:33.951319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:40.816064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.361820+0000","last_change":"2024-09-17T07:28:33.361820+0000","last_active":"2024-09-17T07:28:33.361820+0000","last_peered":"2024-09-17T07:28:33.361820+0000","last_clean":"2024-09-17T07:28:33.361820+0000","last_became_active":"2024-09-17T07:28:31.359703+0000","last_became_peered":"2024-09-17T07:28:31.359703+0000","last_unstale":"2024-09-17T07:28:33.361820+0000","last_undegraded":"2024-09-17T07:28:33.361820+0000","last_fullsized":"2024-09-17T07:28:33.361820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:59.468099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000487639,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T07:28:33.950496+0000","last_change":"2024-09-17T07:28:26.547435+0000","last_active":"2024-09-17T07:28:33.950496+0000","last_peered":"2024-09-17T07:28:33.950496+0000","last_clean":"2024-09-17T07:28:33.950496+0000","last_became_active":"2024-09-17T07:28:26.546300+0000","last_became_peered":"2024-09-17T07:28:26.546300+0000","last_unstale":"2024-09-17T07:28:33.950496+0000","last_undegraded":"2024-09-17T07:28:33.950496+0000","last_fullsized":"2024-09-17T07:28:33.950496+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:02:23.225041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T07:28:42.059932+0000","last_change":"2024-09-17T07:28:42.059932+0000","last_active":"2024-09-17T07:28:42.059932+0000","last_peered":"2024-09-17T07:28:42.059932+0000","last_clean":"2024-09-17T07:28:42.059932+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:28:42.059932+0000","last_undegraded":"2024-09-17T07:28:42.059932+0000","last_fullsized":"2024-09-17T07:28:42.059932+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65196,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":667238,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":76870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":229376,"data_stored":76889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31480,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340360,"statfs":{"total":96636764160,"available":96604528640,"internally_reserved":0,"allocated":290816,"data_stored":667238,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:28:42.191 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:42.191 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:42.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:28:42.285 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:42.284+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:42.348 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:42.348 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:42.356 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:28:42.397 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:42.396+0000 7f9a047b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33358) UID: 0 2024-09-17T07:28:42.467 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:42.465+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:42.504 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:42.504 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:42.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:28:42.596 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:42.595+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:42.660 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:42.660 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:42.669 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:28:42.669 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:42.668+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:42.770 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:42.768+0000 7f9a047b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33358) UID: 0 2024-09-17T07:28:42.816 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:42.816 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:42.825 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:28:42.871 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:42.870+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:42.971 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:42.972 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:42.972 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:42.970+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:42.980 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:28:43.068 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-17T07:28:43.072 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:43.072+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:43.137 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:43.137 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:43.145 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:28:43.173 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:43.173+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:43.254 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:43.254 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:43.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:28:43.274 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:43.274+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:43.370 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:43.370 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:43.375 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:43.374+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:43.379 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:28:43.475 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:43.474+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:43.485 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:43.485 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:43.494 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:28:43.576 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:43.575+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:43.610 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:43.610 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:43.619 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:28:43.677 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:43.677+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:43.724 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:43.724 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:43.732 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:28:43.816 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:43.816+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:43.876 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:43.877 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:43.879 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:43.877+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:43.885 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:28:43.980 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:43.980+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:44.029 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:44.029 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:44.037 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:28:44.039 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:28:44.121 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:44.121+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:44.177 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:28:44.178 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:28:44.181 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:44.181 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:44.182 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:44.180+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:44.190 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:28:44.195 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:28:44.195 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:28:44.211 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:28:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:28:44.227 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:28:44.228 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:28:44.244 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:28:44.244 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:28:44.261 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:28:44.261 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:28:44.278 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:28:44.278 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:28:44.282 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:44.282+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:44.296 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:28:44.296 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:28:44.336 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:44.336 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:28:44.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:28:44.337 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:28:44.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.337 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4226.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-17T07:28:44.337 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:31.356276+0000", 2024-09-17T07:28:44.337 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.979698354, 2024-09-17T07:28:44.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053924159999999997, 2024-09-17T07:28:44.337 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:44.337 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:44.337 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:44.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:44.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4226", 2024-09-17T07:28:44.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/1373512931", 2024-09-17T07:28:44.338 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:28:44.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.338 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:44.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:44.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356276+0000", 2024-09-17T07:28:44.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:44.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356276+0000", 2024-09-17T07:28:44.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:44.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356280+0000", 2024-09-17T07:28:44.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.019e-06 2024-09-17T07:28:44.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:44.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356287+0000", 2024-09-17T07:28:44.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.267e-06 2024-09-17T07:28:44.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:44.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356297+0000", 2024-09-17T07:28:44.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5699999999999999e-06 2024-09-17T07:28:44.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:44.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356305+0000", 2024-09-17T07:28:44.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5040000000000002e-06 2024-09-17T07:28:44.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:44.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356392+0000", 2024-09-17T07:28:44.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7012e-05 2024-09-17T07:28:44.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-17T07:28:44.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356402+0000", 2024-09-17T07:28:44.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9650000000000003e-06 2024-09-17T07:28:44.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:44.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.358226+0000", 2024-09-17T07:28:44.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018234620000000001 2024-09-17T07:28:44.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:28:44.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.358232+0000", 2024-09-17T07:28:44.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2820000000000002e-06 2024-09-17T07:28:44.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:44.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.360426+0000", 2024-09-17T07:28:44.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021943890000000001 2024-09-17T07:28:44.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:44.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.360556+0000", 2024-09-17T07:28:44.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012972 2024-09-17T07:28:44.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-17T07:28:44.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.360649+0000", 2024-09-17T07:28:44.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2942000000000006e-05 2024-09-17T07:28:44.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:44.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361494+0000", 2024-09-17T07:28:44.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00084479200000000003 2024-09-17T07:28:44.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:44.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361559+0000", 2024-09-17T07:28:44.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5347999999999998e-05 2024-09-17T07:28:44.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:44.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361616+0000", 2024-09-17T07:28:44.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6836000000000003e-05 2024-09-17T07:28:44.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:44.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361668+0000", 2024-09-17T07:28:44.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2308000000000001e-05 2024-09-17T07:28:44.344 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.344 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:44.344 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.344 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4226.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-17T07:28:44.344 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:31.361461+0000", 2024-09-17T07:28:44.344 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.974513104, 2024-09-17T07:28:44.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000213981, 2024-09-17T07:28:44.345 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:44.345 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:44.345 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:28:44.345 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:44.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:44.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361461+0000", 2024-09-17T07:28:44.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:44.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361461+0000", 2024-09-17T07:28:44.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:44.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361462+0000", 2024-09-17T07:28:44.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.034e-06 2024-09-17T07:28:44.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:44.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361468+0000", 2024-09-17T07:28:44.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-06 2024-09-17T07:28:44.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:44.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361480+0000", 2024-09-17T07:28:44.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1671999999999999e-05 2024-09-17T07:28:44.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:44.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361492+0000", 2024-09-17T07:28:44.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1773000000000001e-05 2024-09-17T07:28:44.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:44.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361539+0000", 2024-09-17T07:28:44.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7120999999999998e-05 2024-09-17T07:28:44.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:44.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361549+0000", 2024-09-17T07:28:44.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9909999999999997e-06 2024-09-17T07:28:44.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:44.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361675+0000", 2024-09-17T07:28:44.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012595999999999999 2024-09-17T07:28:44.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.348 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:44.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.348 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4226.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-17T07:28:44.348 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:31.364999+0000", 2024-09-17T07:28:44.349 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.970975135, 2024-09-17T07:28:44.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023785899999999999, 2024-09-17T07:28:44.349 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:44.349 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:44.349 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:28:44.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:44.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4226", 2024-09-17T07:28:44.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/1373512931", 2024-09-17T07:28:44.349 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-17T07:28:44.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.349 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:44.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:44.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.364999+0000", 2024-09-17T07:28:44.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:44.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.364999+0000", 2024-09-17T07:28:44.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:44.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.365001+0000", 2024-09-17T07:28:44.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0430000000000002e-06 2024-09-17T07:28:44.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:44.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.365009+0000", 2024-09-17T07:28:44.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5390000000000002e-06 2024-09-17T07:28:44.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:44.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.365025+0000", 2024-09-17T07:28:44.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6694999999999999e-05 2024-09-17T07:28:44.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:44.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.365045+0000", 2024-09-17T07:28:44.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0021999999999999e-05 2024-09-17T07:28:44.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:44.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.365113+0000", 2024-09-17T07:28:44.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8176999999999995e-05 2024-09-17T07:28:44.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:44.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.365237+0000", 2024-09-17T07:28:44.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012338299999999999 2024-09-17T07:28:44.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.352 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:44.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.352 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4226.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-17T07:28:44.352 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:31.365401+0000", 2024-09-17T07:28:44.353 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.970572941, 2024-09-17T07:28:44.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000212976, 2024-09-17T07:28:44.353 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:44.353 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:44.353 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:28:44.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:44.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4226", 2024-09-17T07:28:44.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/1373512931", 2024-09-17T07:28:44.353 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-17T07:28:44.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.353 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:44.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:44.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.365401+0000", 2024-09-17T07:28:44.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:44.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.365401+0000", 2024-09-17T07:28:44.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:44.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.365402+0000", 2024-09-17T07:28:44.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0869999999999999e-06 2024-09-17T07:28:44.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:44.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.365418+0000", 2024-09-17T07:28:44.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.573e-05 2024-09-17T07:28:44.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:44.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.365432+0000", 2024-09-17T07:28:44.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4185000000000001e-05 2024-09-17T07:28:44.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:44.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.365442+0000", 2024-09-17T07:28:44.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0020000000000001e-05 2024-09-17T07:28:44.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:44.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.365500+0000", 2024-09-17T07:28:44.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7453e-05 2024-09-17T07:28:44.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:44.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.365614+0000", 2024-09-17T07:28:44.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011450099999999999 2024-09-17T07:28:44.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.356 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:44.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.356 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4226.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-17T07:28:44.356 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:32.358109+0000", 2024-09-17T07:28:44.357 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 11.977865532999999, 2024-09-17T07:28:44.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033031800000000002, 2024-09-17T07:28:44.357 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:44.357 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:44.357 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:44.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:44.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4226", 2024-09-17T07:28:44.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/1373512931", 2024-09-17T07:28:44.357 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-17T07:28:44.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.357 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:44.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:44.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.358109+0000", 2024-09-17T07:28:44.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:44.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.358109+0000", 2024-09-17T07:28:44.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:44.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.358110+0000", 2024-09-17T07:28:44.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6700000000000001e-06 2024-09-17T07:28:44.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:44.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.358123+0000", 2024-09-17T07:28:44.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2797e-05 2024-09-17T07:28:44.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:44.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.358142+0000", 2024-09-17T07:28:44.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8904000000000001e-05 2024-09-17T07:28:44.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:44.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.358170+0000", 2024-09-17T07:28:44.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.849e-05 2024-09-17T07:28:44.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.358236+0000", 2024-09-17T07:28:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5681999999999999e-05 2024-09-17T07:28:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-17T07:28:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.358242+0000", 2024-09-17T07:28:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8279999999999999e-06 2024-09-17T07:28:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.359869+0000", 2024-09-17T07:28:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016265780000000001 2024-09-17T07:28:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:44.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.359987+0000", 2024-09-17T07:28:44.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000118077 2024-09-17T07:28:44.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-17T07:28:44.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.360107+0000", 2024-09-17T07:28:44.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011995200000000001 2024-09-17T07:28:44.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:44.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.360776+0000", 2024-09-17T07:28:44.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00066936600000000001 2024-09-17T07:28:44.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:44.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.361344+0000", 2024-09-17T07:28:44.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056853800000000005 2024-09-17T07:28:44.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:44.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.361380+0000", 2024-09-17T07:28:44.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5200000000000002e-05 2024-09-17T07:28:44.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:44.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.361412+0000", 2024-09-17T07:28:44.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2098000000000003e-05 2024-09-17T07:28:44.362 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.363 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:44.363 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.363 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4226.0:5 2.2 e21/19 ondisk, result = 0)", 2024-09-17T07:28:44.363 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:32.361257+0000", 2024-09-17T07:28:44.363 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 11.974717051000001, 2024-09-17T07:28:44.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000163673, 2024-09-17T07:28:44.363 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:44.363 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:44.363 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:28:44.364 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:44.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:44.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.361257+0000", 2024-09-17T07:28:44.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:44.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.361257+0000", 2024-09-17T07:28:44.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:44.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.361258+0000", 2024-09-17T07:28:44.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.13e-06 2024-09-17T07:28:44.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:44.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.361263+0000", 2024-09-17T07:28:44.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9660000000000004e-06 2024-09-17T07:28:44.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:44.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.361277+0000", 2024-09-17T07:28:44.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3448000000000001e-05 2024-09-17T07:28:44.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:44.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.361284+0000", 2024-09-17T07:28:44.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9469999999999997e-06 2024-09-17T07:28:44.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:44.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.361333+0000", 2024-09-17T07:28:44.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9344000000000001e-05 2024-09-17T07:28:44.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:44.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.361341+0000", 2024-09-17T07:28:44.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8679999999999999e-06 2024-09-17T07:28:44.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:44.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:32.361421+0000", 2024-09-17T07:28:44.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9969999999999995e-05 2024-09-17T07:28:44.367 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.367 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:44.367 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.367 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4226.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-17T07:28:44.367 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:33.362989+0000", 2024-09-17T07:28:44.367 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.972984740999999, 2024-09-17T07:28:44.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001514545, 2024-09-17T07:28:44.367 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:44.368 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:44.368 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:44.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:44.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4226", 2024-09-17T07:28:44.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/1373512931", 2024-09-17T07:28:44.368 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-17T07:28:44.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.368 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:44.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:44.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.362989+0000", 2024-09-17T07:28:44.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.362989+0000", 2024-09-17T07:28:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.362991+0000", 2024-09-17T07:28:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.584e-06 2024-09-17T07:28:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.362999+0000", 2024-09-17T07:28:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8960000000000003e-06 2024-09-17T07:28:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:44.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.363012+0000", 2024-09-17T07:28:44.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3234000000000001e-05 2024-09-17T07:28:44.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:44.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.363021+0000", 2024-09-17T07:28:44.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8340000000000005e-06 2024-09-17T07:28:44.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:44.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.363079+0000", 2024-09-17T07:28:44.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8111e-05 2024-09-17T07:28:44.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:44.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.363151+0000", 2024-09-17T07:28:44.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1602999999999999e-05 2024-09-17T07:28:44.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-17T07:28:44.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.363271+0000", 2024-09-17T07:28:44.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120324 2024-09-17T07:28:44.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:44.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.364020+0000", 2024-09-17T07:28:44.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000748615 2024-09-17T07:28:44.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:44.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.364433+0000", 2024-09-17T07:28:44.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000412934 2024-09-17T07:28:44.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:44.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.364471+0000", 2024-09-17T07:28:44.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8656999999999999e-05 2024-09-17T07:28:44.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:44.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.364504+0000", 2024-09-17T07:28:44.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2753000000000003e-05 2024-09-17T07:28:44.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.373 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:44.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.373 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4226.0:7 2.2 e22/19 ondisk, result = 0)", 2024-09-17T07:28:44.373 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:33.364344+0000", 2024-09-17T07:28:44.374 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.971630393, 2024-09-17T07:28:44.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016833399999999999, 2024-09-17T07:28:44.374 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:44.374 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:44.374 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:28:44.374 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:44.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:44.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.364344+0000", 2024-09-17T07:28:44.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:44.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.364344+0000", 2024-09-17T07:28:44.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:44.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:44.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.364345+0000", 2024-09-17T07:28:44.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4020000000000001e-06 2024-09-17T07:28:44.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:44.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.364355+0000", 2024-09-17T07:28:44.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0234999999999999e-05 2024-09-17T07:28:44.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:44.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.364369+0000", 2024-09-17T07:28:44.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3859000000000001e-05 2024-09-17T07:28:44.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:44.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.364376+0000", 2024-09-17T07:28:44.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4949999999999996e-06 2024-09-17T07:28:44.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:44.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.364422+0000", 2024-09-17T07:28:44.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5819999999999998e-05 2024-09-17T07:28:44.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:44.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.364429+0000", 2024-09-17T07:28:44.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6550000000000004e-06 2024-09-17T07:28:44.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:44.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:44.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:44.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:33.364512+0000", 2024-09-17T07:28:44.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2867999999999993e-05 2024-09-17T07:28:44.377 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.377 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:44.377 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.377 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:44.377 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:44.377 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:44.378 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:28:44.420 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:44.419+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:44.484 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:44.482+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:44.522 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:44.522 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:28:44.522 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:28:44.522 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:44.530 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:28:44.584 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:44.583+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:44.633 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:28:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:28:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:28:44.633 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:44.641 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:28:44.684 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:44.684+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:44.784 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:28:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:28:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-17T07:28:44.785 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:44.785 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:44.784+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:44.793 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:28:44.886 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:44.885+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:44.936 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:44.936 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:28:44.936 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:28:44.936 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:44.944 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:28:44.986 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:44.985+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:45.087 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:45.087+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:45.090 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T07:28:45.091 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:45.091 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:28:45.091 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:28:45.091 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:28:45.092 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:45.101 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:28:45.188 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:45.188+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:45.248 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:45.248 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:28:45.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:28:45.248 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:28:45.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.248 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.248 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:28:45.248 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.911234902, 2024-09-17T07:28:45.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:28:45.248 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.249 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.249 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.249 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.249 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.249 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.249 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:28:45.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.249 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:28:45.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:28:45.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:28:45.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:28:45.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:28:45.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:28:45.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:28:45.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:28:45.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:28:45.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:28:45.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:28:45.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:28:45.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:28:45.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:28:45.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:28:45.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:28:45.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:28:45.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:28:45.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:28:45.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:28:45.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:28:45.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:28:45.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:28:45.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:28:45.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:28:45.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:28:45.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:28:45.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:28:45.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:28:45.255 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.255 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.255 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.255 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.255 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.657961+0000", 2024-09-17T07:28:45.255 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.58794335, 2024-09-17T07:28:45.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053647690000000001, 2024-09-17T07:28:45.256 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.256 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.256 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.256 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-17T07:28:45.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.256 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.657961+0000", 2024-09-17T07:28:45.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.657961+0000", 2024-09-17T07:28:45.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.657962+0000", 2024-09-17T07:28:45.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1600000000000001e-07 2024-09-17T07:28:45.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.657966+0000", 2024-09-17T07:28:45.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7100000000000001e-06 2024-09-17T07:28:45.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.657972+0000", 2024-09-17T07:28:45.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1290000000000001e-06 2024-09-17T07:28:45.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.657978+0000", 2024-09-17T07:28:45.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:28:45.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.661150+0000", 2024-09-17T07:28:45.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031715960000000001 2024-09-17T07:28:45.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.661195+0000", 2024-09-17T07:28:45.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4916999999999997e-05 2024-09-17T07:28:45.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.661266+0000", 2024-09-17T07:28:45.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1116000000000001e-05 2024-09-17T07:28:45.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.663165+0000", 2024-09-17T07:28:45.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018989899999999999 2024-09-17T07:28:45.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.663276+0000", 2024-09-17T07:28:45.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000110549 2024-09-17T07:28:45.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.663299+0000", 2024-09-17T07:28:45.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2901e-05 2024-09-17T07:28:45.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.663326+0000", 2024-09-17T07:28:45.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7664000000000001e-05 2024-09-17T07:28:45.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.261 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.261 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.261 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.657998+0000", 2024-09-17T07:28:45.261 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.587906880999999, 2024-09-17T07:28:45.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0065022709999999996, 2024-09-17T07:28:45.262 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.262 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.262 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.262 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-17T07:28:45.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.262 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.657998+0000", 2024-09-17T07:28:45.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.657998+0000", 2024-09-17T07:28:45.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.657999+0000", 2024-09-17T07:28:45.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8899999999999999e-07 2024-09-17T07:28:45.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658084+0000", 2024-09-17T07:28:45.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5570999999999998e-05 2024-09-17T07:28:45.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658093+0000", 2024-09-17T07:28:45.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2450000000000004e-06 2024-09-17T07:28:45.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658101+0000", 2024-09-17T07:28:45.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3440000000000003e-06 2024-09-17T07:28:45.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.661496+0000", 2024-09-17T07:28:45.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033949370000000002 2024-09-17T07:28:45.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.661546+0000", 2024-09-17T07:28:45.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9894e-05 2024-09-17T07:28:45.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.661633+0000", 2024-09-17T07:28:45.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7873000000000004e-05 2024-09-17T07:28:45.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.663206+0000", 2024-09-17T07:28:45.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001572234 2024-09-17T07:28:45.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664441+0000", 2024-09-17T07:28:45.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012348330000000001 2024-09-17T07:28:45.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664472+0000", 2024-09-17T07:28:45.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1538999999999997e-05 2024-09-17T07:28:45.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664500+0000", 2024-09-17T07:28:45.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8112000000000001e-05 2024-09-17T07:28:45.267 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.267 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.267 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.267 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.267 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.658116+0000", 2024-09-17T07:28:45.267 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.587789255000001, 2024-09-17T07:28:45.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064317030000000004, 2024-09-17T07:28:45.268 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.268 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.268 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.268 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.268 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.268 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.268 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-17T07:28:45.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.268 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658116+0000", 2024-09-17T07:28:45.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658116+0000", 2024-09-17T07:28:45.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658116+0000", 2024-09-17T07:28:45.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1699999999999997e-07 2024-09-17T07:28:45.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658121+0000", 2024-09-17T07:28:45.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599999999999998e-06 2024-09-17T07:28:45.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658127+0000", 2024-09-17T07:28:45.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:28:45.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658133+0000", 2024-09-17T07:28:45.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6010000000000002e-06 2024-09-17T07:28:45.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.662242+0000", 2024-09-17T07:28:45.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041090889999999998 2024-09-17T07:28:45.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.662290+0000", 2024-09-17T07:28:45.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7657999999999997e-05 2024-09-17T07:28:45.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.662371+0000", 2024-09-17T07:28:45.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1216000000000003e-05 2024-09-17T07:28:45.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.663374+0000", 2024-09-17T07:28:45.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001003213 2024-09-17T07:28:45.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664507+0000", 2024-09-17T07:28:45.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011327310000000001 2024-09-17T07:28:45.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664530+0000", 2024-09-17T07:28:45.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2582000000000002e-05 2024-09-17T07:28:45.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664547+0000", 2024-09-17T07:28:45.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7686e-05 2024-09-17T07:28:45.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.273 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.274 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.274 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.274 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.658145+0000", 2024-09-17T07:28:45.274 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.587759384999998, 2024-09-17T07:28:45.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0068169040000000004, 2024-09-17T07:28:45.274 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.274 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.274 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.274 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-17T07:28:45.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.275 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658145+0000", 2024-09-17T07:28:45.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658145+0000", 2024-09-17T07:28:45.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658146+0000", 2024-09-17T07:28:45.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8999999999999996e-07 2024-09-17T07:28:45.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658151+0000", 2024-09-17T07:28:45.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3990000000000003e-06 2024-09-17T07:28:45.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658159+0000", 2024-09-17T07:28:45.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7090000000000002e-06 2024-09-17T07:28:45.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658164+0000", 2024-09-17T07:28:45.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1699999999999996e-06 2024-09-17T07:28:45.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.662782+0000", 2024-09-17T07:28:45.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.004617655 2024-09-17T07:28:45.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.662829+0000", 2024-09-17T07:28:45.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6901999999999999e-05 2024-09-17T07:28:45.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.662909+0000", 2024-09-17T07:28:45.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0339000000000004e-05 2024-09-17T07:28:45.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664428+0000", 2024-09-17T07:28:45.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015191250000000001 2024-09-17T07:28:45.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664917+0000", 2024-09-17T07:28:45.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00048883900000000003 2024-09-17T07:28:45.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664941+0000", 2024-09-17T07:28:45.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3570999999999998e-05 2024-09-17T07:28:45.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664962+0000", 2024-09-17T07:28:45.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1605000000000001e-05 2024-09-17T07:28:45.280 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.280 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.280 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.280 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.280 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.658186+0000", 2024-09-17T07:28:45.280 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.587718913, 2024-09-17T07:28:45.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070915600000000002, 2024-09-17T07:28:45.280 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.280 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.280 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.280 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.280 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.281 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.281 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-17T07:28:45.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.281 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658186+0000", 2024-09-17T07:28:45.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658186+0000", 2024-09-17T07:28:45.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658187+0000", 2024-09-17T07:28:45.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7599999999999997e-07 2024-09-17T07:28:45.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658248+0000", 2024-09-17T07:28:45.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1239000000000001e-05 2024-09-17T07:28:45.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658256+0000", 2024-09-17T07:28:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9820000000000008e-06 2024-09-17T07:28:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658262+0000", 2024-09-17T07:28:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9800000000000003e-06 2024-09-17T07:28:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.663408+0000", 2024-09-17T07:28:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051461450000000004 2024-09-17T07:28:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.663468+0000", 2024-09-17T07:28:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9861000000000002e-05 2024-09-17T07:28:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.663573+0000", 2024-09-17T07:28:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105394 2024-09-17T07:28:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664910+0000", 2024-09-17T07:28:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013365250000000001 2024-09-17T07:28:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.665174+0000", 2024-09-17T07:28:45.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026413199999999998 2024-09-17T07:28:45.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.665219+0000", 2024-09-17T07:28:45.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5289e-05 2024-09-17T07:28:45.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.665277+0000", 2024-09-17T07:28:45.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8337000000000001e-05 2024-09-17T07:28:45.285 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.658277+0000", 2024-09-17T07:28:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.587628028000001, 2024-09-17T07:28:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070770349999999997, 2024-09-17T07:28:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-17T07:28:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658277+0000", 2024-09-17T07:28:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658277+0000", 2024-09-17T07:28:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658278+0000", 2024-09-17T07:28:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5799999999999998e-07 2024-09-17T07:28:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658282+0000", 2024-09-17T07:28:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5979999999999999e-06 2024-09-17T07:28:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658291+0000", 2024-09-17T07:28:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3530000000000001e-06 2024-09-17T07:28:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658299+0000", 2024-09-17T07:28:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9000000000000006e-06 2024-09-17T07:28:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.663912+0000", 2024-09-17T07:28:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0056128419999999998 2024-09-17T07:28:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.663963+0000", 2024-09-17T07:28:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0726000000000002e-05 2024-09-17T07:28:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664060+0000", 2024-09-17T07:28:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7319999999999997e-05 2024-09-17T07:28:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664996+0000", 2024-09-17T07:28:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00093608999999999999 2024-09-17T07:28:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.665283+0000", 2024-09-17T07:28:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000286295 2024-09-17T07:28:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.665311+0000", 2024-09-17T07:28:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7977999999999999e-05 2024-09-17T07:28:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.665354+0000", 2024-09-17T07:28:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3174999999999999e-05 2024-09-17T07:28:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.658311+0000", 2024-09-17T07:28:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.587593567999999, 2024-09-17T07:28:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0078339670000000007, 2024-09-17T07:28:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-17T07:28:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658311+0000", 2024-09-17T07:28:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658311+0000", 2024-09-17T07:28:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658312+0000", 2024-09-17T07:28:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6100000000000001e-07 2024-09-17T07:28:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658316+0000", 2024-09-17T07:28:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4349999999999999e-06 2024-09-17T07:28:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658336+0000", 2024-09-17T07:28:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0262e-05 2024-09-17T07:28:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.658345+0000", 2024-09-17T07:28:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-17T07:28:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664578+0000", 2024-09-17T07:28:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062328920000000003 2024-09-17T07:28:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664635+0000", 2024-09-17T07:28:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6777999999999997e-05 2024-09-17T07:28:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.664721+0000", 2024-09-17T07:28:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5969000000000003e-05 2024-09-17T07:28:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.665385+0000", 2024-09-17T07:28:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00066401100000000005 2024-09-17T07:28:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.666067+0000", 2024-09-17T07:28:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00068232799999999995 2024-09-17T07:28:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.666103+0000", 2024-09-17T07:28:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6247e-05 2024-09-17T07:28:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.666145+0000", 2024-09-17T07:28:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1876999999999999e-05 2024-09-17T07:28:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.951527+0000", 2024-09-17T07:28:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.294378025, 2024-09-17T07:28:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066466190000000003, 2024-09-17T07:28:45.299 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.299 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.299 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.299 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.299 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.299 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.299 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 70 2024-09-17T07:28:45.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.299 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951527+0000", 2024-09-17T07:28:45.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951527+0000", 2024-09-17T07:28:45.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951527+0000", 2024-09-17T07:28:45.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.01e-07 2024-09-17T07:28:45.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951530+0000", 2024-09-17T07:28:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6910000000000002e-06 2024-09-17T07:28:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951535+0000", 2024-09-17T07:28:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3890000000000004e-06 2024-09-17T07:28:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951538+0000", 2024-09-17T07:28:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.128e-06 2024-09-17T07:28:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.954520+0000", 2024-09-17T07:28:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0029817680000000001 2024-09-17T07:28:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.954567+0000", 2024-09-17T07:28:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7039999999999997e-05 2024-09-17T07:28:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.954676+0000", 2024-09-17T07:28:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000108383 2024-09-17T07:28:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.956587+0000", 2024-09-17T07:28:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019114410000000001 2024-09-17T07:28:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.958106+0000", 2024-09-17T07:28:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00151913 2024-09-17T07:28:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.958134+0000", 2024-09-17T07:28:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8209e-05 2024-09-17T07:28:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.958173+0000", 2024-09-17T07:28:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9038999999999997e-05 2024-09-17T07:28:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.951543+0000", 2024-09-17T07:28:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.294361464000001, 2024-09-17T07:28:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0092465189999999999, 2024-09-17T07:28:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 71 2024-09-17T07:28:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951543+0000", 2024-09-17T07:28:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951543+0000", 2024-09-17T07:28:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951544+0000", 2024-09-17T07:28:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7500000000000001e-07 2024-09-17T07:28:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951546+0000", 2024-09-17T07:28:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1739999999999999e-06 2024-09-17T07:28:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951551+0000", 2024-09-17T07:28:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:28:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951554+0000", 2024-09-17T07:28:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9569999999999998e-06 2024-09-17T07:28:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.954989+0000", 2024-09-17T07:28:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034353529999999999 2024-09-17T07:28:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.955045+0000", 2024-09-17T07:28:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5506e-05 2024-09-17T07:28:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.955123+0000", 2024-09-17T07:28:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7955999999999997e-05 2024-09-17T07:28:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.956628+0000", 2024-09-17T07:28:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001505143 2024-09-17T07:28:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.960701+0000", 2024-09-17T07:28:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.004073302 2024-09-17T07:28:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.960749+0000", 2024-09-17T07:28:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7589000000000001e-05 2024-09-17T07:28:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.960790+0000", 2024-09-17T07:28:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1205000000000003e-05 2024-09-17T07:28:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.951560+0000", 2024-09-17T07:28:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.294344765000002, 2024-09-17T07:28:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0093019690000000002, 2024-09-17T07:28:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-17T07:28:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951560+0000", 2024-09-17T07:28:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951560+0000", 2024-09-17T07:28:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951560+0000", 2024-09-17T07:28:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0499999999999999e-07 2024-09-17T07:28:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951602+0000", 2024-09-17T07:28:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1869999999999997e-05 2024-09-17T07:28:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951612+0000", 2024-09-17T07:28:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5459999999999997e-06 2024-09-17T07:28:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951615+0000", 2024-09-17T07:28:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7239999999999998e-06 2024-09-17T07:28:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.955920+0000", 2024-09-17T07:28:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0043045430000000001 2024-09-17T07:28:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.955966+0000", 2024-09-17T07:28:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6124000000000001e-05 2024-09-17T07:28:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.956040+0000", 2024-09-17T07:28:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4307000000000005e-05 2024-09-17T07:28:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.958051+0000", 2024-09-17T07:28:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020108560000000001 2024-09-17T07:28:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.960797+0000", 2024-09-17T07:28:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0027455259999999999 2024-09-17T07:28:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.960824+0000", 2024-09-17T07:28:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7418000000000001e-05 2024-09-17T07:28:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.960862+0000", 2024-09-17T07:28:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7750000000000003e-05 2024-09-17T07:28:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.951643+0000", 2024-09-17T07:28:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.294261347999999, 2024-09-17T07:28:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0092845589999999995, 2024-09-17T07:28:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-17T07:28:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951643+0000", 2024-09-17T07:28:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951643+0000", 2024-09-17T07:28:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951644+0000", 2024-09-17T07:28:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-17T07:28:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951647+0000", 2024-09-17T07:28:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:28:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951651+0000", 2024-09-17T07:28:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1230000000000002e-06 2024-09-17T07:28:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951655+0000", 2024-09-17T07:28:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.065e-06 2024-09-17T07:28:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.956650+0000", 2024-09-17T07:28:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049947259999999997 2024-09-17T07:28:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.956699+0000", 2024-09-17T07:28:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8498000000000002e-05 2024-09-17T07:28:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.956786+0000", 2024-09-17T07:28:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7186000000000002e-05 2024-09-17T07:28:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.958097+0000", 2024-09-17T07:28:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00131084 2024-09-17T07:28:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.960867+0000", 2024-09-17T07:28:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0027705 2024-09-17T07:28:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.960893+0000", 2024-09-17T07:28:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5633e-05 2024-09-17T07:28:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.960928+0000", 2024-09-17T07:28:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5320000000000001e-05 2024-09-17T07:28:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.951660+0000", 2024-09-17T07:28:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.294244849999998, 2024-09-17T07:28:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0093311599999999998, 2024-09-17T07:28:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-17T07:28:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951660+0000", 2024-09-17T07:28:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951660+0000", 2024-09-17T07:28:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951660+0000", 2024-09-17T07:28:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1300000000000001e-07 2024-09-17T07:28:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951663+0000", 2024-09-17T07:28:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3089999999999998e-06 2024-09-17T07:28:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951673+0000", 2024-09-17T07:28:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0158e-05 2024-09-17T07:28:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951677+0000", 2024-09-17T07:28:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-06 2024-09-17T07:28:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.957037+0000", 2024-09-17T07:28:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053598140000000001 2024-09-17T07:28:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.957083+0000", 2024-09-17T07:28:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6452000000000002e-05 2024-09-17T07:28:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.957160+0000", 2024-09-17T07:28:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6485999999999999e-05 2024-09-17T07:28:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.958591+0000", 2024-09-17T07:28:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014318449999999999 2024-09-17T07:28:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.960933+0000", 2024-09-17T07:28:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002341264 2024-09-17T07:28:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.960957+0000", 2024-09-17T07:28:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4312999999999999e-05 2024-09-17T07:28:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.960991+0000", 2024-09-17T07:28:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4106000000000002e-05 2024-09-17T07:28:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.951688+0000", 2024-09-17T07:28:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.294217119999999, 2024-09-17T07:28:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011520801000000001, 2024-09-17T07:28:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-17T07:28:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951688+0000", 2024-09-17T07:28:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951688+0000", 2024-09-17T07:28:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951688+0000", 2024-09-17T07:28:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1800000000000002e-07 2024-09-17T07:28:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951740+0000", 2024-09-17T07:28:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1635999999999998e-05 2024-09-17T07:28:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951746+0000", 2024-09-17T07:28:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.976e-06 2024-09-17T07:28:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951750+0000", 2024-09-17T07:28:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9580000000000001e-06 2024-09-17T07:28:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.957416+0000", 2024-09-17T07:28:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0056669169999999996 2024-09-17T07:28:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.957464+0000", 2024-09-17T07:28:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8000999999999997e-05 2024-09-17T07:28:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.957538+0000", 2024-09-17T07:28:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3646000000000006e-05 2024-09-17T07:28:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.958624+0000", 2024-09-17T07:28:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00108584 2024-09-17T07:28:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.963131+0000", 2024-09-17T07:28:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045072430000000002 2024-09-17T07:28:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.963171+0000", 2024-09-17T07:28:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9706999999999998e-05 2024-09-17T07:28:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.963208+0000", 2024-09-17T07:28:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7558999999999997e-05 2024-09-17T07:28:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.951756+0000", 2024-09-17T07:28:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.294148530000001, 2024-09-17T07:28:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011516192999999999, 2024-09-17T07:28:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-17T07:28:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951756+0000", 2024-09-17T07:28:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951756+0000", 2024-09-17T07:28:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951757+0000", 2024-09-17T07:28:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.53e-07 2024-09-17T07:28:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951759+0000", 2024-09-17T07:28:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6180000000000002e-06 2024-09-17T07:28:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951763+0000", 2024-09-17T07:28:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6389999999999998e-06 2024-09-17T07:28:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.951766+0000", 2024-09-17T07:28:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6759999999999999e-06 2024-09-17T07:28:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.958207+0000", 2024-09-17T07:28:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064418289999999996 2024-09-17T07:28:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.958255+0000", 2024-09-17T07:28:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7307e-05 2024-09-17T07:28:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.958335+0000", 2024-09-17T07:28:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0419000000000004e-05 2024-09-17T07:28:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.959168+0000", 2024-09-17T07:28:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083263300000000005 2024-09-17T07:28:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.963214+0000", 2024-09-17T07:28:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040466190000000004 2024-09-17T07:28:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.963239+0000", 2024-09-17T07:28:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.472e-05 2024-09-17T07:28:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.963272+0000", 2024-09-17T07:28:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3380000000000002e-05 2024-09-17T07:28:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:90 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:27.234019+0000", 2024-09-17T07:28:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.011885608, 2024-09-17T07:28:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0065092709999999996, 2024-09-17T07:28:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 90 2024-09-17T07:28:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234019+0000", 2024-09-17T07:28:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234019+0000", 2024-09-17T07:28:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234020+0000", 2024-09-17T07:28:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:28:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234107+0000", 2024-09-17T07:28:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7239999999999998e-05 2024-09-17T07:28:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234112+0000", 2024-09-17T07:28:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9710000000000003e-06 2024-09-17T07:28:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234115+0000", 2024-09-17T07:28:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5090000000000002e-06 2024-09-17T07:28:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234810+0000", 2024-09-17T07:28:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00069451799999999998 2024-09-17T07:28:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234858+0000", 2024-09-17T07:28:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8303999999999997e-05 2024-09-17T07:28:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234939+0000", 2024-09-17T07:28:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1119999999999996e-05 2024-09-17T07:28:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.236908+0000", 2024-09-17T07:28:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019691969999999998 2024-09-17T07:28:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.240425+0000", 2024-09-17T07:28:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003516585 2024-09-17T07:28:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.240474+0000", 2024-09-17T07:28:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8816999999999997e-05 2024-09-17T07:28:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.240528+0000", 2024-09-17T07:28:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4604000000000001e-05 2024-09-17T07:28:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:91 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:27.234126+0000", 2024-09-17T07:28:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.011778921000001, 2024-09-17T07:28:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0065039379999999999, 2024-09-17T07:28:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 91 2024-09-17T07:28:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234126+0000", 2024-09-17T07:28:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234126+0000", 2024-09-17T07:28:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234126+0000", 2024-09-17T07:28:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0699999999999997e-07 2024-09-17T07:28:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234129+0000", 2024-09-17T07:28:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7779999999999999e-06 2024-09-17T07:28:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234134+0000", 2024-09-17T07:28:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-17T07:28:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234137+0000", 2024-09-17T07:28:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9179999999999998e-06 2024-09-17T07:28:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.235369+0000", 2024-09-17T07:28:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00123175 2024-09-17T07:28:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.235412+0000", 2024-09-17T07:28:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3472e-05 2024-09-17T07:28:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.235482+0000", 2024-09-17T07:28:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9597999999999993e-05 2024-09-17T07:28:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.236956+0000", 2024-09-17T07:28:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00147448 2024-09-17T07:28:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.240536+0000", 2024-09-17T07:28:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035801349999999999 2024-09-17T07:28:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.240584+0000", 2024-09-17T07:28:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7815000000000002e-05 2024-09-17T07:28:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.240630+0000", 2024-09-17T07:28:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5813000000000003e-05 2024-09-17T07:28:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:27.234141+0000", 2024-09-17T07:28:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.011763430999999, 2024-09-17T07:28:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0065820660000000001, 2024-09-17T07:28:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 92 2024-09-17T07:28:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234141+0000", 2024-09-17T07:28:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234141+0000", 2024-09-17T07:28:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234142+0000", 2024-09-17T07:28:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5499999999999999e-07 2024-09-17T07:28:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234144+0000", 2024-09-17T07:28:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.962e-06 2024-09-17T07:28:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234149+0000", 2024-09-17T07:28:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2630000000000003e-06 2024-09-17T07:28:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234152+0000", 2024-09-17T07:28:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5659999999999998e-06 2024-09-17T07:28:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.235869+0000", 2024-09-17T07:28:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017163770000000001 2024-09-17T07:28:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.235934+0000", 2024-09-17T07:28:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4756000000000001e-05 2024-09-17T07:28:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.236009+0000", 2024-09-17T07:28:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5421999999999997e-05 2024-09-17T07:28:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.237353+0000", 2024-09-17T07:28:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013437670000000001 2024-09-17T07:28:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.240635+0000", 2024-09-17T07:28:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0032821730000000002 2024-09-17T07:28:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.240686+0000", 2024-09-17T07:28:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0810000000000003e-05 2024-09-17T07:28:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.240723+0000", 2024-09-17T07:28:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7715000000000001e-05 2024-09-17T07:28:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:27.234159+0000", 2024-09-17T07:28:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.011746103, 2024-09-17T07:28:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.006649421, 2024-09-17T07:28:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 93 2024-09-17T07:28:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234159+0000", 2024-09-17T07:28:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234159+0000", 2024-09-17T07:28:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234159+0000", 2024-09-17T07:28:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.67e-07 2024-09-17T07:28:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234202+0000", 2024-09-17T07:28:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2772999999999999e-05 2024-09-17T07:28:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234207+0000", 2024-09-17T07:28:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4260000000000002e-06 2024-09-17T07:28:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234210+0000", 2024-09-17T07:28:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.151e-06 2024-09-17T07:28:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.236248+0000", 2024-09-17T07:28:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020377669999999998 2024-09-17T07:28:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.236327+0000", 2024-09-17T07:28:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8825999999999994e-05 2024-09-17T07:28:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.236446+0000", 2024-09-17T07:28:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000118787 2024-09-17T07:28:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.237581+0000", 2024-09-17T07:28:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011357909999999999 2024-09-17T07:28:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.240730+0000", 2024-09-17T07:28:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003148316 2024-09-17T07:28:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.240775+0000", 2024-09-17T07:28:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5102999999999998e-05 2024-09-17T07:28:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.240808+0000", 2024-09-17T07:28:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3213999999999997e-05 2024-09-17T07:28:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:28:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:27.234215+0000", 2024-09-17T07:28:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.011689496999999, 2024-09-17T07:28:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0066363610000000003, 2024-09-17T07:28:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:28:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:28:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 94 2024-09-17T07:28:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234215+0000", 2024-09-17T07:28:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234215+0000", 2024-09-17T07:28:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234216+0000", 2024-09-17T07:28:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-17T07:28:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234218+0000", 2024-09-17T07:28:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:28:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234222+0000", 2024-09-17T07:28:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2300000000000002e-06 2024-09-17T07:28:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.234225+0000", 2024-09-17T07:28:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9909999999999998e-06 2024-09-17T07:28:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.236980+0000", 2024-09-17T07:28:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002754734 2024-09-17T07:28:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.237024+0000", 2024-09-17T07:28:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3893000000000001e-05 2024-09-17T07:28:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.237116+0000", 2024-09-17T07:28:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1855999999999996e-05 2024-09-17T07:28:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.237803+0000", 2024-09-17T07:28:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000687103 2024-09-17T07:28:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.240812+0000", 2024-09-17T07:28:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00300914 2024-09-17T07:28:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.240834+0000", 2024-09-17T07:28:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1648e-05 2024-09-17T07:28:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:27.240852+0000", 2024-09-17T07:28:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8046e-05 2024-09-17T07:28:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:45.370 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:45.372 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:45.371+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:45.372 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:28:45.473 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:45.471+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:45.481 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:45.481 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:45.481 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:28:45.481 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:45.490 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:28:45.574 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:45.573+0000 7f9a047b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33358) UID: 0 2024-09-17T07:28:45.639 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:45.639 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:45.639 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:28:45.639 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:28:45.639 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:45.648 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:28:45.689 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:45.687+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:45.775 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:45.775+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:45.798 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:45.798 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:28:45.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:28:45.798 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:28:45.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.798 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-17T07:28:45.798 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.564282+0000", 2024-09-17T07:28:45.798 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.231866062000002, 2024-09-17T07:28:45.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001156984, 2024-09-17T07:28:45.799 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.799 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564282+0000", 2024-09-17T07:28:45.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564282+0000", 2024-09-17T07:28:45.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564283+0000", 2024-09-17T07:28:45.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4099999999999998e-07 2024-09-17T07:28:45.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564480+0000", 2024-09-17T07:28:45.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000196524 2024-09-17T07:28:45.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564496+0000", 2024-09-17T07:28:45.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6160000000000001e-05 2024-09-17T07:28:45.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564501+0000", 2024-09-17T07:28:45.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4960000000000004e-06 2024-09-17T07:28:45.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564545+0000", 2024-09-17T07:28:45.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3389000000000001e-05 2024-09-17T07:28:45.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564558+0000", 2024-09-17T07:28:45.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3507e-05 2024-09-17T07:28:45.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565395+0000", 2024-09-17T07:28:45.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083644000000000004 2024-09-17T07:28:45.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565439+0000", 2024-09-17T07:28:45.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4727000000000001e-05 2024-09-17T07:28:45.802 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.803 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.803 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-17T07:28:45.803 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.564808+0000", 2024-09-17T07:28:45.803 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.231340786000001, 2024-09-17T07:28:45.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011919439999999999, 2024-09-17T07:28:45.803 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.803 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.803 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.803 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564808+0000", 2024-09-17T07:28:45.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564808+0000", 2024-09-17T07:28:45.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564808+0000", 2024-09-17T07:28:45.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6400000000000002e-07 2024-09-17T07:28:45.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564941+0000", 2024-09-17T07:28:45.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000132841 2024-09-17T07:28:45.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564950+0000", 2024-09-17T07:28:45.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0469999999999999e-06 2024-09-17T07:28:45.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564955+0000", 2024-09-17T07:28:45.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9459999999999997e-06 2024-09-17T07:28:45.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565002+0000", 2024-09-17T07:28:45.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6835000000000001e-05 2024-09-17T07:28:45.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565016+0000", 2024-09-17T07:28:45.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3619e-05 2024-09-17T07:28:45.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565928+0000", 2024-09-17T07:28:45.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00091201800000000001 2024-09-17T07:28:45.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566000+0000", 2024-09-17T07:28:45.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1973999999999999e-05 2024-09-17T07:28:45.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.807 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.807 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:22 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'12, mlcod=17'8)", 2024-09-17T07:28:45.807 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.565349+0000", 2024-09-17T07:28:45.807 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.230799337000001, 2024-09-17T07:28:45.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001151947, 2024-09-17T07:28:45.808 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.808 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.808 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565349+0000", 2024-09-17T07:28:45.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565349+0000", 2024-09-17T07:28:45.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565350+0000", 2024-09-17T07:28:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1099999999999995e-07 2024-09-17T07:28:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565465+0000", 2024-09-17T07:28:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011475 2024-09-17T07:28:45.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565474+0000", 2024-09-17T07:28:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9840000000000007e-06 2024-09-17T07:28:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565479+0000", 2024-09-17T07:28:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5099999999999998e-06 2024-09-17T07:28:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565534+0000", 2024-09-17T07:28:45.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4403000000000001e-05 2024-09-17T07:28:45.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565566+0000", 2024-09-17T07:28:45.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2262999999999999e-05 2024-09-17T07:28:45.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566440+0000", 2024-09-17T07:28:45.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087397600000000005 2024-09-17T07:28:45.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.812 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566501+0000", 2024-09-17T07:28:45.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.135e-05 2024-09-17T07:28:45.812 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.812 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.812 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.812 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:23 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'13, mlcod=17'8)", 2024-09-17T07:28:45.812 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.565838+0000", 2024-09-17T07:28:45.812 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.230310295999999, 2024-09-17T07:28:45.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011229479999999999, 2024-09-17T07:28:45.813 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.813 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565838+0000", 2024-09-17T07:28:45.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565838+0000", 2024-09-17T07:28:45.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565839+0000", 2024-09-17T07:28:45.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9500000000000002e-07 2024-09-17T07:28:45.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565927+0000", 2024-09-17T07:28:45.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8246999999999997e-05 2024-09-17T07:28:45.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565948+0000", 2024-09-17T07:28:45.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1019e-05 2024-09-17T07:28:45.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565954+0000", 2024-09-17T07:28:45.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6389999999999997e-06 2024-09-17T07:28:45.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566038+0000", 2024-09-17T07:28:45.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3969999999999997e-05 2024-09-17T07:28:45.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566056+0000", 2024-09-17T07:28:45.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799e-05 2024-09-17T07:28:45.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566926+0000", 2024-09-17T07:28:45.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087067700000000004 2024-09-17T07:28:45.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566961+0000", 2024-09-17T07:28:45.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4901999999999999e-05 2024-09-17T07:28:45.818 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.818 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.819 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.819 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:24 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'14, mlcod=17'8)", 2024-09-17T07:28:45.819 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.566286+0000", 2024-09-17T07:28:45.819 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.229862185999998, 2024-09-17T07:28:45.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001007825, 2024-09-17T07:28:45.819 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.819 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.819 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566286+0000", 2024-09-17T07:28:45.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566286+0000", 2024-09-17T07:28:45.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566287+0000", 2024-09-17T07:28:45.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4900000000000005e-07 2024-09-17T07:28:45.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566373+0000", 2024-09-17T07:28:45.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6323999999999995e-05 2024-09-17T07:28:45.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566388+0000", 2024-09-17T07:28:45.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4229e-05 2024-09-17T07:28:45.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566398+0000", 2024-09-17T07:28:45.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0275000000000001e-05 2024-09-17T07:28:45.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566531+0000", 2024-09-17T07:28:45.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133182 2024-09-17T07:28:45.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566546+0000", 2024-09-17T07:28:45.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4917e-05 2024-09-17T07:28:45.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.567257+0000", 2024-09-17T07:28:45.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00071128900000000004 2024-09-17T07:28:45.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.567294+0000", 2024-09-17T07:28:45.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6860000000000003e-05 2024-09-17T07:28:45.823 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.823 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.823 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.824 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:28 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'18, mlcod=17'16)", 2024-09-17T07:28:45.824 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.579193+0000", 2024-09-17T07:28:45.824 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.216954989000001, 2024-09-17T07:28:45.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093351199999999995, 2024-09-17T07:28:45.824 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.824 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.824 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.824 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.579193+0000", 2024-09-17T07:28:45.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.579193+0000", 2024-09-17T07:28:45.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.579196+0000", 2024-09-17T07:28:45.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2400000000000002e-06 2024-09-17T07:28:45.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.579212+0000", 2024-09-17T07:28:45.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.626e-05 2024-09-17T07:28:45.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.579244+0000", 2024-09-17T07:28:45.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2119000000000002e-05 2024-09-17T07:28:45.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.579259+0000", 2024-09-17T07:28:45.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.508e-05 2024-09-17T07:28:45.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.579387+0000", 2024-09-17T07:28:45.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012828199999999999 2024-09-17T07:28:45.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.579432+0000", 2024-09-17T07:28:45.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4861999999999998e-05 2024-09-17T07:28:45.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.580085+0000", 2024-09-17T07:28:45.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00065315900000000005 2024-09-17T07:28:45.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.580127+0000", 2024-09-17T07:28:45.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1510000000000001e-05 2024-09-17T07:28:45.828 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.828 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.828 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.828 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:53 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'35, mlcod=17'33)", 2024-09-17T07:28:45.829 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.666689+0000", 2024-09-17T07:28:45.829 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.129459281999999, 2024-09-17T07:28:45.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00094790100000000002, 2024-09-17T07:28:45.829 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.829 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.829 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.666689+0000", 2024-09-17T07:28:45.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.666689+0000", 2024-09-17T07:28:45.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.666690+0000", 2024-09-17T07:28:45.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:28:45.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.666763+0000", 2024-09-17T07:28:45.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3807000000000007e-05 2024-09-17T07:28:45.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.666773+0000", 2024-09-17T07:28:45.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3239999999999992e-06 2024-09-17T07:28:45.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.666779+0000", 2024-09-17T07:28:45.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.857e-06 2024-09-17T07:28:45.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.666816+0000", 2024-09-17T07:28:45.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7561999999999997e-05 2024-09-17T07:28:45.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.666829+0000", 2024-09-17T07:28:45.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.325e-05 2024-09-17T07:28:45.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.667594+0000", 2024-09-17T07:28:45.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076498199999999999 2024-09-17T07:28:45.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.667637+0000", 2024-09-17T07:28:45.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.269e-05 2024-09-17T07:28:45.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.833 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.833 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:59 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'41, mlcod=17'39)", 2024-09-17T07:28:45.833 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.948346+0000", 2024-09-17T07:28:45.833 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.847802602000002, 2024-09-17T07:28:45.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087197999999999996, 2024-09-17T07:28:45.833 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.834 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.834 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.948346+0000", 2024-09-17T07:28:45.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.948346+0000", 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.948348+0000", 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.813e-06 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.948363+0000", 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4876e-05 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.948391+0000", 2024-09-17T07:28:45.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8050000000000001e-05 2024-09-17T07:28:45.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.948411+0000", 2024-09-17T07:28:45.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0509999999999998e-05 2024-09-17T07:28:45.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.948538+0000", 2024-09-17T07:28:45.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000127131 2024-09-17T07:28:45.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.948580+0000", 2024-09-17T07:28:45.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2109e-05 2024-09-17T07:28:45.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.949186+0000", 2024-09-17T07:28:45.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000605953 2024-09-17T07:28:45.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.949218+0000", 2024-09-17T07:28:45.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1538000000000002e-05 2024-09-17T07:28:45.838 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.838 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.839 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.839 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-17T07:28:45.839 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.952425+0000", 2024-09-17T07:28:45.839 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.843723870000002, 2024-09-17T07:28:45.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00099157200000000003, 2024-09-17T07:28:45.839 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.839 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.839 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.952425+0000", 2024-09-17T07:28:45.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.952425+0000", 2024-09-17T07:28:45.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.952425+0000", 2024-09-17T07:28:45.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4200000000000001e-07 2024-09-17T07:28:45.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.952476+0000", 2024-09-17T07:28:45.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1053000000000001e-05 2024-09-17T07:28:45.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.952485+0000", 2024-09-17T07:28:45.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0769999999999997e-06 2024-09-17T07:28:45.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.952491+0000", 2024-09-17T07:28:45.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5319999999999997e-06 2024-09-17T07:28:45.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.952520+0000", 2024-09-17T07:28:45.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.974e-05 2024-09-17T07:28:45.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.952531+0000", 2024-09-17T07:28:45.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0747000000000001e-05 2024-09-17T07:28:45.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.953371+0000", 2024-09-17T07:28:45.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083967500000000001 2024-09-17T07:28:45.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.953416+0000", 2024-09-17T07:28:45.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:28:45.844 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.844 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.844 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.844 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-17T07:28:45.844 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.953642+0000", 2024-09-17T07:28:45.844 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.842506156999999, 2024-09-17T07:28:45.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00094047800000000002, 2024-09-17T07:28:45.844 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.845 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.845 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.845 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.953642+0000", 2024-09-17T07:28:45.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.953642+0000", 2024-09-17T07:28:45.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.953643+0000", 2024-09-17T07:28:45.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1899999999999998e-07 2024-09-17T07:28:45.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.953645+0000", 2024-09-17T07:28:45.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6359999999999998e-06 2024-09-17T07:28:45.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.953653+0000", 2024-09-17T07:28:45.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6660000000000008e-06 2024-09-17T07:28:45.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.953657+0000", 2024-09-17T07:28:45.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:28:45.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.953697+0000", 2024-09-17T07:28:45.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9653999999999997e-05 2024-09-17T07:28:45.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.953713+0000", 2024-09-17T07:28:45.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6481999999999999e-05 2024-09-17T07:28:45.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.954559+0000", 2024-09-17T07:28:45.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00084548100000000003 2024-09-17T07:28:45.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.954583+0000", 2024-09-17T07:28:45.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4034999999999999e-05 2024-09-17T07:28:45.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.849 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.849 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-17T07:28:45.849 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.954053+0000", 2024-09-17T07:28:45.849 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.842095028999999, 2024-09-17T07:28:45.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00096147500000000005, 2024-09-17T07:28:45.849 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.849 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.849 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.954053+0000", 2024-09-17T07:28:45.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.954053+0000", 2024-09-17T07:28:45.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.954054+0000", 2024-09-17T07:28:45.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3700000000000001e-07 2024-09-17T07:28:45.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.954186+0000", 2024-09-17T07:28:45.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013183699999999999 2024-09-17T07:28:45.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.954194+0000", 2024-09-17T07:28:45.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2709999999999992e-06 2024-09-17T07:28:45.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.954198+0000", 2024-09-17T07:28:45.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6789999999999998e-06 2024-09-17T07:28:45.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.954237+0000", 2024-09-17T07:28:45.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9295999999999998e-05 2024-09-17T07:28:45.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.954251+0000", 2024-09-17T07:28:45.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3709999999999999e-05 2024-09-17T07:28:45.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.954973+0000", 2024-09-17T07:28:45.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072227399999999996 2024-09-17T07:28:45.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.955015+0000", 2024-09-17T07:28:45.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:28:45.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.853 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.853 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:72 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'50, mlcod=17'46)", 2024-09-17T07:28:45.854 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.956104+0000", 2024-09-17T07:28:45.854 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.840044220999999, 2024-09-17T07:28:45.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010639600000000001, 2024-09-17T07:28:45.854 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.854 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956104+0000", 2024-09-17T07:28:45.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956104+0000", 2024-09-17T07:28:45.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956105+0000", 2024-09-17T07:28:45.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2300000000000001e-07 2024-09-17T07:28:45.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956161+0000", 2024-09-17T07:28:45.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6539999999999997e-05 2024-09-17T07:28:45.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956172+0000", 2024-09-17T07:28:45.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0328e-05 2024-09-17T07:28:45.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956178+0000", 2024-09-17T07:28:45.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4829999999999996e-06 2024-09-17T07:28:45.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956247+0000", 2024-09-17T07:28:45.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8365e-05 2024-09-17T07:28:45.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956261+0000", 2024-09-17T07:28:45.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4279e-05 2024-09-17T07:28:45.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.957135+0000", 2024-09-17T07:28:45.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087454099999999995 2024-09-17T07:28:45.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.957168+0000", 2024-09-17T07:28:45.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2801e-05 2024-09-17T07:28:45.858 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.858 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.858 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.858 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:74 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'52, mlcod=17'46)", 2024-09-17T07:28:45.858 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.957210+0000", 2024-09-17T07:28:45.858 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.838938666000001, 2024-09-17T07:28:45.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00085887099999999998, 2024-09-17T07:28:45.858 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.859 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.859 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.957210+0000", 2024-09-17T07:28:45.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.957210+0000", 2024-09-17T07:28:45.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.957210+0000", 2024-09-17T07:28:45.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8800000000000003e-07 2024-09-17T07:28:45.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.957215+0000", 2024-09-17T07:28:45.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1399999999999999e-06 2024-09-17T07:28:45.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.957224+0000", 2024-09-17T07:28:45.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3860000000000007e-06 2024-09-17T07:28:45.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.957229+0000", 2024-09-17T07:28:45.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4149999999999998e-06 2024-09-17T07:28:45.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.957267+0000", 2024-09-17T07:28:45.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7679999999999998e-05 2024-09-17T07:28:45.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.957278+0000", 2024-09-17T07:28:45.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1511e-05 2024-09-17T07:28:45.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.958044+0000", 2024-09-17T07:28:45.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076525200000000001 2024-09-17T07:28:45.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.958069+0000", 2024-09-17T07:28:45.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4998999999999999e-05 2024-09-17T07:28:45.862 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.862 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.862 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.863 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:78 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'56, mlcod=17'53)", 2024-09-17T07:28:45.863 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.964380+0000", 2024-09-17T07:28:45.863 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.831768123, 2024-09-17T07:28:45.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00099083900000000009, 2024-09-17T07:28:45.863 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.863 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.863 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.863 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.964380+0000", 2024-09-17T07:28:45.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.964380+0000", 2024-09-17T07:28:45.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.964381+0000", 2024-09-17T07:28:45.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1600000000000001e-07 2024-09-17T07:28:45.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.964468+0000", 2024-09-17T07:28:45.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6581000000000003e-05 2024-09-17T07:28:45.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.964478+0000", 2024-09-17T07:28:45.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0806e-05 2024-09-17T07:28:45.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.964485+0000", 2024-09-17T07:28:45.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7630000000000003e-06 2024-09-17T07:28:45.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.964534+0000", 2024-09-17T07:28:45.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9206000000000003e-05 2024-09-17T07:28:45.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.964556+0000", 2024-09-17T07:28:45.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1483000000000001e-05 2024-09-17T07:28:45.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.965340+0000", 2024-09-17T07:28:45.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078381200000000005 2024-09-17T07:28:45.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.965371+0000", 2024-09-17T07:28:45.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1572000000000002e-05 2024-09-17T07:28:45.867 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.867 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.867 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.867 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:79 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-17T07:28:45.867 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.964833+0000", 2024-09-17T07:28:45.867 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.831315873000001, 2024-09-17T07:28:45.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001014498, 2024-09-17T07:28:45.868 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.868 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.868 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.964833+0000", 2024-09-17T07:28:45.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.964833+0000", 2024-09-17T07:28:45.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.964833+0000", 2024-09-17T07:28:45.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.89e-07 2024-09-17T07:28:45.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.964913+0000", 2024-09-17T07:28:45.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9926999999999996e-05 2024-09-17T07:28:45.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.964929+0000", 2024-09-17T07:28:45.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6320999999999998e-05 2024-09-17T07:28:45.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.964934+0000", 2024-09-17T07:28:45.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5020000000000003e-06 2024-09-17T07:28:45.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.964977+0000", 2024-09-17T07:28:45.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2920999999999998e-05 2024-09-17T07:28:45.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.965003+0000", 2024-09-17T07:28:45.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6242e-05 2024-09-17T07:28:45.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.965821+0000", 2024-09-17T07:28:45.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00081831500000000001 2024-09-17T07:28:45.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.965847+0000", 2024-09-17T07:28:45.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5780999999999999e-05 2024-09-17T07:28:45.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.872 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.872 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:92 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'66, mlcod=17'61)", 2024-09-17T07:28:45.872 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:27.236050+0000", 2024-09-17T07:28:45.872 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.560098769, 2024-09-17T07:28:45.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000946991, 2024-09-17T07:28:45.872 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.872 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236050+0000", 2024-09-17T07:28:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236050+0000", 2024-09-17T07:28:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236050+0000", 2024-09-17T07:28:45.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999996e-07 2024-09-17T07:28:45.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236053+0000", 2024-09-17T07:28:45.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2270000000000002e-06 2024-09-17T07:28:45.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236070+0000", 2024-09-17T07:28:45.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6246e-05 2024-09-17T07:28:45.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236074+0000", 2024-09-17T07:28:45.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0049999999999998e-06 2024-09-17T07:28:45.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236125+0000", 2024-09-17T07:28:45.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0986000000000003e-05 2024-09-17T07:28:45.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236141+0000", 2024-09-17T07:28:45.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5974e-05 2024-09-17T07:28:45.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236976+0000", 2024-09-17T07:28:45.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083570300000000003 2024-09-17T07:28:45.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236997+0000", 2024-09-17T07:28:45.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.037e-05 2024-09-17T07:28:45.876 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.877 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.877 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.877 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:93 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'67, mlcod=17'61)", 2024-09-17T07:28:45.877 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:27.236498+0000", 2024-09-17T07:28:45.877 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.559650008999999, 2024-09-17T07:28:45.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000932386, 2024-09-17T07:28:45.877 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.877 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.877 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.877 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236498+0000", 2024-09-17T07:28:45.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236498+0000", 2024-09-17T07:28:45.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236499+0000", 2024-09-17T07:28:45.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.13e-07 2024-09-17T07:28:45.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.879 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236560+0000", 2024-09-17T07:28:45.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0523999999999999e-05 2024-09-17T07:28:45.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.879 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236582+0000", 2024-09-17T07:28:45.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2535e-05 2024-09-17T07:28:45.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236588+0000", 2024-09-17T07:28:45.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1480000000000003e-06 2024-09-17T07:28:45.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236622+0000", 2024-09-17T07:28:45.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3556999999999997e-05 2024-09-17T07:28:45.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.236633+0000", 2024-09-17T07:28:45.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1508000000000001e-05 2024-09-17T07:28:45.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.237408+0000", 2024-09-17T07:28:45.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00077485100000000001 2024-09-17T07:28:45.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:27.237431+0000", 2024-09-17T07:28:45.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2650000000000002e-05 2024-09-17T07:28:45.881 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.881 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.881 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.882 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T07:28:45.882 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:45.084937+0000", 2024-09-17T07:28:45.882 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.711211818, 2024-09-17T07:28:45.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0046658450000000001, 2024-09-17T07:28:45.882 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.882 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.882 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:45.882 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:45.882 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:45.882 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 11 2024-09-17T07:28:45.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.883 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.084937+0000", 2024-09-17T07:28:45.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.084937+0000", 2024-09-17T07:28:45.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.084939+0000", 2024-09-17T07:28:45.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1600000000000001e-06 2024-09-17T07:28:45.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.084950+0000", 2024-09-17T07:28:45.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1194e-05 2024-09-17T07:28:45.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.084967+0000", 2024-09-17T07:28:45.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7346999999999998e-05 2024-09-17T07:28:45.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.084978+0000", 2024-09-17T07:28:45.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0562e-05 2024-09-17T07:28:45.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.085049+0000", 2024-09-17T07:28:45.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0859999999999996e-05 2024-09-17T07:28:45.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:28:45.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.085055+0000", 2024-09-17T07:28:45.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.806e-06 2024-09-17T07:28:45.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.087859+0000", 2024-09-17T07:28:45.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028046429999999999 2024-09-17T07:28:45.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.087967+0000", 2024-09-17T07:28:45.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000107797 2024-09-17T07:28:45.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:28:45.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.088152+0000", 2024-09-17T07:28:45.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00018521799999999999 2024-09-17T07:28:45.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:45.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089243+0000", 2024-09-17T07:28:45.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010904910000000001 2024-09-17T07:28:45.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089532+0000", 2024-09-17T07:28:45.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00028896999999999999 2024-09-17T07:28:45.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089572+0000", 2024-09-17T07:28:45.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0618000000000003e-05 2024-09-17T07:28:45.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089603+0000", 2024-09-17T07:28:45.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0179e-05 2024-09-17T07:28:45.888 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.888 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.888 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.889 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:12 3.1b 3:d9441778:::rbd_header.1183951e9e6e:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T07:28:45.889 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:45.089737+0000", 2024-09-17T07:28:45.889 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.70641163600000001, 2024-09-17T07:28:45.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013646730000000001, 2024-09-17T07:28:45.889 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.889 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.889 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.889 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:45.889 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:45.889 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:45.889 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 12 2024-09-17T07:28:45.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.889 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089737+0000", 2024-09-17T07:28:45.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089737+0000", 2024-09-17T07:28:45.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089738+0000", 2024-09-17T07:28:45.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.24e-06 2024-09-17T07:28:45.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089745+0000", 2024-09-17T07:28:45.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.601e-06 2024-09-17T07:28:45.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089758+0000", 2024-09-17T07:28:45.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3291e-05 2024-09-17T07:28:45.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089764+0000", 2024-09-17T07:28:45.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4010000000000002e-06 2024-09-17T07:28:45.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089825+0000", 2024-09-17T07:28:45.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0729000000000001e-05 2024-09-17T07:28:45.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089967+0000", 2024-09-17T07:28:45.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000141977 2024-09-17T07:28:45.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:45.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.090211+0000", 2024-09-17T07:28:45.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00024368400000000001 2024-09-17T07:28:45.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:45.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.090928+0000", 2024-09-17T07:28:45.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00071752999999999997 2024-09-17T07:28:45.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.091038+0000", 2024-09-17T07:28:45.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010932100000000001 2024-09-17T07:28:45.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.091072+0000", 2024-09-17T07:28:45.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4857000000000002e-05 2024-09-17T07:28:45.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.091102+0000", 2024-09-17T07:28:45.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9042000000000001e-05 2024-09-17T07:28:45.895 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.895 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.895 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.895 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:25 3.1b 3:d9441778:::rbd_header.1183951e9e6e:head [watch watch cookie 140227662426288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T07:28:45.895 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:45.101961+0000", 2024-09-17T07:28:45.895 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.69418755099999996, 2024-09-17T07:28:45.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012175949999999999, 2024-09-17T07:28:45.895 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:45.895 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:45.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:45.896 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:45.896 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:45.896 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:45.896 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 25 2024-09-17T07:28:45.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.896 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:45.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:45.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.101961+0000", 2024-09-17T07:28:45.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:45.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.101961+0000", 2024-09-17T07:28:45.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:45.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:45.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.101962+0000", 2024-09-17T07:28:45.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2130000000000001e-06 2024-09-17T07:28:45.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:45.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.101968+0000", 2024-09-17T07:28:45.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.609e-06 2024-09-17T07:28:45.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:45.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.101984+0000", 2024-09-17T07:28:45.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.628e-05 2024-09-17T07:28:45.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:45.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.101993+0000", 2024-09-17T07:28:45.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6619999999999994e-06 2024-09-17T07:28:45.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:45.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.102043+0000", 2024-09-17T07:28:45.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9920999999999999e-05 2024-09-17T07:28:45.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.899 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:45.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.102095+0000", 2024-09-17T07:28:45.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2583000000000001e-05 2024-09-17T07:28:45.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.899 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:45.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.102212+0000", 2024-09-17T07:28:45.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000117125 2024-09-17T07:28:45.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.900 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:45.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.102914+0000", 2024-09-17T07:28:45.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00070177800000000002 2024-09-17T07:28:45.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.900 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:45.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.103032+0000", 2024-09-17T07:28:45.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000117714 2024-09-17T07:28:45.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.900 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:45.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.103090+0000", 2024-09-17T07:28:45.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8424999999999997e-05 2024-09-17T07:28:45.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:45.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:45.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:45.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.103179+0000", 2024-09-17T07:28:45.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8284999999999999e-05 2024-09-17T07:28:45.901 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.901 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.901 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.901 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:45.901 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:45.901 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:45.903 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:28:45.903 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:45.901+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:46.004 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:46.002+0000 7f9a047b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33358) UID: 0 2024-09-17T07:28:46.051 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:46.051 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:46.051 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:28:46.051 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:46.060 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:28:46.078 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-17T07:28:46.104 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:46.104+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:46.186 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:46.186 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:46.186 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:28:46.186 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:28:46.186 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:46.195 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:28:46.238 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:46.237+0000 7f9a047b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33358) UID: 0 2024-09-17T07:28:46.302 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:46.302 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:28:46.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:28:46.302 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:28:46.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.303 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4226.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-17T07:28:46.303 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:32.360310+0000", 2024-09-17T07:28:46.303 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.941979058999999, 2024-09-17T07:28:46.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000829469, 2024-09-17T07:28:46.303 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:46.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:46.303 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:46.303 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:46.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:46.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:32.360310+0000", 2024-09-17T07:28:46.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:46.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:46.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:32.360310+0000", 2024-09-17T07:28:46.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:46.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:46.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:32.360310+0000", 2024-09-17T07:28:46.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7800000000000001e-07 2024-09-17T07:28:46.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:46.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:32.360315+0000", 2024-09-17T07:28:46.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2490000000000001e-06 2024-09-17T07:28:46.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:46.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:32.360328+0000", 2024-09-17T07:28:46.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2326e-05 2024-09-17T07:28:46.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:46.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:32.360339+0000", 2024-09-17T07:28:46.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1504e-05 2024-09-17T07:28:46.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:46.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:32.360394+0000", 2024-09-17T07:28:46.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4948999999999999e-05 2024-09-17T07:28:46.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:46.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:32.360419+0000", 2024-09-17T07:28:46.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4922999999999998e-05 2024-09-17T07:28:46.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:46.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:32.361101+0000", 2024-09-17T07:28:46.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00068212899999999996 2024-09-17T07:28:46.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:46.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:32.361140+0000", 2024-09-17T07:28:46.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8210999999999997e-05 2024-09-17T07:28:46.308 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:46.308 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:46.308 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:46.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.308 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4226.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-17T07:28:46.308 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:33.363407+0000", 2024-09-17T07:28:46.308 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 12.938881823999999, 2024-09-17T07:28:46.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00081126400000000002, 2024-09-17T07:28:46.309 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:46.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:46.309 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:46.309 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:46.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:46.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:33.363407+0000", 2024-09-17T07:28:46.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:46.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:46.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:33.363407+0000", 2024-09-17T07:28:46.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:46.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:46.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:33.363407+0000", 2024-09-17T07:28:46.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7100000000000001e-07 2024-09-17T07:28:46.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:46.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:33.363415+0000", 2024-09-17T07:28:46.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9929999999999995e-06 2024-09-17T07:28:46.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:46.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:33.363427+0000", 2024-09-17T07:28:46.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1061e-05 2024-09-17T07:28:46.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:46.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:33.363439+0000", 2024-09-17T07:28:46.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2417e-05 2024-09-17T07:28:46.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:46.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:33.363496+0000", 2024-09-17T07:28:46.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7093999999999999e-05 2024-09-17T07:28:46.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:46.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:33.363514+0000", 2024-09-17T07:28:46.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8093000000000001e-05 2024-09-17T07:28:46.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:46.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:33.364180+0000", 2024-09-17T07:28:46.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00066606399999999996 2024-09-17T07:28:46.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:46.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:33.364219+0000", 2024-09-17T07:28:46.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8371000000000002e-05 2024-09-17T07:28:46.313 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:46.313 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:46.313 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:46.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.314 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:9 3.1a e26/23 3:5cea7035:::rbd_info:head v 26'2, mlcod=0'0)", 2024-09-17T07:28:46.314 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:45.080421+0000", 2024-09-17T07:28:46.314 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.2218677440000001, 2024-09-17T07:28:46.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000852103, 2024-09-17T07:28:46.314 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:46.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:46.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:46.314 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:46.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:46.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.080421+0000", 2024-09-17T07:28:46.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:46.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:46.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.080421+0000", 2024-09-17T07:28:46.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:46.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:46.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.080422+0000", 2024-09-17T07:28:46.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.97e-07 2024-09-17T07:28:46.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:46.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.080428+0000", 2024-09-17T07:28:46.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6969999999999996e-06 2024-09-17T07:28:46.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:46.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.080441+0000", 2024-09-17T07:28:46.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2962000000000001e-05 2024-09-17T07:28:46.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:46.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.080450+0000", 2024-09-17T07:28:46.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0990000000000005e-06 2024-09-17T07:28:46.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:46.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.080504+0000", 2024-09-17T07:28:46.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3372999999999999e-05 2024-09-17T07:28:46.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:46.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.080521+0000", 2024-09-17T07:28:46.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7716000000000001e-05 2024-09-17T07:28:46.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:46.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.081230+0000", 2024-09-17T07:28:46.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00070869200000000002 2024-09-17T07:28:46.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:46.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:46.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:46.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.081273+0000", 2024-09-17T07:28:46.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3367e-05 2024-09-17T07:28:46.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:46.319 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:46.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:46.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:46.319 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:46.319 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:46.320 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:28:46.321 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:46.320+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:46.321 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:28:46.421 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:46.420+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:46.472 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:46.472 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:46.472 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:28:46.472 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:46.481 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:28:46.501 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:28:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:28:46.532 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:28:46.532 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:28:46.532 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:46.530+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:46.550 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:28:46.550 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:28:46.567 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:28:46.567 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:28:46.584 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:28:46.585 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:28:46.610 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:46.611 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:46.615 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:28:46.615 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:28:46.619 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:28:46.623 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:46.622+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:46.637 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:28:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:28:46.654 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:28:46.654 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:28:46.724 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:46.722+0000 7f9a047b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33358) UID: 0 2024-09-17T07:28:46.747 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:46.747 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:46.756 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:28:46.825 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:46.824+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:46.913 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:46.914 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:46.922 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:28:46.925 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:46.924+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:47.026 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:47.024+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:47.029 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:47.029 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:47.038 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:28:47.070 INFO:teuthology.orchestra.run.smithi045.stdout:2 write 0x59ee2e2 thru 0x59f2ebd (0x4bdc bytes) 2024-09-17T07:28:47.087 INFO:teuthology.orchestra.run.smithi045.stdout:3 write 0xbe30330 thru 0xbe33629 (0x32fa bytes) 2024-09-17T07:28:47.091 INFO:teuthology.orchestra.run.smithi045.stdout:truncating to largest ever: 0x603ea5a 2024-09-17T07:28:47.098 INFO:teuthology.orchestra.run.smithi045.stdout:4 trunc from 0xbe3362a to 0x603ea5a 2024-09-17T07:28:47.108 INFO:teuthology.orchestra.run.smithi045.stdout:5 read 0x258d7a7 thru 0x258f57b (0x1dd5 bytes) 2024-09-17T07:28:47.118 INFO:teuthology.orchestra.run.smithi045.stdout:6 writesame 0x988bec9 thru 0x9898e19 data_size 0x3(0xcf51 bytes) 2024-09-17T07:28:47.126 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:47.125+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:47.129 INFO:teuthology.orchestra.run.smithi045.stdout:7 write 0xe506c33 thru 0xe50cac6 (0x5e94 bytes) 2024-09-17T07:28:47.133 INFO:teuthology.orchestra.run.smithi045.stdout:8 read 0xd60a34e thru 0xd6191c1 (0xee74 bytes) 2024-09-17T07:28:47.133 INFO:teuthology.orchestra.run.smithi045.stdout:9 punch from 0x5517e48 to 0x551b9ee, (0x3ba6 bytes) 2024-09-17T07:28:47.133 INFO:teuthology.orchestra.run.smithi045.stdout:truncating to largest ever: 0x9a1a4be 2024-09-17T07:28:47.138 INFO:teuthology.orchestra.run.smithi045.stdout:10 trunc from 0xe50cac7 to 0x9a1a4be 2024-09-17T07:28:47.148 INFO:teuthology.orchestra.run.smithi045.stdout:11 punch from 0x367082b to 0x367d78a, (0xcf5f bytes) 2024-09-17T07:28:47.148 INFO:teuthology.orchestra.run.smithi045.stdout:12 write 0x46823ea thru 0x4686bb3 (0x47ca bytes) 2024-09-17T07:28:47.151 INFO:teuthology.orchestra.run.smithi045.stdout:13 punch from 0x73adfeb to 0x73bcafe, (0xeb13 bytes) 2024-09-17T07:28:47.151 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:47.151 INFO:teuthology.orchestra.run.smithi045.stdout:15 write 0x5e7cf16 thru 0x5e7ed71 (0x1e5c bytes) 2024-09-17T07:28:47.154 INFO:teuthology.orchestra.run.smithi045.stdout:16 clone 1 order 20 su 32768 sc 6 2024-09-17T07:28:47.169 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T07:28:47.169 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:28:47.169 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-17T07:28:47.169 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T07:28:47.170 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T07:28:47.170 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-17T07:28:47.191 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:47.192 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:47.201 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:28:47.245 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:47.244+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:47.278 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:47.278 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T07:28:47.278 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:47.311 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:47.311 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:47.319 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:28:47.361 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:47.361+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:47.429 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:47.429+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:47.463 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:47.463 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:47.471 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:28:47.555 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:47.555+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:47.614 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:47.615 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:47.623 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:28:47.663 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:47.663+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:47.733 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:47.732+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:47.767 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:47.767 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:47.776 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:28:47.833 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:47.833+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:47.878 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:47.878 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:47.887 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:28:47.937 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:47.933+0000 7f9a047b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33358) UID: 0 2024-09-17T07:28:48.030 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:48.030 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:48.035 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:48.035+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:48.038 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:28:48.092 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T07:28:48.136 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:48.135+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:48.186 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:48.186 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:48.195 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:28:48.236 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:48.235+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:48.265 INFO:teuthology.orchestra.run.smithi045.stdout:17 write 0xc6fc9cf thru 0xc7009a4 (0x3fd6 bytes) 2024-09-17T07:28:48.275 INFO:teuthology.orchestra.run.smithi045.stdout:truncating to largest ever: 0xbdda370 2024-09-17T07:28:48.275 INFO:teuthology.orchestra.run.smithi045.stdout:18 trunc from 0xc7009a5 to 0xbdda370 2024-09-17T07:28:48.292 INFO:teuthology.orchestra.run.smithi045.stdout:19 write 0xa1b84e0 thru 0xa1c58a2 (0xd3c3 bytes) 2024-09-17T07:28:48.298 INFO:teuthology.orchestra.run.smithi045.stdout:20 read 0xb4ef318 thru 0xb4fec47 (0xf930 bytes) 2024-09-17T07:28:48.299 INFO:teuthology.orchestra.run.smithi045.stdout:21 read 0x5a09cd4 thru 0x5a0bff0 (0x231d bytes) 2024-09-17T07:28:48.299 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:48.299 INFO:teuthology.orchestra.run.smithi045.stdout:23 write 0x7ffc567 thru 0x8008149 (0xbbe3 bytes) 2024-09-17T07:28:48.315 INFO:teuthology.orchestra.run.smithi045.stdout:24 writesame 0xc57e413 thru 0xc58270f data_size 0x1(0x42fd bytes) 2024-09-17T07:28:48.326 INFO:teuthology.orchestra.run.smithi045.stdout:25 trunc from 0xc582710 to 0x1cca67e 2024-09-17T07:28:48.337 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:48.337+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:48.348 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:48.348 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:48.355 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:48.358 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:28:48.376 INFO:teuthology.orchestra.run.smithi045.stdout:27 write 0xe89c7a7 thru 0xe8a5745 (0x8f9f bytes) 2024-09-17T07:28:48.379 INFO:teuthology.orchestra.run.smithi045.stdout:29 punch from 0x49f2017 to 0x49fc5b6, (0xa59f bytes) 2024-09-17T07:28:48.390 INFO:teuthology.orchestra.run.smithi045.stdout:31 trunc from 0xe8a5746 to 0x55926ab 2024-09-17T07:28:48.399 INFO:teuthology.orchestra.run.smithi045.stdout:32 read 0x4b9b10f thru 0x4b9ffd9 (0x4ecb bytes) 2024-09-17T07:28:48.399 INFO:teuthology.orchestra.run.smithi045.stdout:34 punch from 0x256022c to 0x256872b, (0x84ff bytes) 2024-09-17T07:28:48.399 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:48.399 INFO:teuthology.orchestra.run.smithi045.stdout:38 read 0x37b3f82 thru 0x37b4560 (0x5df bytes) 2024-09-17T07:28:48.410 INFO:teuthology.orchestra.run.smithi045.stdout:39 write 0x8e74659 thru 0x8e8319d (0xeb45 bytes) 2024-09-17T07:28:48.418 INFO:teuthology.orchestra.run.smithi045.stdout:40 read 0x6ac2270 thru 0x6ad0d38 (0xeac9 bytes) 2024-09-17T07:28:48.418 INFO:teuthology.orchestra.run.smithi045.stdout:41 punch from 0x8222345 to 0x82252e3, (0x2f9e bytes) 2024-09-17T07:28:48.418 INFO:teuthology.orchestra.run.smithi045.stdout:42 punch from 0x50f17bb to 0x50fcd7e, (0xb5c3 bytes) 2024-09-17T07:28:48.418 INFO:teuthology.orchestra.run.smithi045.stdout:truncating to largest ever: 0xcdb2e64 2024-09-17T07:28:48.422 INFO:teuthology.orchestra.run.smithi045.stdout:43 trunc from 0x8e8319e to 0xcdb2e64 2024-09-17T07:28:48.435 INFO:teuthology.orchestra.run.smithi045.stdout:44 writesame 0xe6aeeed thru 0xe6b962f data_size 0x7(0xa743 bytes) 2024-09-17T07:28:48.435 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:48.435 INFO:teuthology.orchestra.run.smithi045.stdout:47 write 0x30a1308 thru 0x30a571f (0x4418 bytes) 2024-09-17T07:28:48.438 INFO:teuthology.orchestra.run.smithi045.stdout:49 write 0xa70d486 thru 0xa71854e (0xb0c9 bytes) 2024-09-17T07:28:48.441 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:48.441+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:truncating to largest ever: 0xe5a5540 2024-09-17T07:28:48.443 INFO:teuthology.orchestra.run.smithi045.stdout:52 trunc from 0xe6b9630 to 0xe5a5540 2024-09-17T07:28:48.448 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:48.448 INFO:teuthology.orchestra.run.smithi045.stdout:54 writesame 0x809e899 thru 0x80a588c data_size 0xa(0x6ff4 bytes) 2024-09-17T07:28:48.452 INFO:teuthology.orchestra.run.smithi045.stdout:55 write 0x16f1896 thru 0x16fa9d5 (0x9140 bytes) 2024-09-17T07:28:48.457 INFO:teuthology.orchestra.run.smithi045.stdout:56 punch from 0xd963340 to 0xd967a1b, (0x46db bytes) 2024-09-17T07:28:48.457 INFO:teuthology.orchestra.run.smithi045.stdout:57 read 0xe459309 thru 0xe468cdc (0xf9d4 bytes) 2024-09-17T07:28:48.457 INFO:teuthology.orchestra.run.smithi045.stdout:58 read 0x435a048 thru 0x435af5d (0xf16 bytes) 2024-09-17T07:28:48.457 INFO:teuthology.orchestra.run.smithi045.stdout:60 read 0x74a09b1 thru 0x74a8b18 (0x8168 bytes) 2024-09-17T07:28:48.457 INFO:teuthology.orchestra.run.smithi045.stdout:61 punch from 0xd6037d5 to 0xd612e8b, (0xf6b6 bytes) 2024-09-17T07:28:48.457 INFO:teuthology.orchestra.run.smithi045.stdout:62 write 0x1781b9d thru 0x178e651 (0xcab5 bytes) 2024-09-17T07:28:48.474 INFO:teuthology.orchestra.run.smithi045.stdout:63 trunc from 0xe5a5540 to 0x9ef369 2024-09-17T07:28:48.504 INFO:teuthology.orchestra.run.smithi045.stdout:64 write 0x2e8d118 thru 0x2e901f3 (0x30dc bytes) 2024-09-17T07:28:48.509 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:48.511 INFO:teuthology.orchestra.run.smithi045.stdout:66 trunc from 0x2e901f4 to 0x1909800 2024-09-17T07:28:48.515 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:48.516 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:48.525 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:28:48.534 INFO:teuthology.orchestra.run.smithi045.stdout:67 write 0x42f58c0 thru 0x4301d93 (0xc4d4 bytes) 2024-09-17T07:28:48.539 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:48.538+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:48.544 INFO:teuthology.orchestra.run.smithi045.stdout:68 read 0x2ebdc5b thru 0x2ebf936 (0x1cdc bytes) 2024-09-17T07:28:48.562 INFO:teuthology.orchestra.run.smithi045.stdout:69 write 0xe4ee805 thru 0xe4f1939 (0x3135 bytes) 2024-09-17T07:28:48.567 INFO:teuthology.orchestra.run.smithi045.stdout:70 punch from 0xc8b4760 to 0xc8c458e, (0xfe2e bytes) 2024-09-17T07:28:48.568 INFO:teuthology.orchestra.run.smithi045.stdout:71 read 0x13bd6ed thru 0x13be6f6 (0x100a bytes) 2024-09-17T07:28:48.568 INFO:teuthology.orchestra.run.smithi045.stdout:72 read 0x890e0ae thru 0x891047b (0x23ce bytes) 2024-09-17T07:28:48.568 INFO:teuthology.orchestra.run.smithi045.stdout:74 read 0x9986bd3 thru 0x998cc6e (0x609c bytes) 2024-09-17T07:28:48.568 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:48.568 INFO:teuthology.orchestra.run.smithi045.stdout:76 read 0x74cd8d5 thru 0x74dc306 (0xea32 bytes) 2024-09-17T07:28:48.572 INFO:teuthology.orchestra.run.smithi045.stdout:77 trunc from 0xe4f193a to 0x9a11fc0 2024-09-17T07:28:48.600 INFO:teuthology.orchestra.run.smithi045.stdout:78 write 0xbe021e1 thru 0xbe10fbe (0xedde bytes) 2024-09-17T07:28:48.607 INFO:teuthology.orchestra.run.smithi045.stdout:79 writesame 0x3f308fc thru 0x3f374c3 data_size 0x4(0x6bc8 bytes) 2024-09-17T07:28:48.620 INFO:teuthology.orchestra.run.smithi045.stdout:80 write 0xc367ca1 thru 0xc374398 (0xc6f8 bytes) 2024-09-17T07:28:48.629 INFO:teuthology.orchestra.run.smithi045.stdout:81 read 0x458634d thru 0x4593b93 (0xd847 bytes) 2024-09-17T07:28:48.640 INFO:teuthology.orchestra.run.smithi045.stdout:82 trunc from 0xc374399 to 0x2562be6 2024-09-17T07:28:48.640 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:48.639+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:48.666 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:28:48.671 INFO:teuthology.orchestra.run.smithi045.stdout:83 writesame 0x5f1bddc thru 0x5f23604 data_size 0x1(0x7829 bytes) 2024-09-17T07:28:48.676 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:48.676 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:48.685 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:28:48.686 INFO:teuthology.orchestra.run.smithi045.stdout:84 write 0xdad188c thru 0xdad803e (0x67b3 bytes) 2024-09-17T07:28:48.691 INFO:teuthology.orchestra.run.smithi045.stdout:85 read 0x268c754 thru 0x268e358 (0x1c05 bytes) 2024-09-17T07:28:48.692 INFO:teuthology.orchestra.run.smithi045.stdout:86 read 0x3eb2f36 thru 0x3ec2f13 (0xffde bytes) 2024-09-17T07:28:48.692 INFO:teuthology.orchestra.run.smithi045.stdout:87 read 0xc1b232c thru 0xc1ba602 (0x82d7 bytes) 2024-09-17T07:28:48.692 INFO:teuthology.orchestra.run.smithi045.stdout:88 read 0xf17bb0 thru 0xf1ae81 (0x32d2 bytes) 2024-09-17T07:28:48.692 INFO:teuthology.orchestra.run.smithi045.stdout:89 read 0x4a45229 thru 0x4a536f3 (0xe4cb bytes) 2024-09-17T07:28:48.692 INFO:teuthology.orchestra.run.smithi045.stdout:90 writesame 0x3ecf59 thru 0x3f575c data_size 0xa(0x8804 bytes) 2024-09-17T07:28:48.697 INFO:teuthology.orchestra.run.smithi045.stdout:91 write 0x1c749da thru 0x1c7be5d (0x7484 bytes) 2024-09-17T07:28:48.715 INFO:teuthology.orchestra.run.smithi045.stdout:92 trunc from 0xdad803f to 0x4710efa 2024-09-17T07:28:48.729 INFO:teuthology.orchestra.run.smithi045.stdout:93 writesame 0x485d635 thru 0x4863691 data_size 0x3(0x605d bytes) 2024-09-17T07:28:48.730 INFO:teuthology.orchestra.run.smithi045.stdout:94 punch from 0x2662b45 to 0x2666e17, (0x42d2 bytes) 2024-09-17T07:28:48.741 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:48.740+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:48.743 INFO:teuthology.orchestra.run.smithi045.stdout:95 write 0xb53936d thru 0xb543f41 (0xabd5 bytes) 2024-09-17T07:28:48.746 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:48.746 INFO:teuthology.orchestra.run.smithi045.stdout:98 writesame 0xa3befbf thru 0xa3c6248 data_size 0x51(0x728a bytes) 2024-09-17T07:28:48.762 INFO:teuthology.orchestra.run.smithi045.stdout:100 trunc from 0xb543f42 to 0x1ea0523 2024-09-17T07:28:48.773 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:48.783 INFO:teuthology.orchestra.run.smithi045.stdout:105 write 0x54a5d5f thru 0x54ad248 (0x74ea bytes) 2024-09-17T07:28:48.786 INFO:teuthology.orchestra.run.smithi045.stdout:106 write 0x4c0923d thru 0x4c0d59c (0x4360 bytes) 2024-09-17T07:28:48.788 INFO:teuthology.orchestra.run.smithi045.stdout:107 read 0x316cc14 thru 0x31710a5 (0x4492 bytes) 2024-09-17T07:28:48.789 INFO:teuthology.orchestra.run.smithi045.stdout:109 read 0x24def16 thru 0x24e4c11 (0x5cfc bytes) 2024-09-17T07:28:48.798 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:48.798 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:48.799 INFO:teuthology.orchestra.run.smithi045.stdout:110 writesame 0x97b5274 thru 0x97c18fe data_size 0x35(0xc68b bytes) 2024-09-17T07:28:48.805 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:28:48.805 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:28:48.807 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:28:48.809 INFO:teuthology.orchestra.run.smithi045.stdout:113 trunc from 0x97c18ff to 0x285987 2024-09-17T07:28:48.825 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:28:48.826 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:28:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:114 trunc from 0x285987 to 0x10db25f 2024-09-17T07:28:48.842 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:48.841+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:48.843 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:28:48.844 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:28:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:115 trunc from 0x10db25f to 0xb63ffee 2024-09-17T07:28:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:truncating to largest ever: 0xe9f7fd3 2024-09-17T07:28:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:116 trunc from 0xb63ffee to 0xe9f7fd3 2024-09-17T07:28:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:117 writesame 0x44d2f24 thru 0x44d6109 data_size 0x3(0x31e6 bytes) 2024-09-17T07:28:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:118 read 0xa46a142 thru 0xa474e4d (0xad0c bytes) 2024-09-17T07:28:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:120 write 0x959fde8 thru 0x95aaf5d (0xb176 bytes) 2024-09-17T07:28:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:121 read 0x3d4ca73 thru 0x3d5b0ab (0xe639 bytes) 2024-09-17T07:28:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:123 writesame 0x729ea5f thru 0x72aaa56 data_size 0x4(0xbff8 bytes) 2024-09-17T07:28:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:124 read 0x7eecd33 thru 0x7ef859a (0xb868 bytes) 2024-09-17T07:28:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:125 writesame 0x8a3d80f thru 0x8a3e0ed data_size 0x1(0x8df bytes) 2024-09-17T07:28:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:126 read 0xad6238d thru 0xad64f33 (0x2ba7 bytes) 2024-09-17T07:28:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:128 write 0x63d2c55 thru 0x63e1104 (0xe4b0 bytes) 2024-09-17T07:28:48.903 INFO:teuthology.orchestra.run.smithi045.stdout:129 write 0x4bbf78b thru 0x4bc145a (0x1cd0 bytes) 2024-09-17T07:28:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:130 trunc from 0xe9f7fd3 to 0x34366d0 2024-09-17T07:28:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:131 writesame 0x48fa4a thru 0x49eca7 data_size 0x36(0xf25e bytes) 2024-09-17T07:28:48.943 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:48.941+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:133 write 0x546d46d thru 0x5478d78 (0xb90c bytes) 2024-09-17T07:28:48.960 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:48.960 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:48.962 INFO:teuthology.orchestra.run.smithi045.stdout:136 punch from 0x192e6ad to 0x192f910, (0x1263 bytes) 2024-09-17T07:28:48.968 INFO:teuthology.orchestra.run.smithi045.stdout:137 writesame 0x561af1e thru 0x56266a6 data_size 0x1(0xb789 bytes) 2024-09-17T07:28:48.970 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:28:48.971 INFO:teuthology.orchestra.run.smithi045.stdout:141 read 0xbbf1a9 thru 0xbc2937 (0x378f bytes) 2024-09-17T07:28:48.974 INFO:teuthology.orchestra.run.smithi045.stdout:142 trunc from 0x56266a7 to 0x1482191 2024-09-17T07:28:48.984 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:48.985 INFO:teuthology.orchestra.run.smithi045.stdout:144 writesame 0x4d7a2a thru 0x4de3b4 data_size 0x1(0x698b bytes) 2024-09-17T07:28:49.006 INFO:teuthology.orchestra.run.smithi045.stdout:145 writesame 0x3b9e5fb thru 0x3ba0c23 data_size 0x1(0x2629 bytes) 2024-09-17T07:28:49.020 INFO:teuthology.orchestra.run.smithi045.stdout:146 write 0xc19cf25 thru 0xc1ac145 (0xf221 bytes) 2024-09-17T07:28:49.025 INFO:teuthology.orchestra.run.smithi045.stdout:147 read 0x6afd40a thru 0x6b0c4c2 (0xf0b9 bytes) 2024-09-17T07:28:49.025 INFO:teuthology.orchestra.run.smithi045.stdout:148 writesame 0xa672f08 thru 0xa675f99 data_size 0x1(0x3092 bytes) 2024-09-17T07:28:49.032 INFO:teuthology.orchestra.run.smithi045.stdout:150 read 0x980f253 thru 0x9810477 (0x1225 bytes) 2024-09-17T07:28:49.032 INFO:teuthology.orchestra.run.smithi045.stdout:152 write 0xa73cc44 thru 0xa73f193 (0x2550 bytes) 2024-09-17T07:28:49.034 INFO:teuthology.orchestra.run.smithi045.stdout:153 write 0x4c17080 thru 0x4c19ec2 (0x2e43 bytes) 2024-09-17T07:28:49.038 INFO:teuthology.orchestra.run.smithi045.stdout:154 writesame 0x6c4446d thru 0x6c520bf data_size 0x1b(0xdc53 bytes) 2024-09-17T07:28:49.043 INFO:teuthology.orchestra.run.smithi045.stdout:155 write 0xb269dc4 thru 0xb273395 (0x95d2 bytes) 2024-09-17T07:28:49.043 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:49.043+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:49.047 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:49.047 INFO:teuthology.orchestra.run.smithi045.stdout:157 punch from 0xa775d74 to 0xa7852c0, (0xf54c bytes) 2024-09-17T07:28:49.047 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:49.047 INFO:teuthology.orchestra.run.smithi045.stdout:159 write 0x7f3ebc3 thru 0x7f4d355 (0xe793 bytes) 2024-09-17T07:28:49.052 INFO:teuthology.orchestra.run.smithi045.stdout:160 punch from 0xa16da76 to 0xa17a0ed, (0xc677 bytes) 2024-09-17T07:28:49.063 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:28:49.063 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:28:49.064 INFO:teuthology.orchestra.run.smithi045.stdout:161 trunc from 0xc1ac146 to 0x1d83e42 2024-09-17T07:28:49.082 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:28:49.082 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:28:49.089 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:49.090 INFO:teuthology.orchestra.run.smithi045.stdout:163 trunc from 0x1d83e42 to 0x2c57dff 2024-09-17T07:28:49.091 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:49.092 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:49.092 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:28:49.092 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:49.098 INFO:teuthology.orchestra.run.smithi045.stdout:164 read 0x223337b thru 0x223bd3e (0x89c4 bytes) 2024-09-17T07:28:49.098 INFO:teuthology.orchestra.run.smithi045.stdout:165 punch from 0x50e69f to 0x514222, (0x5b83 bytes) 2024-09-17T07:28:49.100 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-17T07:28:49.104 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:28:49.106 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:28:49.106 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:28:49.108 INFO:teuthology.orchestra.run.smithi045.stdout:166 write 0x592560a thru 0x592b160 (0x5b57 bytes) 2024-09-17T07:28:49.118 INFO:teuthology.orchestra.run.smithi045.stdout:169 trunc from 0x592b161 to 0x22ed14b 2024-09-17T07:28:49.128 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:28:49.128 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:28:49.140 INFO:teuthology.orchestra.run.smithi045.stdout:173 trunc from 0x22ed14b to 0xbc9d608 2024-09-17T07:28:49.144 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:49.143+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:49.145 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:28:49.146 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:28:49.146 INFO:teuthology.orchestra.run.smithi045.stdout:174 write 0xacaad08 thru 0xacb295f (0x7c58 bytes) 2024-09-17T07:28:49.154 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:49.154 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:49.154 INFO:teuthology.orchestra.run.smithi045.stdout:179 read 0xb0edb1b thru 0xb0faea6 (0xd38c bytes) 2024-09-17T07:28:49.154 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:49.154 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:49.154 INFO:teuthology.orchestra.run.smithi045.stdout:184 read 0x18a0312 thru 0x18a33fe (0x30ed bytes) 2024-09-17T07:28:49.154 INFO:teuthology.orchestra.run.smithi045.stdout:186 write 0x1fffbb5 thru 0x200dbe1 (0xe02d bytes) 2024-09-17T07:28:49.163 INFO:teuthology.orchestra.run.smithi045.stdout:188 write 0x6c08d04 thru 0x6c09948 (0xc45 bytes) 2024-09-17T07:28:49.169 INFO:teuthology.orchestra.run.smithi045.stdout:189 punch from 0x156c6f3 to 0x1575317, (0x8c24 bytes) 2024-09-17T07:28:49.169 INFO:teuthology.orchestra.run.smithi045.stdout:190 writesame 0x9997340 thru 0x999aa31 data_size 0xd(0x36f2 bytes) 2024-09-17T07:28:49.175 INFO:teuthology.orchestra.run.smithi045.stdout:192 writesame 0x4c2bf75 thru 0x4c2cc81 data_size 0x1(0xd0d bytes) 2024-09-17T07:28:49.180 INFO:teuthology.orchestra.run.smithi045.stdout:193 punch from 0x70dd20 to 0x716129, (0x8409 bytes) 2024-09-17T07:28:49.180 INFO:teuthology.orchestra.run.smithi045.stdout:194 read 0x9c362e8 thru 0x9c3cfe9 (0x6d02 bytes) 2024-09-17T07:28:49.180 INFO:teuthology.orchestra.run.smithi045.stdout:195 read 0x71ae822 thru 0x71b50f0 (0x68cf bytes) 2024-09-17T07:28:49.180 INFO:teuthology.orchestra.run.smithi045.stdout:197 writesame 0xd39b43 thru 0xd45267 data_size 0x1(0xb725 bytes) 2024-09-17T07:28:49.195 INFO:teuthology.orchestra.run.smithi045.stdout:198 read 0x34d4df3 thru 0x34deca0 (0x9eae bytes) 2024-09-17T07:28:49.195 INFO:teuthology.orchestra.run.smithi045.stdout:200 write 0x73eb948 thru 0x73f8c09 (0xd2c2 bytes) 2024-09-17T07:28:49.202 INFO:teuthology.orchestra.run.smithi045.stdout:201 read 0x54e7943 thru 0x54f6b79 (0xf237 bytes) 2024-09-17T07:28:49.212 INFO:teuthology.orchestra.run.smithi045.stdout:202 writesame 0xebb3c73 thru 0xebbe4e2 data_size 0x28(0xa870 bytes) 2024-09-17T07:28:49.213 INFO:teuthology.orchestra.run.smithi045.stdout:203 read 0xbb77240 thru 0xbb7a1f6 (0x2fb7 bytes) 2024-09-17T07:28:49.213 INFO:teuthology.orchestra.run.smithi045.stdout:206 write 0xe178b06 thru 0xe17b33f (0x283a bytes) 2024-09-17T07:28:49.216 INFO:teuthology.orchestra.run.smithi045.stdout:207 write 0x6a28e4a thru 0x6a2ca86 (0x3c3d bytes) 2024-09-17T07:28:49.222 INFO:teuthology.orchestra.run.smithi045.stdout:208 write 0x9db3705 thru 0x9db61d9 (0x2ad5 bytes) 2024-09-17T07:28:49.227 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:49.227 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:49.227 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:28:49.227 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:28:49.227 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:49.228 INFO:teuthology.orchestra.run.smithi045.stdout:209 write 0x1644800 thru 0x16544d3 (0xfcd4 bytes) 2024-09-17T07:28:49.235 INFO:teuthology.orchestra.run.smithi045.stdout:211 read 0xdea55c0 thru 0xdea5648 (0x89 bytes) 2024-09-17T07:28:49.235 INFO:teuthology.orchestra.run.smithi045.stdout:212 read 0x712f3bd thru 0x7137daa (0x89ee bytes) 2024-09-17T07:28:49.236 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:28:49.238 INFO:teuthology.orchestra.run.smithi045.stdout:213 trunc from 0xebbe4e3 to 0xbc3da6e 2024-09-17T07:28:49.247 INFO:teuthology.orchestra.run.smithi045.stdout:214 write 0xb9da5ab thru 0xb9e7d85 (0xd7db bytes) 2024-09-17T07:28:49.248 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:49.246+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:49.252 INFO:teuthology.orchestra.run.smithi045.stdout:215 write 0x68f59ab thru 0x6903f66 (0xe5bc bytes) 2024-09-17T07:28:49.268 INFO:teuthology.orchestra.run.smithi045.stdout:216 trunc from 0xbc3da6e to 0x324dcdf 2024-09-17T07:28:49.290 INFO:teuthology.orchestra.run.smithi045.stdout:217 write 0x684dd31 thru 0x6858cf1 (0xafc1 bytes) 2024-09-17T07:28:49.294 INFO:teuthology.orchestra.run.smithi045.stdout:218 read 0x153d0f1 thru 0x1543df7 (0x6d07 bytes) 2024-09-17T07:28:49.301 INFO:teuthology.orchestra.run.smithi045.stdout:219 trunc from 0x6858cf2 to 0x101b9ed 2024-09-17T07:28:49.318 INFO:teuthology.orchestra.run.smithi045.stdout:221 read 0xd137f thru 0xdb114 (0x9d96 bytes) 2024-09-17T07:28:49.329 INFO:teuthology.orchestra.run.smithi045.stdout:222 writesame 0x50650b0 thru 0x506fe43 data_size 0x8a(0xad94 bytes) 2024-09-17T07:28:49.331 INFO:teuthology.orchestra.run.smithi045.stdout:223 trunc from 0x506fe44 to 0x796e308 2024-09-17T07:28:49.336 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:49.345 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:49.345+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:49.349 INFO:teuthology.orchestra.run.smithi045.stdout:225 write 0xeafc9e8 thru 0xeb0086e (0x3e87 bytes) 2024-09-17T07:28:49.354 INFO:teuthology.orchestra.run.smithi045.stdout:226 read 0xd89b252 thru 0xd8a9393 (0xe142 bytes) 2024-09-17T07:28:49.354 INFO:teuthology.orchestra.run.smithi045.stdout:227 read 0xc722cd5 thru 0xc726ca8 (0x3fd4 bytes) 2024-09-17T07:28:49.354 INFO:teuthology.orchestra.run.smithi045.stdout:228 writesame 0x2c6cb6c thru 0x2c7885e data_size 0x1(0xbcf3 bytes) 2024-09-17T07:28:49.361 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:49.361 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:28:49.361 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:28:49.361 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:28:49.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.361 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-17T07:28:49.361 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.564282+0000", 2024-09-17T07:28:49.361 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.794854818000001, 2024-09-17T07:28:49.361 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001156984, 2024-09-17T07:28:49.361 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.362 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.362 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.362 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.362 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.362 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564282+0000", 2024-09-17T07:28:49.362 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.363 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.363 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564282+0000", 2024-09-17T07:28:49.363 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.363 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.363 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564283+0000", 2024-09-17T07:28:49.363 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4099999999999998e-07 2024-09-17T07:28:49.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.363 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.363 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564480+0000", 2024-09-17T07:28:49.364 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000196524 2024-09-17T07:28:49.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.364 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.364 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564496+0000", 2024-09-17T07:28:49.364 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6160000000000001e-05 2024-09-17T07:28:49.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.364 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.364 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564501+0000", 2024-09-17T07:28:49.365 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4960000000000004e-06 2024-09-17T07:28:49.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.365 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.365 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564545+0000", 2024-09-17T07:28:49.365 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3389000000000001e-05 2024-09-17T07:28:49.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.365 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.365 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564558+0000", 2024-09-17T07:28:49.366 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3507e-05 2024-09-17T07:28:49.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.366 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.366 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565395+0000", 2024-09-17T07:28:49.366 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083644000000000004 2024-09-17T07:28:49.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.366 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.366 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565439+0000", 2024-09-17T07:28:49.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4727000000000001e-05 2024-09-17T07:28:49.367 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.367 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.367 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.367 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-17T07:28:49.367 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.564808+0000", 2024-09-17T07:28:49.367 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.794329542, 2024-09-17T07:28:49.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011919439999999999, 2024-09-17T07:28:49.367 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.368 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.368 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.368 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.368 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564808+0000", 2024-09-17T07:28:49.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.368 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.368 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564808+0000", 2024-09-17T07:28:49.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.369 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564808+0000", 2024-09-17T07:28:49.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6400000000000002e-07 2024-09-17T07:28:49.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.369 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564941+0000", 2024-09-17T07:28:49.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000132841 2024-09-17T07:28:49.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564950+0000", 2024-09-17T07:28:49.370 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0469999999999999e-06 2024-09-17T07:28:49.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.564955+0000", 2024-09-17T07:28:49.370 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9459999999999997e-06 2024-09-17T07:28:49.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.371 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.371 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565002+0000", 2024-09-17T07:28:49.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6835000000000001e-05 2024-09-17T07:28:49.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.371 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.371 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565016+0000", 2024-09-17T07:28:49.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3619e-05 2024-09-17T07:28:49.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.371 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.371 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565928+0000", 2024-09-17T07:28:49.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00091201800000000001 2024-09-17T07:28:49.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.372 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.372 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566000+0000", 2024-09-17T07:28:49.372 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1973999999999999e-05 2024-09-17T07:28:49.372 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.372 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.372 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.372 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:22 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'12, mlcod=17'8)", 2024-09-17T07:28:49.373 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.565349+0000", 2024-09-17T07:28:49.373 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.793788093, 2024-09-17T07:28:49.373 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001151947, 2024-09-17T07:28:49.373 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.373 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.373 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.373 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565349+0000", 2024-09-17T07:28:49.373 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565349+0000", 2024-09-17T07:28:49.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565350+0000", 2024-09-17T07:28:49.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1099999999999995e-07 2024-09-17T07:28:49.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565465+0000", 2024-09-17T07:28:49.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011475 2024-09-17T07:28:49.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565474+0000", 2024-09-17T07:28:49.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9840000000000007e-06 2024-09-17T07:28:49.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565479+0000", 2024-09-17T07:28:49.376 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5099999999999998e-06 2024-09-17T07:28:49.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.376 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.376 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565534+0000", 2024-09-17T07:28:49.376 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4403000000000001e-05 2024-09-17T07:28:49.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.376 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.376 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565566+0000", 2024-09-17T07:28:49.376 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2262999999999999e-05 2024-09-17T07:28:49.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.377 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.377 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566440+0000", 2024-09-17T07:28:49.377 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087397600000000005 2024-09-17T07:28:49.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.377 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.377 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566501+0000", 2024-09-17T07:28:49.377 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.135e-05 2024-09-17T07:28:49.377 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.377 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.378 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.378 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:23 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'13, mlcod=17'8)", 2024-09-17T07:28:49.378 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.565838+0000", 2024-09-17T07:28:49.378 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.793299051999998, 2024-09-17T07:28:49.378 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011229479999999999, 2024-09-17T07:28:49.378 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.378 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.378 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.378 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565838+0000", 2024-09-17T07:28:49.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565838+0000", 2024-09-17T07:28:49.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.380 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565839+0000", 2024-09-17T07:28:49.380 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9500000000000002e-07 2024-09-17T07:28:49.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.380 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.380 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565927+0000", 2024-09-17T07:28:49.381 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8246999999999997e-05 2024-09-17T07:28:49.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.381 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.381 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565948+0000", 2024-09-17T07:28:49.381 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1019e-05 2024-09-17T07:28:49.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.381 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.381 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.565954+0000", 2024-09-17T07:28:49.382 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6389999999999997e-06 2024-09-17T07:28:49.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.382 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.382 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566038+0000", 2024-09-17T07:28:49.382 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3969999999999997e-05 2024-09-17T07:28:49.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.382 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.382 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566056+0000", 2024-09-17T07:28:49.382 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799e-05 2024-09-17T07:28:49.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.383 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.383 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566926+0000", 2024-09-17T07:28:49.383 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087067700000000004 2024-09-17T07:28:49.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.383 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.383 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.566961+0000", 2024-09-17T07:28:49.383 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4901999999999999e-05 2024-09-17T07:28:49.383 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.383 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.383 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.384 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.384 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4212.0:72 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'50, mlcod=17'46)", 2024-09-17T07:28:49.384 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:26.956104+0000", 2024-09-17T07:28:49.384 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.403032976999999, 2024-09-17T07:28:49.384 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010639600000000001, 2024-09-17T07:28:49.384 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.384 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.384 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.385 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.385 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956104+0000", 2024-09-17T07:28:49.385 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.385 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.385 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.385 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956104+0000", 2024-09-17T07:28:49.385 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.385 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.385 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.385 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956105+0000", 2024-09-17T07:28:49.386 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2300000000000001e-07 2024-09-17T07:28:49.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.386 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.386 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956161+0000", 2024-09-17T07:28:49.386 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6539999999999997e-05 2024-09-17T07:28:49.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.386 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.386 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956172+0000", 2024-09-17T07:28:49.386 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0328e-05 2024-09-17T07:28:49.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.387 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.387 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956178+0000", 2024-09-17T07:28:49.387 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4829999999999996e-06 2024-09-17T07:28:49.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.387 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.387 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956247+0000", 2024-09-17T07:28:49.387 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8365e-05 2024-09-17T07:28:49.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.388 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.388 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.956261+0000", 2024-09-17T07:28:49.388 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4279e-05 2024-09-17T07:28:49.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.388 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.388 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.957135+0000", 2024-09-17T07:28:49.388 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087454099999999995 2024-09-17T07:28:49.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.389 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.389 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:26.957168+0000", 2024-09-17T07:28:49.389 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2801e-05 2024-09-17T07:28:49.389 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.389 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.389 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.389 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.389 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T07:28:49.389 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:45.084937+0000", 2024-09-17T07:28:49.389 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.274200574, 2024-09-17T07:28:49.389 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0046658450000000001, 2024-09-17T07:28:49.389 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.390 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.390 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.390 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 11 2024-09-17T07:28:49.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.390 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.391 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.391 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.084937+0000", 2024-09-17T07:28:49.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.391 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.391 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.084937+0000", 2024-09-17T07:28:49.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.391 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.391 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.084939+0000", 2024-09-17T07:28:49.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1600000000000001e-06 2024-09-17T07:28:49.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.084950+0000", 2024-09-17T07:28:49.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1194e-05 2024-09-17T07:28:49.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.084967+0000", 2024-09-17T07:28:49.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7346999999999998e-05 2024-09-17T07:28:49.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.393 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.084978+0000", 2024-09-17T07:28:49.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0562e-05 2024-09-17T07:28:49.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.393 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.085049+0000", 2024-09-17T07:28:49.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0859999999999996e-05 2024-09-17T07:28:49.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.394 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:28:49.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.085055+0000", 2024-09-17T07:28:49.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.806e-06 2024-09-17T07:28:49.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.394 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.087859+0000", 2024-09-17T07:28:49.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028046429999999999 2024-09-17T07:28:49.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.395 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.087967+0000", 2024-09-17T07:28:49.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000107797 2024-09-17T07:28:49.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:28:49.395 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.088152+0000", 2024-09-17T07:28:49.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00018521799999999999 2024-09-17T07:28:49.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089243+0000", 2024-09-17T07:28:49.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010904910000000001 2024-09-17T07:28:49.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.396 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089532+0000", 2024-09-17T07:28:49.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00028896999999999999 2024-09-17T07:28:49.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.396 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.397 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089572+0000", 2024-09-17T07:28:49.397 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0618000000000003e-05 2024-09-17T07:28:49.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.397 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.397 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089603+0000", 2024-09-17T07:28:49.397 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0179e-05 2024-09-17T07:28:49.397 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.397 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.397 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.398 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:12 3.1b 3:d9441778:::rbd_header.1183951e9e6e:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T07:28:49.398 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:45.089737+0000", 2024-09-17T07:28:49.398 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.2694003919999997, 2024-09-17T07:28:49.398 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013646730000000001, 2024-09-17T07:28:49.398 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.398 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.398 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.399 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.399 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 12 2024-09-17T07:28:49.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.399 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089737+0000", 2024-09-17T07:28:49.399 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089737+0000", 2024-09-17T07:28:49.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089738+0000", 2024-09-17T07:28:49.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.24e-06 2024-09-17T07:28:49.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089745+0000", 2024-09-17T07:28:49.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.601e-06 2024-09-17T07:28:49.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089758+0000", 2024-09-17T07:28:49.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3291e-05 2024-09-17T07:28:49.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.402 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.402 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089764+0000", 2024-09-17T07:28:49.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4010000000000002e-06 2024-09-17T07:28:49.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089825+0000", 2024-09-17T07:28:49.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0729000000000001e-05 2024-09-17T07:28:49.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.089967+0000", 2024-09-17T07:28:49.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000141977 2024-09-17T07:28:49.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:49.404 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.090211+0000", 2024-09-17T07:28:49.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00024368400000000001 2024-09-17T07:28:49.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.404 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.090928+0000", 2024-09-17T07:28:49.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00071752999999999997 2024-09-17T07:28:49.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.091038+0000", 2024-09-17T07:28:49.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010932100000000001 2024-09-17T07:28:49.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.091072+0000", 2024-09-17T07:28:49.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4857000000000002e-05 2024-09-17T07:28:49.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.406 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.091102+0000", 2024-09-17T07:28:49.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9042000000000001e-05 2024-09-17T07:28:49.406 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.406 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.406 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.406 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:25 3.1b 3:d9441778:::rbd_header.1183951e9e6e:head [watch watch cookie 140227662426288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T07:28:49.406 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:45.101961+0000", 2024-09-17T07:28:49.406 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.2571763069999999, 2024-09-17T07:28:49.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012175949999999999, 2024-09-17T07:28:49.406 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.407 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 25 2024-09-17T07:28:49.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.407 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.407 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.407 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.101961+0000", 2024-09-17T07:28:49.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.408 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.408 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.101961+0000", 2024-09-17T07:28:49.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.408 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.408 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.101962+0000", 2024-09-17T07:28:49.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2130000000000001e-06 2024-09-17T07:28:49.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.409 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.101968+0000", 2024-09-17T07:28:49.409 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.609e-06 2024-09-17T07:28:49.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.101984+0000", 2024-09-17T07:28:49.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.628e-05 2024-09-17T07:28:49.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.410 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.101993+0000", 2024-09-17T07:28:49.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6619999999999994e-06 2024-09-17T07:28:49.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.410 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.102043+0000", 2024-09-17T07:28:49.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9920999999999999e-05 2024-09-17T07:28:49.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.411 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.102095+0000", 2024-09-17T07:28:49.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2583000000000001e-05 2024-09-17T07:28:49.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.411 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:49.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.102212+0000", 2024-09-17T07:28:49.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000117125 2024-09-17T07:28:49.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.102914+0000", 2024-09-17T07:28:49.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00070177800000000002 2024-09-17T07:28:49.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.103032+0000", 2024-09-17T07:28:49.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000117714 2024-09-17T07:28:49.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.413 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.103090+0000", 2024-09-17T07:28:49.413 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8424999999999997e-05 2024-09-17T07:28:49.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.413 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.413 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:45.103179+0000", 2024-09-17T07:28:49.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8284999999999999e-05 2024-09-17T07:28:49.414 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.414 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.414 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.414 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:670 3.1f 3:fd14b5b2:::rbd_data.1183350c2d82.000000000000000a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.414 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.813123+0000", 2024-09-17T07:28:49.414 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.54601381100000002, 2024-09-17T07:28:49.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001614316, 2024-09-17T07:28:49.414 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.415 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.415 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.415 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.415 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 670 2024-09-17T07:28:49.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.415 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813123+0000", 2024-09-17T07:28:49.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.416 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813123+0000", 2024-09-17T07:28:49.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.416 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813125+0000", 2024-09-17T07:28:49.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8840000000000001e-06 2024-09-17T07:28:49.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813133+0000", 2024-09-17T07:28:49.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6009999999999997e-06 2024-09-17T07:28:49.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813154+0000", 2024-09-17T07:28:49.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.145e-05 2024-09-17T07:28:49.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813164+0000", 2024-09-17T07:28:49.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0006e-05 2024-09-17T07:28:49.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813242+0000", 2024-09-17T07:28:49.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8041999999999996e-05 2024-09-17T07:28:49.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813332+0000", 2024-09-17T07:28:49.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9755999999999999e-05 2024-09-17T07:28:49.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:28:49.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813445+0000", 2024-09-17T07:28:49.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011285600000000001 2024-09-17T07:28:49.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.814174+0000", 2024-09-17T07:28:49.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072865199999999999 2024-09-17T07:28:49.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.814605+0000", 2024-09-17T07:28:49.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00043143199999999999 2024-09-17T07:28:49.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.421 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.814677+0000", 2024-09-17T07:28:49.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2078999999999999e-05 2024-09-17T07:28:49.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.421 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.814738+0000", 2024-09-17T07:28:49.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0557999999999999e-05 2024-09-17T07:28:49.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.421 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.422 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:672 3.1b 3:d8f4eb85:::rbd_data.1183350c2d82.000000000000001c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.422 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.813191+0000", 2024-09-17T07:28:49.422 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.54594649699999998, 2024-09-17T07:28:49.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001663304, 2024-09-17T07:28:49.422 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.422 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.422 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.422 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.422 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 672 2024-09-17T07:28:49.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.423 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813191+0000", 2024-09-17T07:28:49.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813191+0000", 2024-09-17T07:28:49.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813191+0000", 2024-09-17T07:28:49.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2499999999999995e-07 2024-09-17T07:28:49.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813197+0000", 2024-09-17T07:28:49.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5289999999999999e-06 2024-09-17T07:28:49.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813215+0000", 2024-09-17T07:28:49.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7955999999999999e-05 2024-09-17T07:28:49.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813221+0000", 2024-09-17T07:28:49.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:28:49.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813279+0000", 2024-09-17T07:28:49.426 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8239000000000003e-05 2024-09-17T07:28:49.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.426 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.426 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813372+0000", 2024-09-17T07:28:49.426 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2564000000000003e-05 2024-09-17T07:28:49.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.426 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:49.427 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.813479+0000", 2024-09-17T07:28:49.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010686799999999999 2024-09-17T07:28:49.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.427 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.427 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.814563+0000", 2024-09-17T07:28:49.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010841589999999999 2024-09-17T07:28:49.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.427 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.427 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.814752+0000", 2024-09-17T07:28:49.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00018914999999999999 2024-09-17T07:28:49.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.814810+0000", 2024-09-17T07:28:49.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8381999999999998e-05 2024-09-17T07:28:49.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.814854+0000", 2024-09-17T07:28:49.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3707e-05 2024-09-17T07:28:49.429 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.429 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.429 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.429 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:678 3.1f 3:fdc18f01:::rbd_data.1183350c2d82.0000000000000000:head [stat,truncate 458752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.429 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.817406+0000", 2024-09-17T07:28:49.429 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.54173143800000001, 2024-09-17T07:28:49.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012105499999999999, 2024-09-17T07:28:49.429 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.430 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.430 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.430 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.430 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.430 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 678 2024-09-17T07:28:49.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.430 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817406+0000", 2024-09-17T07:28:49.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.431 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.431 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817406+0000", 2024-09-17T07:28:49.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.431 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.431 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817407+0000", 2024-09-17T07:28:49.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3200000000000001e-06 2024-09-17T07:28:49.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817413+0000", 2024-09-17T07:28:49.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4319999999999998e-06 2024-09-17T07:28:49.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817428+0000", 2024-09-17T07:28:49.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5004999999999999e-05 2024-09-17T07:28:49.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817436+0000", 2024-09-17T07:28:49.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9419999999999995e-06 2024-09-17T07:28:49.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817497+0000", 2024-09-17T07:28:49.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1601999999999997e-05 2024-09-17T07:28:49.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817595+0000", 2024-09-17T07:28:49.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7870999999999999e-05 2024-09-17T07:28:49.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:28:49.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817748+0000", 2024-09-17T07:28:49.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015332200000000001 2024-09-17T07:28:49.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.818307+0000", 2024-09-17T07:28:49.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000558612 2024-09-17T07:28:49.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.818516+0000", 2024-09-17T07:28:49.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000208834 2024-09-17T07:28:49.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.818575+0000", 2024-09-17T07:28:49.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8915000000000001e-05 2024-09-17T07:28:49.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.818616+0000", 2024-09-17T07:28:49.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1695e-05 2024-09-17T07:28:49.437 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.437 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.437 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.437 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:681 3.12 3:48316f5c:::rbd_data.1183350c2d82.0000000000000005:head [stat,truncate 425984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.437 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.817666+0000", 2024-09-17T07:28:49.437 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.54147131299999995, 2024-09-17T07:28:49.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0051808669999999996, 2024-09-17T07:28:49.437 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.438 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.438 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.438 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.438 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.438 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 681 2024-09-17T07:28:49.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.438 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817666+0000", 2024-09-17T07:28:49.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817666+0000", 2024-09-17T07:28:49.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817667+0000", 2024-09-17T07:28:49.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7299999999999995e-07 2024-09-17T07:28:49.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817672+0000", 2024-09-17T07:28:49.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5350000000000004e-06 2024-09-17T07:28:49.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817684+0000", 2024-09-17T07:28:49.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1627999999999999e-05 2024-09-17T07:28:49.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817694+0000", 2024-09-17T07:28:49.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0523000000000001e-05 2024-09-17T07:28:49.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817757+0000", 2024-09-17T07:28:49.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2422999999999995e-05 2024-09-17T07:28:49.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817823+0000", 2024-09-17T07:28:49.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6365999999999994e-05 2024-09-17T07:28:49.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:28:49.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.817899+0000", 2024-09-17T07:28:49.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5536000000000003e-05 2024-09-17T07:28:49.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.818564+0000", 2024-09-17T07:28:49.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00066542100000000002 2024-09-17T07:28:49.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.822775+0000", 2024-09-17T07:28:49.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0042112460000000001 2024-09-17T07:28:49.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.822805+0000", 2024-09-17T07:28:49.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0184000000000001e-05 2024-09-17T07:28:49.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.822847+0000", 2024-09-17T07:28:49.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1331999999999997e-05 2024-09-17T07:28:49.444 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.444 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.444 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.444 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:789 3.1b 3:d98f5c1c:::rbd_data.1183350c2d82.0000000000000055:head [set-alloc-hint object_size 1048576 write_size 1048576,write 87149~11155] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.444 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.956932+0000", 2024-09-17T07:28:49.444 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.40220524299999999, 2024-09-17T07:28:49.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0047296539999999998, 2024-09-17T07:28:49.444 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.444 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.445 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.445 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.445 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.445 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.445 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 789 2024-09-17T07:28:49.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.445 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.956932+0000", 2024-09-17T07:28:49.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.956932+0000", 2024-09-17T07:28:49.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.956934+0000", 2024-09-17T07:28:49.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8390000000000001e-06 2024-09-17T07:28:49.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.956949+0000", 2024-09-17T07:28:49.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4975e-05 2024-09-17T07:28:49.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.956967+0000", 2024-09-17T07:28:49.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7697e-05 2024-09-17T07:28:49.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.956974+0000", 2024-09-17T07:28:49.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2949999999999996e-06 2024-09-17T07:28:49.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.957031+0000", 2024-09-17T07:28:49.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6848000000000002e-05 2024-09-17T07:28:49.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.957141+0000", 2024-09-17T07:28:49.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011058899999999999 2024-09-17T07:28:49.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:49.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.957236+0000", 2024-09-17T07:28:49.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5229000000000006e-05 2024-09-17T07:28:49.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.957931+0000", 2024-09-17T07:28:49.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00069421299999999995 2024-09-17T07:28:49.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.961594+0000", 2024-09-17T07:28:49.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036637480000000001 2024-09-17T07:28:49.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.961629+0000", 2024-09-17T07:28:49.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4987999999999998e-05 2024-09-17T07:28:49.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.961662+0000", 2024-09-17T07:28:49.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2233000000000001e-05 2024-09-17T07:28:49.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.450 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.451 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.451 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:804 3.1b 3:d98f5c1c:::rbd_data.1183350c2d82.0000000000000055:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.451 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.976226+0000", 2024-09-17T07:28:49.451 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.38291162099999998, 2024-09-17T07:28:49.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001187235, 2024-09-17T07:28:49.451 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.451 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.451 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.451 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.451 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.451 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 804 2024-09-17T07:28:49.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.452 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.976226+0000", 2024-09-17T07:28:49.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.976226+0000", 2024-09-17T07:28:49.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.976228+0000", 2024-09-17T07:28:49.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1210000000000001e-06 2024-09-17T07:28:49.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.976237+0000", 2024-09-17T07:28:49.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4180000000000003e-06 2024-09-17T07:28:49.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.976258+0000", 2024-09-17T07:28:49.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0407e-05 2024-09-17T07:28:49.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.976267+0000", 2024-09-17T07:28:49.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8430000000000005e-06 2024-09-17T07:28:49.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.976334+0000", 2024-09-17T07:28:49.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6265000000000003e-05 2024-09-17T07:28:49.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.976397+0000", 2024-09-17T07:28:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3726999999999995e-05 2024-09-17T07:28:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.976474+0000", 2024-09-17T07:28:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6440999999999995e-05 2024-09-17T07:28:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.977034+0000", 2024-09-17T07:28:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000560258 2024-09-17T07:28:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.977289+0000", 2024-09-17T07:28:49.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00025525 2024-09-17T07:28:49.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.977347+0000", 2024-09-17T07:28:49.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7194000000000002e-05 2024-09-17T07:28:49.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.977413+0000", 2024-09-17T07:28:49.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6310999999999996e-05 2024-09-17T07:28:49.456 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.456 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.457 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.457 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:818 3.12 3:48316f5c:::rbd_data.1183350c2d82.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.457 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.985133+0000", 2024-09-17T07:28:49.457 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.37400416800000003, 2024-09-17T07:28:49.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.012417920000000001, 2024-09-17T07:28:49.457 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.457 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.458 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.458 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.458 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.458 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 818 2024-09-17T07:28:49.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.458 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.458 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.458 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985133+0000", 2024-09-17T07:28:49.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.458 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985133+0000", 2024-09-17T07:28:49.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985134+0000", 2024-09-17T07:28:49.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3209999999999999e-06 2024-09-17T07:28:49.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985141+0000", 2024-09-17T07:28:49.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.443e-06 2024-09-17T07:28:49.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985154+0000", 2024-09-17T07:28:49.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3302000000000001e-05 2024-09-17T07:28:49.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985162+0000", 2024-09-17T07:28:49.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.644e-06 2024-09-17T07:28:49.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985226+0000", 2024-09-17T07:28:49.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4620000000000001e-05 2024-09-17T07:28:49.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985333+0000", 2024-09-17T07:28:49.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010641099999999999 2024-09-17T07:28:49.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:28:49.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.987842+0000", 2024-09-17T07:28:49.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025096720000000001 2024-09-17T07:28:49.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.997364+0000", 2024-09-17T07:28:49.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.009521346 2024-09-17T07:28:49.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.997424+0000", 2024-09-17T07:28:49.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0282000000000003e-05 2024-09-17T07:28:49.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.997495+0000", 2024-09-17T07:28:49.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1334000000000005e-05 2024-09-17T07:28:49.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.997551+0000", 2024-09-17T07:28:49.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5544999999999998e-05 2024-09-17T07:28:49.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.464 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.464 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop_reply(client.4483.0:818 3.12 e28/23 ondisk, result = 0)", 2024-09-17T07:28:49.464 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.993741+0000", 2024-09-17T07:28:49.465 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.36539657599999997, 2024-09-17T07:28:49.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003822663, 2024-09-17T07:28:49.465 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.465 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.465 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:49.465 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.993741+0000", 2024-09-17T07:28:49.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.468 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.468 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.993741+0000", 2024-09-17T07:28:49.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.993742+0000", 2024-09-17T07:28:49.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9699999999999994e-07 2024-09-17T07:28:49.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.993745+0000", 2024-09-17T07:28:49.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7900000000000001e-06 2024-09-17T07:28:49.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.993759+0000", 2024-09-17T07:28:49.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3481e-05 2024-09-17T07:28:49.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.993769+0000", 2024-09-17T07:28:49.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0375e-05 2024-09-17T07:28:49.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.997405+0000", 2024-09-17T07:28:49.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036352279999999999 2024-09-17T07:28:49.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.997415+0000", 2024-09-17T07:28:49.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0404000000000001e-05 2024-09-17T07:28:49.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.997563+0000", 2024-09-17T07:28:49.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000148388 2024-09-17T07:28:49.472 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.473 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.473 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.473 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:842 3.1 3:871a2656:::rbd_data.1183350c2d82.00000000000000c5:head [set-alloc-hint object_size 1048576 write_size 1048576,write 262144~16710] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.473 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.024114+0000", 2024-09-17T07:28:49.473 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.33502346799999999, 2024-09-17T07:28:49.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013557350000000001, 2024-09-17T07:28:49.473 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.473 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.473 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.473 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.474 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.474 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.474 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 842 2024-09-17T07:28:49.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.474 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.474 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.474 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.024114+0000", 2024-09-17T07:28:49.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.474 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.474 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.024114+0000", 2024-09-17T07:28:49.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.024116+0000", 2024-09-17T07:28:49.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7349999999999999e-06 2024-09-17T07:28:49.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.024128+0000", 2024-09-17T07:28:49.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2938999999999999e-05 2024-09-17T07:28:49.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.024141+0000", 2024-09-17T07:28:49.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.253e-05 2024-09-17T07:28:49.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.024149+0000", 2024-09-17T07:28:49.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.106e-06 2024-09-17T07:28:49.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.024218+0000", 2024-09-17T07:28:49.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8591999999999997e-05 2024-09-17T07:28:49.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.024350+0000", 2024-09-17T07:28:49.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013189699999999999 2024-09-17T07:28:49.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:49.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.024496+0000", 2024-09-17T07:28:49.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014651599999999999 2024-09-17T07:28:49.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.025286+0000", 2024-09-17T07:28:49.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00079015800000000001 2024-09-17T07:28:49.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.025386+0000", 2024-09-17T07:28:49.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9382000000000005e-05 2024-09-17T07:28:49.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.025429+0000", 2024-09-17T07:28:49.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3336e-05 2024-09-17T07:28:49.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.025470+0000", 2024-09-17T07:28:49.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0543999999999997e-05 2024-09-17T07:28:49.479 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.479 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.479 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.479 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:846 3.17 3:edbcd0af:::rbd_data.1183350c2d82.00000000000000a7:head [set-alloc-hint object_size 1048576 write_size 1048576,write 904260~9552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.480 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.033912+0000", 2024-09-17T07:28:49.480 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.32522513800000002, 2024-09-17T07:28:49.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013532069999999999, 2024-09-17T07:28:49.480 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.480 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.480 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.480 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.480 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.480 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 846 2024-09-17T07:28:49.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.480 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.033912+0000", 2024-09-17T07:28:49.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.033912+0000", 2024-09-17T07:28:49.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.033913+0000", 2024-09-17T07:28:49.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.384e-06 2024-09-17T07:28:49.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.033925+0000", 2024-09-17T07:28:49.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1780000000000001e-05 2024-09-17T07:28:49.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.033939+0000", 2024-09-17T07:28:49.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4205e-05 2024-09-17T07:28:49.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.033946+0000", 2024-09-17T07:28:49.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0839999999999997e-06 2024-09-17T07:28:49.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.034014+0000", 2024-09-17T07:28:49.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8368999999999995e-05 2024-09-17T07:28:49.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.034109+0000", 2024-09-17T07:28:49.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5068000000000005e-05 2024-09-17T07:28:49.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:28:49.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.034273+0000", 2024-09-17T07:28:49.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00016421900000000001 2024-09-17T07:28:49.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.035097+0000", 2024-09-17T07:28:49.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00082394 2024-09-17T07:28:49.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.035199+0000", 2024-09-17T07:28:49.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000101877 2024-09-17T07:28:49.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.035233+0000", 2024-09-17T07:28:49.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4272e-05 2024-09-17T07:28:49.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.035265+0000", 2024-09-17T07:28:49.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2008999999999998e-05 2024-09-17T07:28:49.485 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.486 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.486 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.486 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:877 3.17 3:edbcd0af:::rbd_data.1183350c2d82.00000000000000a7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.486 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.069283+0000", 2024-09-17T07:28:49.486 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.28985377400000001, 2024-09-17T07:28:49.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0065074670000000003, 2024-09-17T07:28:49.486 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.486 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.486 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.486 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.486 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.486 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.487 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 877 2024-09-17T07:28:49.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.487 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069283+0000", 2024-09-17T07:28:49.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069283+0000", 2024-09-17T07:28:49.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.488 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.488 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069286+0000", 2024-09-17T07:28:49.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:28:49.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.488 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.488 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069293+0000", 2024-09-17T07:28:49.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7030000000000006e-06 2024-09-17T07:28:49.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.488 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.488 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069314+0000", 2024-09-17T07:28:49.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.033e-05 2024-09-17T07:28:49.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069328+0000", 2024-09-17T07:28:49.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4044e-05 2024-09-17T07:28:49.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069400+0000", 2024-09-17T07:28:49.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2360999999999994e-05 2024-09-17T07:28:49.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069496+0000", 2024-09-17T07:28:49.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6267999999999994e-05 2024-09-17T07:28:49.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:28:49.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069619+0000", 2024-09-17T07:28:49.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012247199999999999 2024-09-17T07:28:49.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070568+0000", 2024-09-17T07:28:49.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00094935899999999999 2024-09-17T07:28:49.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075682+0000", 2024-09-17T07:28:49.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.005114164 2024-09-17T07:28:49.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.491 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.491 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075743+0000", 2024-09-17T07:28:49.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0421000000000003e-05 2024-09-17T07:28:49.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.491 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.491 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075791+0000", 2024-09-17T07:28:49.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8087999999999998e-05 2024-09-17T07:28:49.491 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.491 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.491 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.492 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:882 3.1 3:871a2656:::rbd_data.1183350c2d82.00000000000000c5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.492 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.073104+0000", 2024-09-17T07:28:49.492 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.28603366600000002, 2024-09-17T07:28:49.492 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00561532, 2024-09-17T07:28:49.492 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.492 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.492 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.492 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.492 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.493 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.493 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 882 2024-09-17T07:28:49.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.493 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073104+0000", 2024-09-17T07:28:49.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073104+0000", 2024-09-17T07:28:49.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.494 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073105+0000", 2024-09-17T07:28:49.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.263e-06 2024-09-17T07:28:49.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.494 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073110+0000", 2024-09-17T07:28:49.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-06 2024-09-17T07:28:49.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.495 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.495 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073125+0000", 2024-09-17T07:28:49.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5509999999999999e-05 2024-09-17T07:28:49.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.495 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.495 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073133+0000", 2024-09-17T07:28:49.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0600000000000008e-06 2024-09-17T07:28:49.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.496 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.496 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073204+0000", 2024-09-17T07:28:49.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0223999999999997e-05 2024-09-17T07:28:49.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.496 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.496 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073297+0000", 2024-09-17T07:28:49.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3134999999999995e-05 2024-09-17T07:28:49.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.496 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:49.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073416+0000", 2024-09-17T07:28:49.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011952599999999999 2024-09-17T07:28:49.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073903+0000", 2024-09-17T07:28:49.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000486417 2024-09-17T07:28:49.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.078659+0000", 2024-09-17T07:28:49.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0047562400000000001 2024-09-17T07:28:49.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.078689+0000", 2024-09-17T07:28:49.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0266000000000001e-05 2024-09-17T07:28:49.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.078719+0000", 2024-09-17T07:28:49.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9719000000000001e-05 2024-09-17T07:28:49.499 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.499 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.499 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.499 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.499 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.499 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:49.501 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:49.500+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:49.502 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:28:49.504 INFO:teuthology.orchestra.run.smithi045.stdout:231 trunc from 0xeb0086f to 0x2fd1d85 2024-09-17T07:28:49.504 INFO:teuthology.orchestra.run.smithi045.stdout:232 write 0x9df2d73 thru 0x9dfce62 (0xa0f0 bytes) 2024-09-17T07:28:49.504 INFO:teuthology.orchestra.run.smithi045.stdout:233 trunc from 0x9dfce63 to 0xdea5658 2024-09-17T07:28:49.505 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:49.505 INFO:teuthology.orchestra.run.smithi045.stdout:236 write 0x41ed50b thru 0x41f9565 (0xc05b bytes) 2024-09-17T07:28:49.505 INFO:teuthology.orchestra.run.smithi045.stdout:237 read 0x4dc07fd thru 0x4dcec9f (0xe4a3 bytes) 2024-09-17T07:28:49.505 INFO:teuthology.orchestra.run.smithi045.stdout:238 writesame 0x64f2d98 thru 0x64fa5cc data_size 0x1(0x7835 bytes) 2024-09-17T07:28:49.505 INFO:teuthology.orchestra.run.smithi045.stdout:240 write 0xd7c76a2 thru 0xd7cd5cf (0x5f2e bytes) 2024-09-17T07:28:49.505 INFO:teuthology.orchestra.run.smithi045.stdout:241 writesame 0xa9010cc thru 0xa906351 data_size 0x15(0x5286 bytes) 2024-09-17T07:28:49.505 INFO:teuthology.orchestra.run.smithi045.stdout:243 write 0x2025322 thru 0x2032765 (0xd444 bytes) 2024-09-17T07:28:49.505 INFO:teuthology.orchestra.run.smithi045.stdout:245 write 0x1314916 thru 0x131d9d5 (0x90c0 bytes) 2024-09-17T07:28:49.505 INFO:teuthology.orchestra.run.smithi045.stdout:246 writesame 0xde1755b thru 0xde2671b data_size 0x1(0xf1c1 bytes) 2024-09-17T07:28:49.505 INFO:teuthology.orchestra.run.smithi045.stdout:247 read 0xd2bc84d thru 0xd2bd2ab (0xa5f bytes) 2024-09-17T07:28:49.506 INFO:teuthology.orchestra.run.smithi045.stdout:248 write 0xb415b66 thru 0xb417414 (0x18af bytes) 2024-09-17T07:28:49.506 INFO:teuthology.orchestra.run.smithi045.stdout:250 read 0x771dfbb thru 0x7720f1b (0x2f61 bytes) 2024-09-17T07:28:49.506 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:49.506 INFO:teuthology.orchestra.run.smithi045.stdout:252 writesame 0x1cc5557 thru 0x1cd2276 data_size 0x30(0xcd20 bytes) 2024-09-17T07:28:49.506 INFO:teuthology.orchestra.run.smithi045.stdout:255 writesame 0x5f5874c thru 0x5f6821f data_size 0x6(0xfad4 bytes) 2024-09-17T07:28:49.507 INFO:teuthology.orchestra.run.smithi045.stdout:257 clone 2 order 18 su 32768 sc 14 2024-09-17T07:28:49.601 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:49.601+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:49.614 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:49.614 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:49.614 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:28:49.614 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:49.623 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:28:49.702 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:49.702+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:49.730 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:49.730 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:49.730 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:28:49.730 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:28:49.730 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:49.739 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:28:49.803 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:49.802+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:49.848 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:49.848 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:28:49.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:28:49.848 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:28:49.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.849 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:755 3.0 3:07b6f4cb:::rbd_data.1183350c2d82.0000000000000063:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.849 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.924070+0000", 2024-09-17T07:28:49.849 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.92257285499999997, 2024-09-17T07:28:49.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001793287, 2024-09-17T07:28:49.849 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.849 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.849 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.849 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.849 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.849 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 755 2024-09-17T07:28:49.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.850 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924070+0000", 2024-09-17T07:28:49.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924070+0000", 2024-09-17T07:28:49.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924072+0000", 2024-09-17T07:28:49.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2670000000000001e-06 2024-09-17T07:28:49.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924083+0000", 2024-09-17T07:28:49.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1156e-05 2024-09-17T07:28:49.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924103+0000", 2024-09-17T07:28:49.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9649000000000001e-05 2024-09-17T07:28:49.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924113+0000", 2024-09-17T07:28:49.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9709999999999999e-06 2024-09-17T07:28:49.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924192+0000", 2024-09-17T07:28:49.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9176000000000002e-05 2024-09-17T07:28:49.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924291+0000", 2024-09-17T07:28:49.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8888000000000006e-05 2024-09-17T07:28:49.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 2", 2024-09-17T07:28:49.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924418+0000", 2024-09-17T07:28:49.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000126847 2024-09-17T07:28:49.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.925716+0000", 2024-09-17T07:28:49.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012978499999999999 2024-09-17T07:28:49.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.925734+0000", 2024-09-17T07:28:49.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8501000000000001e-05 2024-09-17T07:28:49.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.925812+0000", 2024-09-17T07:28:49.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7911000000000007e-05 2024-09-17T07:28:49.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.925863+0000", 2024-09-17T07:28:49.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1070999999999999e-05 2024-09-17T07:28:49.854 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.854 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.855 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.855 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:757 3.19 3:9fcd161c:::rbd_data.1183350c2d82.0000000000000073:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.855 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.924149+0000", 2024-09-17T07:28:49.855 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.92249375199999994, 2024-09-17T07:28:49.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0018967770000000001, 2024-09-17T07:28:49.855 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.855 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.855 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.855 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.855 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.855 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 757 2024-09-17T07:28:49.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.856 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924149+0000", 2024-09-17T07:28:49.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924149+0000", 2024-09-17T07:28:49.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924150+0000", 2024-09-17T07:28:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8400000000000003e-07 2024-09-17T07:28:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924152+0000", 2024-09-17T07:28:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.711e-06 2024-09-17T07:28:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924160+0000", 2024-09-17T07:28:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0779999999999996e-06 2024-09-17T07:28:49.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924182+0000", 2024-09-17T07:28:49.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1642e-05 2024-09-17T07:28:49.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924263+0000", 2024-09-17T07:28:49.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1046999999999998e-05 2024-09-17T07:28:49.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924338+0000", 2024-09-17T07:28:49.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5012000000000006e-05 2024-09-17T07:28:49.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 3", 2024-09-17T07:28:49.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924442+0000", 2024-09-17T07:28:49.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010368499999999999 2024-09-17T07:28:49.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.925692+0000", 2024-09-17T07:28:49.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001250187 2024-09-17T07:28:49.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.925946+0000", 2024-09-17T07:28:49.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00025422299999999998 2024-09-17T07:28:49.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.926004+0000", 2024-09-17T07:28:49.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8251000000000002e-05 2024-09-17T07:28:49.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.926046+0000", 2024-09-17T07:28:49.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1156999999999999e-05 2024-09-17T07:28:49.861 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.861 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.861 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.861 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:758 3.19 3:9e3affcb:::rbd_data.1183350c2d82.0000000000000074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.861 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.924214+0000", 2024-09-17T07:28:49.861 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.92242840400000004, 2024-09-17T07:28:49.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020922570000000001, 2024-09-17T07:28:49.861 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.861 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.861 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.862 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.862 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 758 2024-09-17T07:28:49.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.863 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924214+0000", 2024-09-17T07:28:49.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924214+0000", 2024-09-17T07:28:49.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924215+0000", 2024-09-17T07:28:49.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4700000000000001e-07 2024-09-17T07:28:49.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924218+0000", 2024-09-17T07:28:49.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0130000000000002e-06 2024-09-17T07:28:49.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924234+0000", 2024-09-17T07:28:49.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6243999999999999e-05 2024-09-17T07:28:49.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924244+0000", 2024-09-17T07:28:49.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0175e-05 2024-09-17T07:28:49.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924803+0000", 2024-09-17T07:28:49.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00055827299999999997 2024-09-17T07:28:49.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924890+0000", 2024-09-17T07:28:49.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7137000000000003e-05 2024-09-17T07:28:49.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 3", 2024-09-17T07:28:49.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924990+0000", 2024-09-17T07:28:49.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010054 2024-09-17T07:28:49.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.926172+0000", 2024-09-17T07:28:49.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011818499999999999 2024-09-17T07:28:49.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.926247+0000", 2024-09-17T07:28:49.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4833000000000007e-05 2024-09-17T07:28:49.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.926285+0000", 2024-09-17T07:28:49.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8575999999999998e-05 2024-09-17T07:28:49.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.926306+0000", 2024-09-17T07:28:49.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0968999999999999e-05 2024-09-17T07:28:49.868 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.869 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.869 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:759 3.4 3:20e84768:::rbd_data.1183350c2d82.0000000000000075:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.869 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.924282+0000", 2024-09-17T07:28:49.869 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.92236082500000005, 2024-09-17T07:28:49.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002113364, 2024-09-17T07:28:49.869 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.869 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.869 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.869 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.870 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.870 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.870 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 759 2024-09-17T07:28:49.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.870 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924282+0000", 2024-09-17T07:28:49.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924282+0000", 2024-09-17T07:28:49.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924283+0000", 2024-09-17T07:28:49.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6599999999999995e-07 2024-09-17T07:28:49.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924285+0000", 2024-09-17T07:28:49.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8930000000000001e-06 2024-09-17T07:28:49.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924293+0000", 2024-09-17T07:28:49.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2910000000000002e-06 2024-09-17T07:28:49.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924299+0000", 2024-09-17T07:28:49.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6760000000000001e-06 2024-09-17T07:28:49.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924355+0000", 2024-09-17T07:28:49.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5859999999999997e-05 2024-09-17T07:28:49.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924435+0000", 2024-09-17T07:28:49.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9867e-05 2024-09-17T07:28:49.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 2", 2024-09-17T07:28:49.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924547+0000", 2024-09-17T07:28:49.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011165100000000001 2024-09-17T07:28:49.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.926234+0000", 2024-09-17T07:28:49.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001687486 2024-09-17T07:28:49.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.926279+0000", 2024-09-17T07:28:49.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4937e-05 2024-09-17T07:28:49.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.926348+0000", 2024-09-17T07:28:49.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8434000000000002e-05 2024-09-17T07:28:49.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.926395+0000", 2024-09-17T07:28:49.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7503000000000002e-05 2024-09-17T07:28:49.876 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.876 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.876 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.876 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:817 3.19 3:9b377470:::rbd_data.1183350c2d82.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.877 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.985115+0000", 2024-09-17T07:28:49.877 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.86152728599999995, 2024-09-17T07:28:49.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0018532819999999999, 2024-09-17T07:28:49.877 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.877 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.877 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.877 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.877 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.877 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 817 2024-09-17T07:28:49.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.878 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985115+0000", 2024-09-17T07:28:49.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985115+0000", 2024-09-17T07:28:49.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985118+0000", 2024-09-17T07:28:49.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5679999999999998e-06 2024-09-17T07:28:49.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985126+0000", 2024-09-17T07:28:49.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1999999999999994e-06 2024-09-17T07:28:49.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985144+0000", 2024-09-17T07:28:49.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8079000000000001e-05 2024-09-17T07:28:49.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985154+0000", 2024-09-17T07:28:49.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0322e-05 2024-09-17T07:28:49.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985230+0000", 2024-09-17T07:28:49.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5167999999999996e-05 2024-09-17T07:28:49.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985332+0000", 2024-09-17T07:28:49.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000102679 2024-09-17T07:28:49.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 3", 2024-09-17T07:28:49.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985637+0000", 2024-09-17T07:28:49.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00030468200000000002 2024-09-17T07:28:49.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.986599+0000", 2024-09-17T07:28:49.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00096154499999999996 2024-09-17T07:28:49.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.986851+0000", 2024-09-17T07:28:49.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00025217099999999998 2024-09-17T07:28:49.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.986912+0000", 2024-09-17T07:28:49.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1507000000000006e-05 2024-09-17T07:28:49.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.986969+0000", 2024-09-17T07:28:49.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6360999999999998e-05 2024-09-17T07:28:49.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.885 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.885 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:872 3.4 3:220acdab:::rbd_data.1183350c2d82.0000000000000070:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:49.885 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.069274+0000", 2024-09-17T07:28:49.885 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.77736896700000002, 2024-09-17T07:28:49.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001956958, 2024-09-17T07:28:49.886 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.886 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.886 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.886 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.886 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.886 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.886 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 872 2024-09-17T07:28:49.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.886 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069274+0000", 2024-09-17T07:28:49.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069274+0000", 2024-09-17T07:28:49.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069275+0000", 2024-09-17T07:28:49.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.685e-06 2024-09-17T07:28:49.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069284+0000", 2024-09-17T07:28:49.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3319999999999992e-06 2024-09-17T07:28:49.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069301+0000", 2024-09-17T07:28:49.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7249e-05 2024-09-17T07:28:49.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069311+0000", 2024-09-17T07:28:49.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0219e-05 2024-09-17T07:28:49.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069396+0000", 2024-09-17T07:28:49.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5171000000000002e-05 2024-09-17T07:28:49.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069497+0000", 2024-09-17T07:28:49.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000100907 2024-09-17T07:28:49.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 2", 2024-09-17T07:28:49.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069621+0000", 2024-09-17T07:28:49.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012422799999999999 2024-09-17T07:28:49.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070920+0000", 2024-09-17T07:28:49.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00129839 2024-09-17T07:28:49.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.071123+0000", 2024-09-17T07:28:49.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00020345699999999999 2024-09-17T07:28:49.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.071178+0000", 2024-09-17T07:28:49.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4842000000000001e-05 2024-09-17T07:28:49.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.071231+0000", 2024-09-17T07:28:49.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2478000000000001e-05 2024-09-17T07:28:49.892 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.892 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.892 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.892 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:944 3.19 3:9ff6f59a:::rbd_data.1183350c2d82.0000000000000021:head [set-alloc-hint object_size 1048576 write_size 1048576,write 359349~1099] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:49.892 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.158519+0000", 2024-09-17T07:28:49.892 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.68812378200000002, 2024-09-17T07:28:49.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015446259999999999, 2024-09-17T07:28:49.893 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.893 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.893 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.893 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.893 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.893 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 944 2024-09-17T07:28:49.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.893 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.158519+0000", 2024-09-17T07:28:49.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.158519+0000", 2024-09-17T07:28:49.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.158521+0000", 2024-09-17T07:28:49.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1940000000000001e-06 2024-09-17T07:28:49.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.158533+0000", 2024-09-17T07:28:49.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1776e-05 2024-09-17T07:28:49.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.158556+0000", 2024-09-17T07:28:49.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2934000000000001e-05 2024-09-17T07:28:49.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.158567+0000", 2024-09-17T07:28:49.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1620999999999999e-05 2024-09-17T07:28:49.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.158648+0000", 2024-09-17T07:28:49.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0359999999999996e-05 2024-09-17T07:28:49.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.158777+0000", 2024-09-17T07:28:49.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012950700000000001 2024-09-17T07:28:49.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 3", 2024-09-17T07:28:49.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.158880+0000", 2024-09-17T07:28:49.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000103218 2024-09-17T07:28:49.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.159755+0000", 2024-09-17T07:28:49.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087498899999999995 2024-09-17T07:28:49.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.159998+0000", 2024-09-17T07:28:49.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000242747 2024-09-17T07:28:49.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.160030+0000", 2024-09-17T07:28:49.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2008999999999998e-05 2024-09-17T07:28:49.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.160063+0000", 2024-09-17T07:28:49.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3271e-05 2024-09-17T07:28:49.898 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.899 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.899 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.899 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:963 3.19 3:9fcd161c:::rbd_data.1183350c2d82.0000000000000073:head [set-alloc-hint object_size 1048576 write_size 1048576,write 342344~18104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:49.899 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.198222+0000", 2024-09-17T07:28:49.899 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.64842053499999996, 2024-09-17T07:28:49.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001686431, 2024-09-17T07:28:49.899 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.900 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.900 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.900 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.900 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 963 2024-09-17T07:28:49.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.900 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198222+0000", 2024-09-17T07:28:49.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198222+0000", 2024-09-17T07:28:49.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198224+0000", 2024-09-17T07:28:49.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:28:49.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198250+0000", 2024-09-17T07:28:49.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6625e-05 2024-09-17T07:28:49.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198266+0000", 2024-09-17T07:28:49.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6016e-05 2024-09-17T07:28:49.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198277+0000", 2024-09-17T07:28:49.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0492000000000001e-05 2024-09-17T07:28:49.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198369+0000", 2024-09-17T07:28:49.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1572999999999999e-05 2024-09-17T07:28:49.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198436+0000", 2024-09-17T07:28:49.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7003999999999997e-05 2024-09-17T07:28:49.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 3", 2024-09-17T07:28:49.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198544+0000", 2024-09-17T07:28:49.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000108344 2024-09-17T07:28:49.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.199562+0000", 2024-09-17T07:28:49.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010186170000000001 2024-09-17T07:28:49.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.199834+0000", 2024-09-17T07:28:49.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027103099999999997 2024-09-17T07:28:49.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.199865+0000", 2024-09-17T07:28:49.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1427999999999998e-05 2024-09-17T07:28:49.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.199908+0000", 2024-09-17T07:28:49.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:28:49.907 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.907 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.907 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.907 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:964 3.19 3:9e3affcb:::rbd_data.1183350c2d82.0000000000000074:head [set-alloc-hint object_size 1048576 write_size 1048576,write 327680~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:49.908 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.198565+0000", 2024-09-17T07:28:49.908 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.64807757399999999, 2024-09-17T07:28:49.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001634468, 2024-09-17T07:28:49.908 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.908 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.908 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.908 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.908 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.908 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 964 2024-09-17T07:28:49.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.909 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198565+0000", 2024-09-17T07:28:49.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198565+0000", 2024-09-17T07:28:49.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198566+0000", 2024-09-17T07:28:49.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1999999999999998e-07 2024-09-17T07:28:49.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198598+0000", 2024-09-17T07:28:49.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2592999999999998e-05 2024-09-17T07:28:49.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198610+0000", 2024-09-17T07:28:49.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2018000000000001e-05 2024-09-17T07:28:49.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198619+0000", 2024-09-17T07:28:49.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2339999999999994e-06 2024-09-17T07:28:49.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.198989+0000", 2024-09-17T07:28:49.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00037053299999999998 2024-09-17T07:28:49.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.912 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.912 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.199044+0000", 2024-09-17T07:28:49.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5111000000000001e-05 2024-09-17T07:28:49.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.912 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 3", 2024-09-17T07:28:49.912 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.199194+0000", 2024-09-17T07:28:49.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014937199999999999 2024-09-17T07:28:49.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.913 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.199918+0000", 2024-09-17T07:28:49.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072447099999999997 2024-09-17T07:28:49.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.913 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.200096+0000", 2024-09-17T07:28:49.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017790300000000001 2024-09-17T07:28:49.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.914 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.200154+0000", 2024-09-17T07:28:49.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8326999999999999e-05 2024-09-17T07:28:49.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.914 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.200199+0000", 2024-09-17T07:28:49.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5086000000000002e-05 2024-09-17T07:28:49.914 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.914 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.914 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.915 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:986 3.0 3:04a2435f:::rbd_data.1183350c2d82.0000000000000017:head [set-alloc-hint object_size 1048576 write_size 1048576,write 720896~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:49.915 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.233493+0000", 2024-09-17T07:28:49.915 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.61314995500000002, 2024-09-17T07:28:49.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001694947, 2024-09-17T07:28:49.915 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.915 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.915 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.915 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.915 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.915 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.916 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 986 2024-09-17T07:28:49.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.916 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.916 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.916 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.233493+0000", 2024-09-17T07:28:49.916 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.916 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.917 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.233493+0000", 2024-09-17T07:28:49.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.917 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.917 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.233494+0000", 2024-09-17T07:28:49.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.761e-06 2024-09-17T07:28:49.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.917 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.917 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.233530+0000", 2024-09-17T07:28:49.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5905000000000003e-05 2024-09-17T07:28:49.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.233549+0000", 2024-09-17T07:28:49.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8766e-05 2024-09-17T07:28:49.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.233559+0000", 2024-09-17T07:28:49.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0086e-05 2024-09-17T07:28:49.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.233635+0000", 2024-09-17T07:28:49.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5449000000000002e-05 2024-09-17T07:28:49.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.233783+0000", 2024-09-17T07:28:49.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000148331 2024-09-17T07:28:49.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 2", 2024-09-17T07:28:49.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.233980+0000", 2024-09-17T07:28:49.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00019727300000000001 2024-09-17T07:28:49.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234768+0000", 2024-09-17T07:28:49.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078732800000000001 2024-09-17T07:28:49.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.920 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.920 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.235123+0000", 2024-09-17T07:28:49.920 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00035566900000000003 2024-09-17T07:28:49.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.920 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.920 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.235158+0000", 2024-09-17T07:28:49.920 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4408e-05 2024-09-17T07:28:49.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.920 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.921 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.235188+0000", 2024-09-17T07:28:49.921 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9971000000000002e-05 2024-09-17T07:28:49.921 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.921 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.921 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.921 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1016 3.19 3:9fcd161c:::rbd_data.1183350c2d82.0000000000000073:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:49.921 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.270056+0000", 2024-09-17T07:28:49.921 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.57658628999999995, 2024-09-17T07:28:49.921 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025381100000000001, 2024-09-17T07:28:49.921 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.921 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.922 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.922 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.922 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.922 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1016 2024-09-17T07:28:49.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.922 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270056+0000", 2024-09-17T07:28:49.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270056+0000", 2024-09-17T07:28:49.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270058+0000", 2024-09-17T07:28:49.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.807e-06 2024-09-17T07:28:49.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270065+0000", 2024-09-17T07:28:49.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9120000000000001e-06 2024-09-17T07:28:49.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270084+0000", 2024-09-17T07:28:49.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8669e-05 2024-09-17T07:28:49.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270094+0000", 2024-09-17T07:28:49.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0319000000000001e-05 2024-09-17T07:28:49.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270196+0000", 2024-09-17T07:28:49.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000101578 2024-09-17T07:28:49.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270282+0000", 2024-09-17T07:28:49.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6759000000000001e-05 2024-09-17T07:28:49.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.926 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 3", 2024-09-17T07:28:49.926 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270388+0000", 2024-09-17T07:28:49.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010599000000000001 2024-09-17T07:28:49.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.926 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.926 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271630+0000", 2024-09-17T07:28:49.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012418290000000001 2024-09-17T07:28:49.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.926 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.926 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.272499+0000", 2024-09-17T07:28:49.927 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00086916200000000004 2024-09-17T07:28:49.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.927 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.927 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.272552+0000", 2024-09-17T07:28:49.927 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2905999999999998e-05 2024-09-17T07:28:49.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.927 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.928 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.272594+0000", 2024-09-17T07:28:49.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2178999999999997e-05 2024-09-17T07:28:49.928 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.928 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.928 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.928 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1017 3.19 3:9e3affcb:::rbd_data.1183350c2d82.0000000000000074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:49.928 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.270121+0000", 2024-09-17T07:28:49.929 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.57652177599999999, 2024-09-17T07:28:49.929 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002655676, 2024-09-17T07:28:49.929 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.929 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.929 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.929 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.929 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.929 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1017 2024-09-17T07:28:49.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.929 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270121+0000", 2024-09-17T07:28:49.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270121+0000", 2024-09-17T07:28:49.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270121+0000", 2024-09-17T07:28:49.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0299999999999999e-07 2024-09-17T07:28:49.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270124+0000", 2024-09-17T07:28:49.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.706e-06 2024-09-17T07:28:49.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270145+0000", 2024-09-17T07:28:49.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0550000000000001e-05 2024-09-17T07:28:49.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270151+0000", 2024-09-17T07:28:49.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6059999999999999e-06 2024-09-17T07:28:49.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.933 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270757+0000", 2024-09-17T07:28:49.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00060567100000000001 2024-09-17T07:28:49.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270831+0000", 2024-09-17T07:28:49.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4507999999999999e-05 2024-09-17T07:28:49.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 3", 2024-09-17T07:28:49.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270935+0000", 2024-09-17T07:28:49.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000103325 2024-09-17T07:28:49.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.272057+0000", 2024-09-17T07:28:49.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011223170000000001 2024-09-17T07:28:49.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.272660+0000", 2024-09-17T07:28:49.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00060319200000000001 2024-09-17T07:28:49.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.272733+0000", 2024-09-17T07:28:49.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2298999999999993e-05 2024-09-17T07:28:49.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.272776+0000", 2024-09-17T07:28:49.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3899000000000001e-05 2024-09-17T07:28:49.937 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.937 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.937 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.937 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1018 3.4 3:20e84768:::rbd_data.1183350c2d82.0000000000000075:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:49.937 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.270224+0000", 2024-09-17T07:28:49.937 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.57641837699999998, 2024-09-17T07:28:49.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001711069, 2024-09-17T07:28:49.938 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.938 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.938 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.938 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.938 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.938 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.938 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1018 2024-09-17T07:28:49.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.938 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270224+0000", 2024-09-17T07:28:49.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270224+0000", 2024-09-17T07:28:49.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270225+0000", 2024-09-17T07:28:49.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9999999999999997e-07 2024-09-17T07:28:49.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270229+0000", 2024-09-17T07:28:49.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1740000000000002e-06 2024-09-17T07:28:49.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270241+0000", 2024-09-17T07:28:49.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1576000000000001e-05 2024-09-17T07:28:49.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270250+0000", 2024-09-17T07:28:49.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3670000000000003e-06 2024-09-17T07:28:49.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270308+0000", 2024-09-17T07:28:49.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7620000000000001e-05 2024-09-17T07:28:49.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270402+0000", 2024-09-17T07:28:49.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4301000000000004e-05 2024-09-17T07:28:49.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 2", 2024-09-17T07:28:49.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270513+0000", 2024-09-17T07:28:49.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000110802 2024-09-17T07:28:49.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271599+0000", 2024-09-17T07:28:49.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010864 2024-09-17T07:28:49.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271828+0000", 2024-09-17T07:28:49.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022920800000000001 2024-09-17T07:28:49.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271885+0000", 2024-09-17T07:28:49.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6407999999999999e-05 2024-09-17T07:28:49.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271935+0000", 2024-09-17T07:28:49.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0513000000000002e-05 2024-09-17T07:28:49.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.944 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.945 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1049 3.19 3:9d006c20:::rbd_data.1183350c2d82.0000000000000068:head [set-alloc-hint object_size 1048576 write_size 1048576,write 393216~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:49.945 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.293261+0000", 2024-09-17T07:28:49.945 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.55338198500000002, 2024-09-17T07:28:49.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016597510000000001, 2024-09-17T07:28:49.945 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.945 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.945 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.945 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.945 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.945 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.945 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1049 2024-09-17T07:28:49.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.946 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293261+0000", 2024-09-17T07:28:49.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293261+0000", 2024-09-17T07:28:49.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293263+0000", 2024-09-17T07:28:49.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0099999999999998e-06 2024-09-17T07:28:49.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293280+0000", 2024-09-17T07:28:49.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7700999999999999e-05 2024-09-17T07:28:49.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293297+0000", 2024-09-17T07:28:49.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7082999999999999e-05 2024-09-17T07:28:49.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293308+0000", 2024-09-17T07:28:49.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0471e-05 2024-09-17T07:28:49.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293378+0000", 2024-09-17T07:28:49.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9725000000000001e-05 2024-09-17T07:28:49.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293540+0000", 2024-09-17T07:28:49.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000162318 2024-09-17T07:28:49.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 3", 2024-09-17T07:28:49.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293709+0000", 2024-09-17T07:28:49.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000169078 2024-09-17T07:28:49.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.294535+0000", 2024-09-17T07:28:49.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00082631200000000005 2024-09-17T07:28:49.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.950 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.294816+0000", 2024-09-17T07:28:49.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00028067000000000001 2024-09-17T07:28:49.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.950 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.950 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.294878+0000", 2024-09-17T07:28:49.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1530000000000002e-05 2024-09-17T07:28:49.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.950 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.950 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.294920+0000", 2024-09-17T07:28:49.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2852999999999998e-05 2024-09-17T07:28:49.950 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.951 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.951 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1054 3.0 3:04a2435f:::rbd_data.1183350c2d82.0000000000000017:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:49.951 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.303247+0000", 2024-09-17T07:28:49.951 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.54339537199999999, 2024-09-17T07:28:49.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020678430000000002, 2024-09-17T07:28:49.951 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.951 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.951 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.951 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.951 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.951 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1054 2024-09-17T07:28:49.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.952 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303247+0000", 2024-09-17T07:28:49.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303247+0000", 2024-09-17T07:28:49.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303250+0000", 2024-09-17T07:28:49.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3159999999999999e-06 2024-09-17T07:28:49.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303259+0000", 2024-09-17T07:28:49.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2650000000000002e-06 2024-09-17T07:28:49.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303279+0000", 2024-09-17T07:28:49.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0605e-05 2024-09-17T07:28:49.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303292+0000", 2024-09-17T07:28:49.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2442e-05 2024-09-17T07:28:49.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303375+0000", 2024-09-17T07:28:49.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3535000000000006e-05 2024-09-17T07:28:49.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303485+0000", 2024-09-17T07:28:49.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000109231 2024-09-17T07:28:49.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 2", 2024-09-17T07:28:49.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303609+0000", 2024-09-17T07:28:49.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012396 2024-09-17T07:28:49.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304829+0000", 2024-09-17T07:28:49.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012204259999999999 2024-09-17T07:28:49.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.305209+0000", 2024-09-17T07:28:49.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00037964999999999999 2024-09-17T07:28:49.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.956 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.305268+0000", 2024-09-17T07:28:49.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9443e-05 2024-09-17T07:28:49.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.956 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.305315+0000", 2024-09-17T07:28:49.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6969999999999999e-05 2024-09-17T07:28:49.957 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.957 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.957 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.957 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1055 3.19 3:9ff6f59a:::rbd_data.1183350c2d82.0000000000000021:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:49.957 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.303337+0000", 2024-09-17T07:28:49.957 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.54330583499999996, 2024-09-17T07:28:49.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022301759999999999, 2024-09-17T07:28:49.957 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.957 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.958 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.958 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.958 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1055 2024-09-17T07:28:49.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.958 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303337+0000", 2024-09-17T07:28:49.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303337+0000", 2024-09-17T07:28:49.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303337+0000", 2024-09-17T07:28:49.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0299999999999999e-07 2024-09-17T07:28:49.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303342+0000", 2024-09-17T07:28:49.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:28:49.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303351+0000", 2024-09-17T07:28:49.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7320000000000004e-06 2024-09-17T07:28:49.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303358+0000", 2024-09-17T07:28:49.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0940000000000002e-06 2024-09-17T07:28:49.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303420+0000", 2024-09-17T07:28:49.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2102999999999998e-05 2024-09-17T07:28:49.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303509+0000", 2024-09-17T07:28:49.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8838999999999994e-05 2024-09-17T07:28:49.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 3", 2024-09-17T07:28:49.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303617+0000", 2024-09-17T07:28:49.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000107975 2024-09-17T07:28:49.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304830+0000", 2024-09-17T07:28:49.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012125110000000001 2024-09-17T07:28:49.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.305467+0000", 2024-09-17T07:28:49.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063747800000000002 2024-09-17T07:28:49.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.305522+0000", 2024-09-17T07:28:49.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5368000000000003e-05 2024-09-17T07:28:49.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.305567+0000", 2024-09-17T07:28:49.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4508999999999997e-05 2024-09-17T07:28:49.963 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.963 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.963 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.964 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1059 3.19 3:9d006c20:::rbd_data.1183350c2d82.0000000000000068:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:49.964 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.303423+0000", 2024-09-17T07:28:49.964 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.54321955499999997, 2024-09-17T07:28:49.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023206759999999998, 2024-09-17T07:28:49.964 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.964 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.964 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.964 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.964 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.964 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.964 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1059 2024-09-17T07:28:49.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.965 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.965 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303423+0000", 2024-09-17T07:28:49.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.965 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303423+0000", 2024-09-17T07:28:49.966 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.966 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.966 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303435+0000", 2024-09-17T07:28:49.966 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2379e-05 2024-09-17T07:28:49.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.966 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.966 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303466+0000", 2024-09-17T07:28:49.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0439000000000001e-05 2024-09-17T07:28:49.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.967 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303527+0000", 2024-09-17T07:28:49.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1061000000000004e-05 2024-09-17T07:28:49.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303612+0000", 2024-09-17T07:28:49.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4667999999999996e-05 2024-09-17T07:28:49.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303985+0000", 2024-09-17T07:28:49.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000373178 2024-09-17T07:28:49.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304087+0000", 2024-09-17T07:28:49.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000102346 2024-09-17T07:28:49.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 3", 2024-09-17T07:28:49.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304254+0000", 2024-09-17T07:28:49.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000166863 2024-09-17T07:28:49.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.305165+0000", 2024-09-17T07:28:49.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00091113999999999995 2024-09-17T07:28:49.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.305681+0000", 2024-09-17T07:28:49.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00051563200000000003 2024-09-17T07:28:49.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.971 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.305721+0000", 2024-09-17T07:28:49.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0031999999999999e-05 2024-09-17T07:28:49.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.971 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.305744+0000", 2024-09-17T07:28:49.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2938e-05 2024-09-17T07:28:49.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.972 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.972 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1159 3.0 3:07b6f4cb:::rbd_data.1183350c2d82.0000000000000063:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:49.972 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.447955+0000", 2024-09-17T07:28:49.973 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.39868762400000002, 2024-09-17T07:28:49.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0048646510000000002, 2024-09-17T07:28:49.973 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:49.973 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.973 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.973 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.973 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1159 2024-09-17T07:28:49.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.974 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.447955+0000", 2024-09-17T07:28:49.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.447955+0000", 2024-09-17T07:28:49.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.447956+0000", 2024-09-17T07:28:49.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4190000000000001e-06 2024-09-17T07:28:49.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.447961+0000", 2024-09-17T07:28:49.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0690000000000001e-06 2024-09-17T07:28:49.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.447973+0000", 2024-09-17T07:28:49.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1891e-05 2024-09-17T07:28:49.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.447981+0000", 2024-09-17T07:28:49.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7619999999999995e-06 2024-09-17T07:28:49.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.448036+0000", 2024-09-17T07:28:49.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4926999999999998e-05 2024-09-17T07:28:49.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.448114+0000", 2024-09-17T07:28:49.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7928000000000003e-05 2024-09-17T07:28:49.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 2", 2024-09-17T07:28:49.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.448937+0000", 2024-09-17T07:28:49.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00082319999999999995 2024-09-17T07:28:49.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:49.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.452722+0000", 2024-09-17T07:28:49.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037851199999999999 2024-09-17T07:28:49.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:49.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.452746+0000", 2024-09-17T07:28:49.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4189000000000002e-05 2024-09-17T07:28:49.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:49.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.452774+0000", 2024-09-17T07:28:49.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7545000000000001e-05 2024-09-17T07:28:49.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.452820+0000", 2024-09-17T07:28:49.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5600999999999999e-05 2024-09-17T07:28:49.980 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.980 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.980 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.980 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1159 3.0 3:07b6f4cb:::rbd_data.1183350c2d82.0000000000000063:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:49.980 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.448557+0000", 2024-09-17T07:28:49.980 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.39808565899999998, 2024-09-17T07:28:49.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0042253899999999999, 2024-09-17T07:28:49.980 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.980 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.980 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "waiting for ondisk", 2024-09-17T07:28:49.981 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:49.981 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:49.981 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:49.981 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1159 2024-09-17T07:28:49.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.981 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.448557+0000", 2024-09-17T07:28:49.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.448557+0000", 2024-09-17T07:28:49.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.448558+0000", 2024-09-17T07:28:49.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-07 2024-09-17T07:28:49.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.448562+0000", 2024-09-17T07:28:49.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9500000000000003e-06 2024-09-17T07:28:49.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.448570+0000", 2024-09-17T07:28:49.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2400000000000007e-06 2024-09-17T07:28:49.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.448579+0000", 2024-09-17T07:28:49.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.884e-06 2024-09-17T07:28:49.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.452659+0000", 2024-09-17T07:28:49.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0040804709999999996 2024-09-17T07:28:49.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.984 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for ondisk", 2024-09-17T07:28:49.984 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.452708+0000", 2024-09-17T07:28:49.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8403999999999999e-05 2024-09-17T07:28:49.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.984 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.984 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.452782+0000", 2024-09-17T07:28:49.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4793e-05 2024-09-17T07:28:49.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.984 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.984 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop_reply(client.4483.0:1159 3.0 e29/23 ondisk, result = 0)", 2024-09-17T07:28:49.985 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.451119+0000", 2024-09-17T07:28:49.985 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.39552312000000001, 2024-09-17T07:28:49.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001709052, 2024-09-17T07:28:49.985 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:49.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:49.985 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:49.985 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:49.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:49.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.451119+0000", 2024-09-17T07:28:49.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:49.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.451119+0000", 2024-09-17T07:28:49.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:49.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:49.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.451120+0000", 2024-09-17T07:28:49.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2500000000000005e-07 2024-09-17T07:28:49.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:49.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.451124+0000", 2024-09-17T07:28:49.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9790000000000004e-06 2024-09-17T07:28:49.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:49.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.451137+0000", 2024-09-17T07:28:49.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2829e-05 2024-09-17T07:28:49.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:49.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.451143+0000", 2024-09-17T07:28:49.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8019999999999997e-06 2024-09-17T07:28:49.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:49.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.452739+0000", 2024-09-17T07:28:49.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001596625 2024-09-17T07:28:49.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:49.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.452744+0000", 2024-09-17T07:28:49.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2579999999999997e-06 2024-09-17T07:28:49.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:49.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:49.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:49.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.452829+0000", 2024-09-17T07:28:49.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4833999999999994e-05 2024-09-17T07:28:49.988 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.989 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.989 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.989 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:49.989 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:49.989 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:49.990 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:28:49.991 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:49.990+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:50.092 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:50.091+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:50.100 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:50.100 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:50.100 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:28:50.100 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:50.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:28:50.159 INFO:teuthology.orchestra.run.smithi045.stdout:leaving image image_client.0-clone1 intact 2024-09-17T07:28:50.192 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:50.191+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:50.224 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:50.224 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:50.224 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:28:50.224 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:28:50.224 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:50.233 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:28:50.293 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:50.293+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:50.305 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-17T07:28:50.394 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:50.394 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:28:50.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:28:50.394 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:28:50.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.395 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:744 3.0 e28/23 3:07b6f4cb:::rbd_data.1183350c2d82.0000000000000063:head v 28'1, mlcod=0'0)", 2024-09-17T07:28:50.395 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.898634+0000", 2024-09-17T07:28:50.395 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.4934762880000001, 2024-09-17T07:28:50.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000817887, 2024-09-17T07:28:50.395 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.395 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.395 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.898634+0000", 2024-09-17T07:28:50.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.396 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.898634+0000", 2024-09-17T07:28:50.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.396 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.898635+0000", 2024-09-17T07:28:50.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006e-06 2024-09-17T07:28:50.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.396 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.898679+0000", 2024-09-17T07:28:50.397 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4462999999999998e-05 2024-09-17T07:28:50.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.397 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.397 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.898693+0000", 2024-09-17T07:28:50.397 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3560000000000001e-05 2024-09-17T07:28:50.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.397 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.397 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.898705+0000", 2024-09-17T07:28:50.397 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2510999999999999e-05 2024-09-17T07:28:50.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.397 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.398 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.898763+0000", 2024-09-17T07:28:50.398 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7574000000000001e-05 2024-09-17T07:28:50.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.398 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.398 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.898776+0000", 2024-09-17T07:28:50.398 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3183999999999999e-05 2024-09-17T07:28:50.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.398 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.398 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.899425+0000", 2024-09-17T07:28:50.398 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00064891199999999997 2024-09-17T07:28:50.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.899451+0000", 2024-09-17T07:28:50.399 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6676999999999999e-05 2024-09-17T07:28:50.399 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.399 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.399 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.399 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:759 3.4 e28/23 3:20e84768:::rbd_data.1183350c2d82.0000000000000075:head v 28'6, mlcod=28'4)", 2024-09-17T07:28:50.399 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:28:50.399 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.4672488050000001, 2024-09-17T07:28:50.399 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011641620000000001, 2024-09-17T07:28:50.399 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.400 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.400 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:28:50.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:28:50.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:28:50.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6100000000000002e-07 2024-09-17T07:28:50.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924866+0000", 2024-09-17T07:28:50.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:28:50.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924875+0000", 2024-09-17T07:28:50.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7909999999999993e-06 2024-09-17T07:28:50.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.402 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.402 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924882+0000", 2024-09-17T07:28:50.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.418e-06 2024-09-17T07:28:50.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.402 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.402 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924965+0000", 2024-09-17T07:28:50.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3029999999999996e-05 2024-09-17T07:28:50.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.402 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924987+0000", 2024-09-17T07:28:50.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1747e-05 2024-09-17T07:28:50.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.925980+0000", 2024-09-17T07:28:50.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00099299900000000005 2024-09-17T07:28:50.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.926025+0000", 2024-09-17T07:28:50.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4977e-05 2024-09-17T07:28:50.403 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.403 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.404 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.404 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:778 3.9 e28/23 3:97508d79:::rbd_header.1183350c2d82:head v 28'42, mlcod=28'40)", 2024-09-17T07:28:50.404 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.949619+0000", 2024-09-17T07:28:50.404 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.4424904000000001, 2024-09-17T07:28:50.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00077993300000000002, 2024-09-17T07:28:50.404 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.404 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.404 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.949619+0000", 2024-09-17T07:28:50.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.949619+0000", 2024-09-17T07:28:50.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.949622+0000", 2024-09-17T07:28:50.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.34e-06 2024-09-17T07:28:50.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.406 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.949638+0000", 2024-09-17T07:28:50.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5913000000000002e-05 2024-09-17T07:28:50.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.406 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.949671+0000", 2024-09-17T07:28:50.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3599000000000001e-05 2024-09-17T07:28:50.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.406 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.949687+0000", 2024-09-17T07:28:50.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5883e-05 2024-09-17T07:28:50.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.407 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.407 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.949833+0000", 2024-09-17T07:28:50.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014632600000000001 2024-09-17T07:28:50.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.407 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.407 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.949854+0000", 2024-09-17T07:28:50.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0540999999999999e-05 2024-09-17T07:28:50.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.407 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.407 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.950361+0000", 2024-09-17T07:28:50.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00050657700000000003 2024-09-17T07:28:50.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.408 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.408 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.950399+0000", 2024-09-17T07:28:50.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8754000000000002e-05 2024-09-17T07:28:50.408 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.408 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.408 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.408 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:820 3.9 e28/23 3:97508d79:::rbd_header.1183350c2d82:head v 28'45, mlcod=28'43)", 2024-09-17T07:28:50.408 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.002387+0000", 2024-09-17T07:28:50.408 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.3897228770000001, 2024-09-17T07:28:50.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00077187399999999998, 2024-09-17T07:28:50.409 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.409 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.409 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.409 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.409 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.002387+0000", 2024-09-17T07:28:50.409 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.409 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.002387+0000", 2024-09-17T07:28:50.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.410 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.002389+0000", 2024-09-17T07:28:50.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.762e-06 2024-09-17T07:28:50.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.410 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.002400+0000", 2024-09-17T07:28:50.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1127999999999999e-05 2024-09-17T07:28:50.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.410 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.002421+0000", 2024-09-17T07:28:50.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0852e-05 2024-09-17T07:28:50.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.411 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.002439+0000", 2024-09-17T07:28:50.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7859e-05 2024-09-17T07:28:50.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.411 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.002503+0000", 2024-09-17T07:28:50.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4641000000000007e-05 2024-09-17T07:28:50.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.002522+0000", 2024-09-17T07:28:50.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8326999999999999e-05 2024-09-17T07:28:50.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.003124+0000", 2024-09-17T07:28:50.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00060223199999999996 2024-09-17T07:28:50.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.003159+0000", 2024-09-17T07:28:50.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5073000000000001e-05 2024-09-17T07:28:50.413 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.413 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.413 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.413 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:856 3.4 e28/23 3:220acdab:::rbd_data.1183350c2d82.0000000000000070:head v 28'7, mlcod=28'5)", 2024-09-17T07:28:50.413 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.042406+0000", 2024-09-17T07:28:50.413 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.3497039580000001, 2024-09-17T07:28:50.413 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00091025500000000001, 2024-09-17T07:28:50.413 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.413 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.413 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.413 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.414 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042406+0000", 2024-09-17T07:28:50.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.414 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042406+0000", 2024-09-17T07:28:50.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.414 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042407+0000", 2024-09-17T07:28:50.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4169999999999999e-06 2024-09-17T07:28:50.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.415 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042438+0000", 2024-09-17T07:28:50.415 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0409e-05 2024-09-17T07:28:50.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.415 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042455+0000", 2024-09-17T07:28:50.415 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6883000000000001e-05 2024-09-17T07:28:50.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042463+0000", 2024-09-17T07:28:50.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6470000000000004e-06 2024-09-17T07:28:50.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.416 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042532+0000", 2024-09-17T07:28:50.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8651000000000004e-05 2024-09-17T07:28:50.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.416 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042554+0000", 2024-09-17T07:28:50.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1889e-05 2024-09-17T07:28:50.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.043272+0000", 2024-09-17T07:28:50.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00071802499999999996 2024-09-17T07:28:50.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.043316+0000", 2024-09-17T07:28:50.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4333999999999999e-05 2024-09-17T07:28:50.417 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.418 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.418 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.418 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:872 3.4 e28/23 3:220acdab:::rbd_data.1183350c2d82.0000000000000070:head v 28'8, mlcod=28'6)", 2024-09-17T07:28:50.418 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.070116+0000", 2024-09-17T07:28:50.418 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.321993784, 2024-09-17T07:28:50.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00085586199999999999, 2024-09-17T07:28:50.418 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.418 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.418 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.418 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070116+0000", 2024-09-17T07:28:50.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070116+0000", 2024-09-17T07:28:50.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070118+0000", 2024-09-17T07:28:50.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5230000000000001e-06 2024-09-17T07:28:50.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070129+0000", 2024-09-17T07:28:50.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1045e-05 2024-09-17T07:28:50.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070151+0000", 2024-09-17T07:28:50.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2323e-05 2024-09-17T07:28:50.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070170+0000", 2024-09-17T07:28:50.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8576e-05 2024-09-17T07:28:50.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.421 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070244+0000", 2024-09-17T07:28:50.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4480000000000005e-05 2024-09-17T07:28:50.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.421 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070268+0000", 2024-09-17T07:28:50.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4392e-05 2024-09-17T07:28:50.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070914+0000", 2024-09-17T07:28:50.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00064570999999999997 2024-09-17T07:28:50.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070972+0000", 2024-09-17T07:28:50.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7812999999999997e-05 2024-09-17T07:28:50.422 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.422 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.422 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.423 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:903 3.9 e29/23 3:97508d79:::rbd_header.1183350c2d82:head v 29'49, mlcod=28'47)", 2024-09-17T07:28:50.423 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.104471+0000", 2024-09-17T07:28:50.423 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.2876390170000001, 2024-09-17T07:28:50.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00090332699999999999, 2024-09-17T07:28:50.423 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.423 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.423 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.423 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104471+0000", 2024-09-17T07:28:50.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104471+0000", 2024-09-17T07:28:50.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104471+0000", 2024-09-17T07:28:50.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:28:50.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104490+0000", 2024-09-17T07:28:50.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8678e-05 2024-09-17T07:28:50.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104501+0000", 2024-09-17T07:28:50.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1559999999999999e-05 2024-09-17T07:28:50.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104510+0000", 2024-09-17T07:28:50.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0610000000000002e-06 2024-09-17T07:28:50.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.426 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.426 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104573+0000", 2024-09-17T07:28:50.426 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3071000000000007e-05 2024-09-17T07:28:50.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.426 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.426 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104593+0000", 2024-09-17T07:28:50.426 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9371000000000001e-05 2024-09-17T07:28:50.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.426 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.426 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.105327+0000", 2024-09-17T07:28:50.426 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073368900000000004 2024-09-17T07:28:50.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.427 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.427 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.105374+0000", 2024-09-17T07:28:50.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7589999999999997e-05 2024-09-17T07:28:50.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.427 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.427 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.427 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:965 3.4 e29/23 3:20e84768:::rbd_data.1183350c2d82.0000000000000075:head v 29'9, mlcod=28'7)", 2024-09-17T07:28:50.427 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.201308+0000", 2024-09-17T07:28:50.427 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.190801478, 2024-09-17T07:28:50.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00077109300000000004, 2024-09-17T07:28:50.428 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.428 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.428 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.428 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.201308+0000", 2024-09-17T07:28:50.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.201308+0000", 2024-09-17T07:28:50.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.201309+0000", 2024-09-17T07:28:50.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.84e-07 2024-09-17T07:28:50.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.201319+0000", 2024-09-17T07:28:50.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0175e-05 2024-09-17T07:28:50.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.201332+0000", 2024-09-17T07:28:50.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2986999999999999e-05 2024-09-17T07:28:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.201341+0000", 2024-09-17T07:28:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7579999999999998e-06 2024-09-17T07:28:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.201405+0000", 2024-09-17T07:28:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4750000000000002e-05 2024-09-17T07:28:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.201434+0000", 2024-09-17T07:28:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8779999999999999e-05 2024-09-17T07:28:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.202026+0000", 2024-09-17T07:28:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00059192200000000004 2024-09-17T07:28:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.202079+0000", 2024-09-17T07:28:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3436999999999997e-05 2024-09-17T07:28:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.431 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:978 3.9 e29/23 3:93118134:::rbd_data.1183350c2d82.0000000000000067:head v 29'53, mlcod=29'51)", 2024-09-17T07:28:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.220202+0000", 2024-09-17T07:28:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.1719080749999999, 2024-09-17T07:28:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00082530500000000003, 2024-09-17T07:28:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.220202+0000", 2024-09-17T07:28:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.220202+0000", 2024-09-17T07:28:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.220203+0000", 2024-09-17T07:28:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0359999999999999e-06 2024-09-17T07:28:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.220226+0000", 2024-09-17T07:28:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2863000000000001e-05 2024-09-17T07:28:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.220238+0000", 2024-09-17T07:28:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2714000000000001e-05 2024-09-17T07:28:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.220245+0000", 2024-09-17T07:28:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4729999999999997e-06 2024-09-17T07:28:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.220294+0000", 2024-09-17T07:28:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9089999999999999e-05 2024-09-17T07:28:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.220309+0000", 2024-09-17T07:28:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4859e-05 2024-09-17T07:28:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.220980+0000", 2024-09-17T07:28:50.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00067088300000000005 2024-09-17T07:28:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.221027+0000", 2024-09-17T07:28:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7386999999999999e-05 2024-09-17T07:28:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:986 3.0 e29/23 3:04a2435f:::rbd_data.1183350c2d82.0000000000000017:head v 29'3, mlcod=28'1)", 2024-09-17T07:28:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.234074+0000", 2024-09-17T07:28:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.158036131, 2024-09-17T07:28:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00094026099999999996, 2024-09-17T07:28:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234074+0000", 2024-09-17T07:28:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234074+0000", 2024-09-17T07:28:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234075+0000", 2024-09-17T07:28:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.344e-06 2024-09-17T07:28:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234117+0000", 2024-09-17T07:28:50.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1971000000000002e-05 2024-09-17T07:28:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234137+0000", 2024-09-17T07:28:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0432e-05 2024-09-17T07:28:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234145+0000", 2024-09-17T07:28:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8159999999999994e-06 2024-09-17T07:28:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234205+0000", 2024-09-17T07:28:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9979999999999998e-05 2024-09-17T07:28:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234230+0000", 2024-09-17T07:28:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4587e-05 2024-09-17T07:28:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234965+0000", 2024-09-17T07:28:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073509300000000003 2024-09-17T07:28:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.235014+0000", 2024-09-17T07:28:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9038e-05 2024-09-17T07:28:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1018 3.4 e29/23 3:20e84768:::rbd_data.1183350c2d82.0000000000000075:head v 29'10, mlcod=28'8)", 2024-09-17T07:28:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.270607+0000", 2024-09-17T07:28:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.1215024709999999, 2024-09-17T07:28:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00091791799999999999, 2024-09-17T07:28:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270607+0000", 2024-09-17T07:28:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270607+0000", 2024-09-17T07:28:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270609+0000", 2024-09-17T07:28:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6619999999999999e-06 2024-09-17T07:28:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270620+0000", 2024-09-17T07:28:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0718e-05 2024-09-17T07:28:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270640+0000", 2024-09-17T07:28:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0760000000000001e-05 2024-09-17T07:28:50.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270650+0000", 2024-09-17T07:28:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6080000000000002e-06 2024-09-17T07:28:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270715+0000", 2024-09-17T07:28:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5265999999999995e-05 2024-09-17T07:28:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270740+0000", 2024-09-17T07:28:50.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4584e-05 2024-09-17T07:28:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271483+0000", 2024-09-17T07:28:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000742941 2024-09-17T07:28:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271525+0000", 2024-09-17T07:28:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2379000000000002e-05 2024-09-17T07:28:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1012 3.9 e29/23 3:93118134:::rbd_data.1183350c2d82.0000000000000067:head v 29'55, mlcod=29'53)", 2024-09-17T07:28:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.270691+0000", 2024-09-17T07:28:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.1214187410000001, 2024-09-17T07:28:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011919299999999999, 2024-09-17T07:28:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270691+0000", 2024-09-17T07:28:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270691+0000", 2024-09-17T07:28:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270692+0000", 2024-09-17T07:28:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9800000000000002e-07 2024-09-17T07:28:50.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270699+0000", 2024-09-17T07:28:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1679999999999998e-06 2024-09-17T07:28:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270717+0000", 2024-09-17T07:28:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8193e-05 2024-09-17T07:28:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270726+0000", 2024-09-17T07:28:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4519999999999997e-06 2024-09-17T07:28:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270779+0000", 2024-09-17T07:28:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3202000000000003e-05 2024-09-17T07:28:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270800+0000", 2024-09-17T07:28:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0846e-05 2024-09-17T07:28:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271841+0000", 2024-09-17T07:28:50.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001040921 2024-09-17T07:28:50.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271883+0000", 2024-09-17T07:28:50.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2150000000000001e-05 2024-09-17T07:28:50.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.450 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.450 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1037 3.9 e29/23 3:97508d79:::rbd_header.1183350c2d82:head v 29'57, mlcod=29'55)", 2024-09-17T07:28:50.450 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.287165+0000", 2024-09-17T07:28:50.451 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.1049449870000001, 2024-09-17T07:28:50.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076849899999999996, 2024-09-17T07:28:50.451 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.451 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.451 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.287165+0000", 2024-09-17T07:28:50.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.287165+0000", 2024-09-17T07:28:50.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.287167+0000", 2024-09-17T07:28:50.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.841e-06 2024-09-17T07:28:50.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.287180+0000", 2024-09-17T07:28:50.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3730000000000001e-05 2024-09-17T07:28:50.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.287201+0000", 2024-09-17T07:28:50.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0435e-05 2024-09-17T07:28:50.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.287210+0000", 2024-09-17T07:28:50.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2350000000000005e-06 2024-09-17T07:28:50.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.287285+0000", 2024-09-17T07:28:50.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4820999999999994e-05 2024-09-17T07:28:50.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.287312+0000", 2024-09-17T07:28:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7189999999999999e-05 2024-09-17T07:28:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.287888+0000", 2024-09-17T07:28:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000575841 2024-09-17T07:28:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.287933+0000", 2024-09-17T07:28:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5405999999999999e-05 2024-09-17T07:28:50.454 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.455 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.455 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.455 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1048 3.9 e29/23 3:93118134:::rbd_data.1183350c2d82.0000000000000067:head v 29'58, mlcod=29'56)", 2024-09-17T07:28:50.455 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.293369+0000", 2024-09-17T07:28:50.455 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.0987411170000001, 2024-09-17T07:28:50.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00088354799999999995, 2024-09-17T07:28:50.455 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.455 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.455 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293369+0000", 2024-09-17T07:28:50.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293369+0000", 2024-09-17T07:28:50.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293369+0000", 2024-09-17T07:28:50.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2900000000000003e-07 2024-09-17T07:28:50.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293381+0000", 2024-09-17T07:28:50.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1773999999999999e-05 2024-09-17T07:28:50.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293393+0000", 2024-09-17T07:28:50.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1600000000000001e-05 2024-09-17T07:28:50.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293399+0000", 2024-09-17T07:28:50.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.657e-06 2024-09-17T07:28:50.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.458 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.458 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293448+0000", 2024-09-17T07:28:50.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8238000000000001e-05 2024-09-17T07:28:50.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.458 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.458 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.293467+0000", 2024-09-17T07:28:50.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9383999999999999e-05 2024-09-17T07:28:50.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.458 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.458 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.294209+0000", 2024-09-17T07:28:50.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00074153999999999995 2024-09-17T07:28:50.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.294252+0000", 2024-09-17T07:28:50.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3625999999999999e-05 2024-09-17T07:28:50.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.459 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.459 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1058 3.9 e29/23 3:93118134:::rbd_data.1183350c2d82.0000000000000067:head v 29'59, mlcod=29'57)", 2024-09-17T07:28:50.459 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.303628+0000", 2024-09-17T07:28:50.459 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.088481966, 2024-09-17T07:28:50.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001104257, 2024-09-17T07:28:50.460 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.460 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.460 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.460 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303628+0000", 2024-09-17T07:28:50.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303628+0000", 2024-09-17T07:28:50.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303629+0000", 2024-09-17T07:28:50.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.347e-06 2024-09-17T07:28:50.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303640+0000", 2024-09-17T07:28:50.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0387999999999999e-05 2024-09-17T07:28:50.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303661+0000", 2024-09-17T07:28:50.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1263999999999998e-05 2024-09-17T07:28:50.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303670+0000", 2024-09-17T07:28:50.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.55e-06 2024-09-17T07:28:50.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303739+0000", 2024-09-17T07:28:50.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8523999999999996e-05 2024-09-17T07:28:50.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303764+0000", 2024-09-17T07:28:50.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4800999999999999e-05 2024-09-17T07:28:50.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304674+0000", 2024-09-17T07:28:50.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00090985600000000001 2024-09-17T07:28:50.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304732+0000", 2024-09-17T07:28:50.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8526999999999997e-05 2024-09-17T07:28:50.463 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.463 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.464 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1054 3.0 e29/23 3:04a2435f:::rbd_data.1183350c2d82.0000000000000017:head v 29'4, mlcod=28'2)", 2024-09-17T07:28:50.464 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.304028+0000", 2024-09-17T07:28:50.464 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.088081735, 2024-09-17T07:28:50.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010024789999999999, 2024-09-17T07:28:50.464 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.464 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304028+0000", 2024-09-17T07:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304028+0000", 2024-09-17T07:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304029+0000", 2024-09-17T07:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1429999999999999e-06 2024-09-17T07:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304038+0000", 2024-09-17T07:28:50.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2480000000000002e-06 2024-09-17T07:28:50.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304050+0000", 2024-09-17T07:28:50.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1756999999999999e-05 2024-09-17T07:28:50.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304058+0000", 2024-09-17T07:28:50.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7589999999999997e-06 2024-09-17T07:28:50.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304129+0000", 2024-09-17T07:28:50.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1318999999999999e-05 2024-09-17T07:28:50.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304149+0000", 2024-09-17T07:28:50.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0128000000000001e-05 2024-09-17T07:28:50.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304990+0000", 2024-09-17T07:28:50.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000840955 2024-09-17T07:28:50.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.468 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.468 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.305031+0000", 2024-09-17T07:28:50.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0170000000000003e-05 2024-09-17T07:28:50.468 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.468 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.468 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.468 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1159 3.0 e29/23 3:07b6f4cb:::rbd_data.1183350c2d82.0000000000000063:head v 29'5, mlcod=29'3)", 2024-09-17T07:28:50.468 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.450139+0000", 2024-09-17T07:28:50.468 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.941971271, 2024-09-17T07:28:50.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00092069299999999999, 2024-09-17T07:28:50.469 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.469 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.469 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.469 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450139+0000", 2024-09-17T07:28:50.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450139+0000", 2024-09-17T07:28:50.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450140+0000", 2024-09-17T07:28:50.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.097e-06 2024-09-17T07:28:50.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450157+0000", 2024-09-17T07:28:50.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7041000000000002e-05 2024-09-17T07:28:50.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450172+0000", 2024-09-17T07:28:50.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5315000000000002e-05 2024-09-17T07:28:50.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450183+0000", 2024-09-17T07:28:50.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0964e-05 2024-09-17T07:28:50.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450247+0000", 2024-09-17T07:28:50.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4348000000000001e-05 2024-09-17T07:28:50.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450271+0000", 2024-09-17T07:28:50.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3244e-05 2024-09-17T07:28:50.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.451016+0000", 2024-09-17T07:28:50.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00074592899999999999 2024-09-17T07:28:50.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.451059+0000", 2024-09-17T07:28:50.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2755e-05 2024-09-17T07:28:50.472 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.472 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.473 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.473 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1196 3.0 e29/23 3:016e7e32:::rbd_data.1183350c2d82.000000000000005f:head v 29'7, mlcod=29'5)", 2024-09-17T07:28:50.473 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.505232+0000", 2024-09-17T07:28:50.473 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.88687738299999996, 2024-09-17T07:28:50.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083826099999999998, 2024-09-17T07:28:50.473 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.473 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.473 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.473 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.505232+0000", 2024-09-17T07:28:50.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.474 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.474 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.505232+0000", 2024-09-17T07:28:50.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.474 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.474 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.505234+0000", 2024-09-17T07:28:50.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3710000000000001e-06 2024-09-17T07:28:50.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.505243+0000", 2024-09-17T07:28:50.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0450000000000006e-06 2024-09-17T07:28:50.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.505262+0000", 2024-09-17T07:28:50.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.908e-05 2024-09-17T07:28:50.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.505271+0000", 2024-09-17T07:28:50.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1160000000000005e-06 2024-09-17T07:28:50.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.505324+0000", 2024-09-17T07:28:50.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3442000000000001e-05 2024-09-17T07:28:50.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.505346+0000", 2024-09-17T07:28:50.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1984000000000002e-05 2024-09-17T07:28:50.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.506039+0000", 2024-09-17T07:28:50.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00069213300000000005 2024-09-17T07:28:50.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.506071+0000", 2024-09-17T07:28:50.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2089999999999999e-05 2024-09-17T07:28:50.477 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.477 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.477 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.477 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1200 3.9 e30/23 3:97508d79:::rbd_header.1183350c2d82:head v 30'71, mlcod=29'69)", 2024-09-17T07:28:50.477 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.100590+0000", 2024-09-17T07:28:50.477 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.29151992700000001, 2024-09-17T07:28:50.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00080794099999999995, 2024-09-17T07:28:50.478 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.478 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.478 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.478 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.100590+0000", 2024-09-17T07:28:50.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.100590+0000", 2024-09-17T07:28:50.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.100590+0000", 2024-09-17T07:28:50.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.05e-07 2024-09-17T07:28:50.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.100595+0000", 2024-09-17T07:28:50.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6450000000000004e-06 2024-09-17T07:28:50.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.100610+0000", 2024-09-17T07:28:50.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5074e-05 2024-09-17T07:28:50.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.100619+0000", 2024-09-17T07:28:50.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5829999999999996e-06 2024-09-17T07:28:50.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.100682+0000", 2024-09-17T07:28:50.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2607999999999994e-05 2024-09-17T07:28:50.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.100714+0000", 2024-09-17T07:28:50.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2415000000000001e-05 2024-09-17T07:28:50.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.101351+0000", 2024-09-17T07:28:50.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063634 2024-09-17T07:28:50.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.101398+0000", 2024-09-17T07:28:50.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7070999999999997e-05 2024-09-17T07:28:50.481 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.481 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.482 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1248 3.0 e30/23 3:04479396:::rbd_id.image_client.0-clone2:head v 30'8, mlcod=29'6)", 2024-09-17T07:28:50.482 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.134306+0000", 2024-09-17T07:28:50.482 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.25780407399999999, 2024-09-17T07:28:50.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00079602600000000003, 2024-09-17T07:28:50.482 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.482 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.482 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.134306+0000", 2024-09-17T07:28:50.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.134306+0000", 2024-09-17T07:28:50.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.134306+0000", 2024-09-17T07:28:50.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1100000000000002e-07 2024-09-17T07:28:50.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.134312+0000", 2024-09-17T07:28:50.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0349999999999998e-06 2024-09-17T07:28:50.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.134327+0000", 2024-09-17T07:28:50.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5235000000000001e-05 2024-09-17T07:28:50.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.134338+0000", 2024-09-17T07:28:50.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0722e-05 2024-09-17T07:28:50.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.134402+0000", 2024-09-17T07:28:50.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3956999999999998e-05 2024-09-17T07:28:50.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.134428+0000", 2024-09-17T07:28:50.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6160999999999999e-05 2024-09-17T07:28:50.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.135054+0000", 2024-09-17T07:28:50.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000625598 2024-09-17T07:28:50.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.135102+0000", 2024-09-17T07:28:50.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8007000000000003e-05 2024-09-17T07:28:50.486 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.486 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.486 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.486 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.486 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.486 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:50.487 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:50.393+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:50.488 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:28:50.488 INFO:teuthology.orchestra.run.smithi045.stdout:258 write 0xa17d69a thru 0xa183ca0 (0x6607 bytes) 2024-09-17T07:28:50.488 INFO:teuthology.orchestra.run.smithi045.stdout:259 write 0x37468d1 thru 0x374d02d (0x675d bytes) 2024-09-17T07:28:50.493 INFO:teuthology.orchestra.run.smithi045.stdout:260 write 0xd6a2cd1 thru 0xd6b0f8f (0xe2bf bytes) 2024-09-17T07:28:50.495 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:50.493+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:50.505 INFO:teuthology.orchestra.run.smithi045.stdout:262 trunc from 0xdea5658 to 0x8d4906b 2024-09-17T07:28:50.534 INFO:teuthology.orchestra.run.smithi045.stdout:266 punch from 0x6d2f941 to 0x6d3cb0c, (0xd1cb bytes) 2024-09-17T07:28:50.543 INFO:teuthology.orchestra.run.smithi045.stdout:267 write 0xc7a5a9f thru 0xc7b24da (0xca3c bytes) 2024-09-17T07:28:50.548 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:50.548 INFO:teuthology.orchestra.run.smithi045.stdout:269 read 0x4e656fe thru 0x4e6976d (0x4070 bytes) 2024-09-17T07:28:50.548 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:50.550 INFO:teuthology.orchestra.run.smithi045.stdout:272 trunc from 0xc7b24db to 0xe0cd47f 2024-09-17T07:28:50.554 INFO:teuthology.orchestra.run.smithi045.stdout:273 writesame 0x28c3c46 thru 0x28ce67d data_size 0x34(0xaa38 bytes) 2024-09-17T07:28:50.558 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:50.564 INFO:teuthology.orchestra.run.smithi045.stdout:275 trunc from 0xe0cd47f to 0x96102d6 2024-09-17T07:28:50.574 INFO:teuthology.orchestra.run.smithi045.stdout:276 read 0x1597640 thru 0x1599fcf (0x2990 bytes) 2024-09-17T07:28:50.576 INFO:teuthology.orchestra.run.smithi045.stdout:278 trunc from 0x96102d6 to 0x6e47e8e 2024-09-17T07:28:50.595 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:50.593+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:50.605 INFO:teuthology.orchestra.run.smithi045.stdout:279 punch from 0xece2bf to 0xed8310, (0xa051 bytes) 2024-09-17T07:28:50.605 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:50.605 INFO:teuthology.orchestra.run.smithi045.stdout:281 punch from 0x2ec77e3 to 0x2ed58bb, (0xe0d8 bytes) 2024-09-17T07:28:50.605 INFO:teuthology.orchestra.run.smithi045.stdout:truncating to largest ever: 0xed0bc3a 2024-09-17T07:28:50.614 INFO:teuthology.orchestra.run.smithi045.stdout:284 trunc from 0x6e47e8e to 0xed0bc3a 2024-09-17T07:28:50.614 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:50.614 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:50.614 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:28:50.614 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:50.619 INFO:teuthology.orchestra.run.smithi045.stdout:285 read 0x1d60b2b thru 0x1d6bdb1 (0xb287 bytes) 2024-09-17T07:28:50.624 INFO:teuthology.orchestra.run.smithi045.stdout:289 trunc from 0xed0bc3a to 0xa611834 2024-09-17T07:28:50.625 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:28:50.630 INFO:teuthology.orchestra.run.smithi045.stdout:290 trunc from 0xa611834 to 0xb135aaf 2024-09-17T07:28:50.636 INFO:teuthology.orchestra.run.smithi045.stdout:292 read 0x4693c85 thru 0x4696c9c (0x3018 bytes) 2024-09-17T07:28:50.636 INFO:teuthology.orchestra.run.smithi045.stdout:293 writesame 0x6651d63 thru 0x665cb6f data_size 0x1(0xae0d bytes) 2024-09-17T07:28:50.651 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:50.651 INFO:teuthology.orchestra.run.smithi045.stdout:295 punch from 0x7bcb8c6 to 0x7bce276, (0x29b0 bytes) 2024-09-17T07:28:50.657 INFO:teuthology.orchestra.run.smithi045.stdout:296 write 0xbaf9396 thru 0xbb0839a (0xf005 bytes) 2024-09-17T07:28:50.664 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:50.665 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:50.665 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:50.665 INFO:teuthology.orchestra.run.smithi045.stdout:300 write 0xb05ecf4 thru 0xb06d7e8 (0xeaf5 bytes) 2024-09-17T07:28:50.669 INFO:teuthology.orchestra.run.smithi045.stdout:301 punch from 0xa5ad703 to 0xa5bcdf6, (0xf6f3 bytes) 2024-09-17T07:28:50.669 INFO:teuthology.orchestra.run.smithi045.stdout:305 write 0x2c90030 thru 0x2c9c062 (0xc033 bytes) 2024-09-17T07:28:50.673 INFO:teuthology.orchestra.run.smithi045.stdout:306 writesame 0xb4a1555 thru 0xb4afd73 data_size 0xd(0xe81f bytes) 2024-09-17T07:28:50.680 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:50.689 INFO:teuthology.orchestra.run.smithi045.stdout:308 write 0xde2164a thru 0xde3095f (0xf316 bytes) 2024-09-17T07:28:50.696 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:50.696+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:50.704 INFO:teuthology.orchestra.run.smithi045.stdout:309 writesame 0xe3d1e67 thru 0xe3dbcd6 data_size 0xa9(0x9e70 bytes) 2024-09-17T07:28:50.709 INFO:teuthology.orchestra.run.smithi045.stdout:310 trunc from 0xe3dbcd7 to 0x9ce8e89 2024-09-17T07:28:50.725 INFO:teuthology.orchestra.run.smithi045.stdout:311 write 0x254908 thru 0x26266b (0xdd64 bytes) 2024-09-17T07:28:50.735 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:50.735 INFO:teuthology.orchestra.run.smithi045.stdout:313 write 0x7d02c7c thru 0x7d0ce6a (0xa1ef bytes) 2024-09-17T07:28:50.744 INFO:teuthology.orchestra.run.smithi045.stdout:314 punch from 0x6193775 to 0x6195a39, (0x22c4 bytes) 2024-09-17T07:28:50.745 INFO:teuthology.orchestra.run.smithi045.stdout:315 read 0x94e4fd3 thru 0x94e98c8 (0x48f6 bytes) 2024-09-17T07:28:50.745 INFO:teuthology.orchestra.run.smithi045.stdout:316 punch from 0x5f81c3a to 0x5f8e51d, (0xc8e3 bytes) 2024-09-17T07:28:50.745 INFO:teuthology.orchestra.run.smithi045.stdout:317 read 0x5cd9cb1 thru 0x5cdd4e0 (0x3830 bytes) 2024-09-17T07:28:50.745 INFO:teuthology.orchestra.run.smithi045.stdout:320 writesame 0x8e3f8d0 thru 0x8e4dc62 data_size 0x17(0xe393 bytes) 2024-09-17T07:28:50.748 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:28:50.748 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:28:50.748 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:50.760 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:28:50.761 INFO:teuthology.orchestra.run.smithi045.stdout:322 writesame 0xdb72c92 thru 0xdb7b67c data_size 0x3(0x89eb bytes) 2024-09-17T07:28:50.761 INFO:teuthology.orchestra.run.smithi045.stdout:323 writesame 0x21167ba thru 0x211a906 data_size 0x1(0x414d bytes) 2024-09-17T07:28:50.769 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:50.770 INFO:teuthology.orchestra.run.smithi045.stdout:327 write 0x14d8834 thru 0x14e0ac5 (0x8292 bytes) 2024-09-17T07:28:50.786 INFO:teuthology.orchestra.run.smithi045.stdout:328 trunc from 0xdb7b67d to 0x5d50683 2024-09-17T07:28:50.797 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:50.797+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:50.815 INFO:teuthology.orchestra.run.smithi045.stdout:333 trunc from 0x5d50683 to 0x73bc379 2024-09-17T07:28:50.820 INFO:teuthology.orchestra.run.smithi045.stdout:335 write 0x4ac29df thru 0x4ac3436 (0xa58 bytes) 2024-09-17T07:28:50.823 INFO:teuthology.orchestra.run.smithi045.stdout:336 read 0x54b096 thru 0x556e81 (0xbdec bytes) 2024-09-17T07:28:50.825 INFO:teuthology.orchestra.run.smithi045.stdout:338 trunc from 0x73bc379 to 0x5e36d22 2024-09-17T07:28:50.832 INFO:teuthology.orchestra.run.smithi045.stdout:339 punch from 0x3fbceab to 0x3fbcef0, (0x45 bytes) 2024-09-17T07:28:50.835 INFO:teuthology.orchestra.run.smithi045.stdout:340 trunc from 0x5e36d22 to 0x38238ee 2024-09-17T07:28:50.855 INFO:teuthology.orchestra.run.smithi045.stdout:341 punch from 0x215c07 to 0x224d33, (0xf12c bytes) 2024-09-17T07:28:50.855 INFO:teuthology.orchestra.run.smithi045.stdout:342 write 0x11ec35 thru 0x12c95c (0xdd28 bytes) 2024-09-17T07:28:50.864 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:50.868 INFO:teuthology.orchestra.run.smithi045.stdout:344 trunc from 0x38238ee to 0x6df969d 2024-09-17T07:28:50.875 INFO:teuthology.orchestra.run.smithi045.stdout:345 write 0x3f5ffd9 thru 0x3f674e1 (0x7509 bytes) 2024-09-17T07:28:50.880 INFO:teuthology.orchestra.run.smithi045.stdout:346 read 0x1320be6 thru 0x13253bb (0x47d6 bytes) 2024-09-17T07:28:50.889 INFO:teuthology.orchestra.run.smithi045.stdout:347 trunc from 0x6df969d to 0xe5606bf 2024-09-17T07:28:50.892 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:28:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:28:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:28:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:869 3.2 3:449b730c:::rbd_data.1183350c2d82.000000000000004d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.069352+0000", 2024-09-17T07:28:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.821307837, 2024-09-17T07:28:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036811930000000001, 2024-09-17T07:28:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 869 2024-09-17T07:28:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069352+0000", 2024-09-17T07:28:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069352+0000", 2024-09-17T07:28:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069352+0000", 2024-09-17T07:28:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6599999999999996e-07 2024-09-17T07:28:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069356+0000", 2024-09-17T07:28:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:28:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069367+0000", 2024-09-17T07:28:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1635e-05 2024-09-17T07:28:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069374+0000", 2024-09-17T07:28:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8399999999999997e-06 2024-09-17T07:28:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069432+0000", 2024-09-17T07:28:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8100000000000003e-05 2024-09-17T07:28:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069519+0000", 2024-09-17T07:28:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6459999999999996e-05 2024-09-17T07:28:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:28:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069627+0000", 2024-09-17T07:28:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010812299999999999 2024-09-17T07:28:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070811+0000", 2024-09-17T07:28:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011842179999999999 2024-09-17T07:28:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.072931+0000", 2024-09-17T07:28:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021195879999999999 2024-09-17T07:28:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.072988+0000", 2024-09-17T07:28:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7015000000000002e-05 2024-09-17T07:28:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073033+0000", 2024-09-17T07:28:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5380000000000003e-05 2024-09-17T07:28:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:874 3.2 3:42e15a00:::rbd_data.1183350c2d82.0000000000000082:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.069393+0000", 2024-09-17T07:28:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.821266139, 2024-09-17T07:28:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0063855170000000003, 2024-09-17T07:28:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 874 2024-09-17T07:28:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069393+0000", 2024-09-17T07:28:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069393+0000", 2024-09-17T07:28:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069394+0000", 2024-09-17T07:28:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.75e-07 2024-09-17T07:28:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069397+0000", 2024-09-17T07:28:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7020000000000001e-06 2024-09-17T07:28:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069405+0000", 2024-09-17T07:28:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.763e-06 2024-09-17T07:28:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069410+0000", 2024-09-17T07:28:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6620000000000002e-06 2024-09-17T07:28:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069903+0000", 2024-09-17T07:28:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00049267500000000004 2024-09-17T07:28:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069970+0000", 2024-09-17T07:28:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7469000000000001e-05 2024-09-17T07:28:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:28:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070074+0000", 2024-09-17T07:28:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000103305 2024-09-17T07:28:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.071385+0000", 2024-09-17T07:28:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013110050000000001 2024-09-17T07:28:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075665+0000", 2024-09-17T07:28:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0042803659999999999 2024-09-17T07:28:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075697+0000", 2024-09-17T07:28:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2181999999999998e-05 2024-09-17T07:28:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075779+0000", 2024-09-17T07:28:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1812999999999997e-05 2024-09-17T07:28:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:878 3.7 3:e4893214:::rbd_data.1183350c2d82.00000000000000b1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.070655+0000", 2024-09-17T07:28:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.820004387, 2024-09-17T07:28:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0052679559999999999, 2024-09-17T07:28:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 878 2024-09-17T07:28:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070655+0000", 2024-09-17T07:28:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070655+0000", 2024-09-17T07:28:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070656+0000", 2024-09-17T07:28:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7300000000000004e-07 2024-09-17T07:28:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070662+0000", 2024-09-17T07:28:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2989999999999996e-06 2024-09-17T07:28:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070676+0000", 2024-09-17T07:28:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4272e-05 2024-09-17T07:28:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070684+0000", 2024-09-17T07:28:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0269999999999995e-06 2024-09-17T07:28:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070799+0000", 2024-09-17T07:28:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000115123 2024-09-17T07:28:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070887+0000", 2024-09-17T07:28:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8426999999999999e-05 2024-09-17T07:28:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:28:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.071007+0000", 2024-09-17T07:28:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119677 2024-09-17T07:28:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.071596+0000", 2024-09-17T07:28:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00058881000000000005 2024-09-17T07:28:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075826+0000", 2024-09-17T07:28:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0042297740000000004 2024-09-17T07:28:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075879+0000", 2024-09-17T07:28:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3350999999999998e-05 2024-09-17T07:28:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075923+0000", 2024-09-17T07:28:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4223e-05 2024-09-17T07:28:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:914 3.2 3:422081bc:::rbd_data.1183350c2d82.0000000000000057:head [set-alloc-hint object_size 1048576 write_size 1048576,write 884736~12641] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.111264+0000", 2024-09-17T07:28:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.77939523, 2024-09-17T07:28:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0030512999999999998, 2024-09-17T07:28:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 914 2024-09-17T07:28:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.111264+0000", 2024-09-17T07:28:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.111264+0000", 2024-09-17T07:28:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.111267+0000", 2024-09-17T07:28:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9210000000000001e-06 2024-09-17T07:28:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.111294+0000", 2024-09-17T07:28:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6787e-05 2024-09-17T07:28:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.111323+0000", 2024-09-17T07:28:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8673000000000001e-05 2024-09-17T07:28:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.111337+0000", 2024-09-17T07:28:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4266e-05 2024-09-17T07:28:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.111424+0000", 2024-09-17T07:28:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7206000000000005e-05 2024-09-17T07:28:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:28:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.111431+0000", 2024-09-17T07:28:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6959999999999999e-06 2024-09-17T07:28:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.113148+0000", 2024-09-17T07:28:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017172489999999999 2024-09-17T07:28:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.113200+0000", 2024-09-17T07:28:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1666000000000003e-05 2024-09-17T07:28:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:28:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.113291+0000", 2024-09-17T07:28:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0917999999999999e-05 2024-09-17T07:28:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.113965+0000", 2024-09-17T07:28:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00067429299999999996 2024-09-17T07:28:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.114221+0000", 2024-09-17T07:28:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00025590699999999999 2024-09-17T07:28:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.114277+0000", 2024-09-17T07:28:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5556999999999997e-05 2024-09-17T07:28:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.114316+0000", 2024-09-17T07:28:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9161e-05 2024-09-17T07:28:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:946 3.2 3:4462f3bd:::rbd_data.1183350c2d82.0000000000000023:head [set-alloc-hint object_size 1048576 write_size 1048576,write 327680~23522] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.161140+0000", 2024-09-17T07:28:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.7295194679999999, 2024-09-17T07:28:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027547180000000002, 2024-09-17T07:28:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 946 2024-09-17T07:28:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.161140+0000", 2024-09-17T07:28:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.161140+0000", 2024-09-17T07:28:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.161142+0000", 2024-09-17T07:28:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6950000000000001e-06 2024-09-17T07:28:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.161170+0000", 2024-09-17T07:28:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8200000000000001e-05 2024-09-17T07:28:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.161187+0000", 2024-09-17T07:28:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6779999999999999e-05 2024-09-17T07:28:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.161196+0000", 2024-09-17T07:28:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2399999999999996e-06 2024-09-17T07:28:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.161253+0000", 2024-09-17T07:28:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7249e-05 2024-09-17T07:28:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.161349+0000", 2024-09-17T07:28:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5812999999999999e-05 2024-09-17T07:28:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:28:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.161447+0000", 2024-09-17T07:28:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8021000000000002e-05 2024-09-17T07:28:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.162059+0000", 2024-09-17T07:28:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061149399999999997 2024-09-17T07:28:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.163821+0000", 2024-09-17T07:28:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001762364 2024-09-17T07:28:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.163859+0000", 2024-09-17T07:28:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7978999999999997e-05 2024-09-17T07:28:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.163895+0000", 2024-09-17T07:28:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5883000000000002e-05 2024-09-17T07:28:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:984 3.7 3:e4777bd3:::rbd_data.1183350c2d82.0000000000000012:head [set-alloc-hint object_size 1048576 write_size 1048576,write 753664~17620] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.230589+0000", 2024-09-17T07:28:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.660070406, 2024-09-17T07:28:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028799170000000001, 2024-09-17T07:28:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 984 2024-09-17T07:28:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.230589+0000", 2024-09-17T07:28:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.230589+0000", 2024-09-17T07:28:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.230591+0000", 2024-09-17T07:28:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.905e-06 2024-09-17T07:28:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.230614+0000", 2024-09-17T07:28:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2705e-05 2024-09-17T07:28:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.230632+0000", 2024-09-17T07:28:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8284e-05 2024-09-17T07:28:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.230643+0000", 2024-09-17T07:28:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.04e-05 2024-09-17T07:28:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.230714+0000", 2024-09-17T07:28:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1132999999999998e-05 2024-09-17T07:28:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.230813+0000", 2024-09-17T07:28:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9239000000000003e-05 2024-09-17T07:28:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:28:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.230915+0000", 2024-09-17T07:28:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000102259 2024-09-17T07:28:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.231674+0000", 2024-09-17T07:28:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000758641 2024-09-17T07:28:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.233362+0000", 2024-09-17T07:28:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001688124 2024-09-17T07:28:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.233414+0000", 2024-09-17T07:28:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2045000000000001e-05 2024-09-17T07:28:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.233469+0000", 2024-09-17T07:28:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5182000000000001e-05 2024-09-17T07:28:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1052 3.7 3:e4777bd3:::rbd_data.1183350c2d82.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.303247+0000", 2024-09-17T07:28:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.5874128939999999, 2024-09-17T07:28:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003696857, 2024-09-17T07:28:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1052 2024-09-17T07:28:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303247+0000", 2024-09-17T07:28:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303247+0000", 2024-09-17T07:28:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303250+0000", 2024-09-17T07:28:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8270000000000002e-06 2024-09-17T07:28:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303259+0000", 2024-09-17T07:28:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3610000000000007e-06 2024-09-17T07:28:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303282+0000", 2024-09-17T07:28:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3190999999999999e-05 2024-09-17T07:28:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303293+0000", 2024-09-17T07:28:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0641e-05 2024-09-17T07:28:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303374+0000", 2024-09-17T07:28:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1137999999999995e-05 2024-09-17T07:28:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303476+0000", 2024-09-17T07:28:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010225699999999999 2024-09-17T07:28:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:28:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303612+0000", 2024-09-17T07:28:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013634500000000001 2024-09-17T07:28:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304357+0000", 2024-09-17T07:28:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00074472399999999999 2024-09-17T07:28:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.306886+0000", 2024-09-17T07:28:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025291519999999998 2024-09-17T07:28:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.306916+0000", 2024-09-17T07:28:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0077e-05 2024-09-17T07:28:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.306944+0000", 2024-09-17T07:28:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7144e-05 2024-09-17T07:28:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1109 3.1e 3:78e7e4bb:::rbd_data.1183350c2d82.000000000000002a:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.357457+0000", 2024-09-17T07:28:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.533202212, 2024-09-17T07:28:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.015696132000000002, 2024-09-17T07:28:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1109 2024-09-17T07:28:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357457+0000", 2024-09-17T07:28:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357457+0000", 2024-09-17T07:28:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357459+0000", 2024-09-17T07:28:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8810000000000001e-06 2024-09-17T07:28:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357467+0000", 2024-09-17T07:28:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3969999999999997e-06 2024-09-17T07:28:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357485+0000", 2024-09-17T07:28:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7898e-05 2024-09-17T07:28:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357494+0000", 2024-09-17T07:28:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3000000000000007e-06 2024-09-17T07:28:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357552+0000", 2024-09-17T07:28:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8044000000000002e-05 2024-09-17T07:28:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357657+0000", 2024-09-17T07:28:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105209 2024-09-17T07:28:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-17T07:28:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.360368+0000", 2024-09-17T07:28:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00271119 2024-09-17T07:28:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.373026+0000", 2024-09-17T07:28:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.012657677000000001 2024-09-17T07:28:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.373076+0000", 2024-09-17T07:28:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0003999999999998e-05 2024-09-17T07:28:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.373115+0000", 2024-09-17T07:28:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9489000000000001e-05 2024-09-17T07:28:50.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.945 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.945 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.373154+0000", 2024-09-17T07:28:50.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8043000000000002e-05 2024-09-17T07:28:50.945 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.945 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.945 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.945 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop_reply(client.4483.0:1109 3.1e e29/23 ondisk, result = 0)", 2024-09-17T07:28:50.945 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.367839+0000", 2024-09-17T07:28:50.945 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.522820123, 2024-09-17T07:28:50.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0053228360000000001, 2024-09-17T07:28:50.946 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.946 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.946 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:50.946 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.367839+0000", 2024-09-17T07:28:50.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.367839+0000", 2024-09-17T07:28:50.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.367841+0000", 2024-09-17T07:28:50.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.319e-06 2024-09-17T07:28:50.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.367846+0000", 2024-09-17T07:28:50.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4149999999999998e-06 2024-09-17T07:28:50.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.367863+0000", 2024-09-17T07:28:50.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6359e-05 2024-09-17T07:28:50.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.367877+0000", 2024-09-17T07:28:50.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4691e-05 2024-09-17T07:28:50.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.373064+0000", 2024-09-17T07:28:50.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0051870120000000004 2024-09-17T07:28:50.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.373072+0000", 2024-09-17T07:28:50.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5549999999999997e-06 2024-09-17T07:28:50.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.373162+0000", 2024-09-17T07:28:50.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0484999999999998e-05 2024-09-17T07:28:50.949 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.949 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.949 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.950 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1158 3.7 3:e1042ee7:::rbd_data.1183350c2d82.0000000000000062:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:50.950 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.445922+0000", 2024-09-17T07:28:50.950 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.444737283, 2024-09-17T07:28:50.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0085511760000000006, 2024-09-17T07:28:50.950 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.950 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.950 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.950 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.950 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1158 2024-09-17T07:28:50.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.950 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.445922+0000", 2024-09-17T07:28:50.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.445922+0000", 2024-09-17T07:28:50.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.445925+0000", 2024-09-17T07:28:50.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6970000000000002e-06 2024-09-17T07:28:50.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.445938+0000", 2024-09-17T07:28:50.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2907e-05 2024-09-17T07:28:50.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.445969+0000", 2024-09-17T07:28:50.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0815999999999998e-05 2024-09-17T07:28:50.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.445989+0000", 2024-09-17T07:28:50.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0205000000000001e-05 2024-09-17T07:28:50.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.446052+0000", 2024-09-17T07:28:50.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2578999999999998e-05 2024-09-17T07:28:50.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.446105+0000", 2024-09-17T07:28:50.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3947999999999999e-05 2024-09-17T07:28:50.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:28:50.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.447592+0000", 2024-09-17T07:28:50.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001486235 2024-09-17T07:28:50.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:50.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.454400+0000", 2024-09-17T07:28:50.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0068086730000000003 2024-09-17T07:28:50.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:50.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.454411+0000", 2024-09-17T07:28:50.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0166999999999999e-05 2024-09-17T07:28:50.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.454444+0000", 2024-09-17T07:28:50.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3547000000000003e-05 2024-09-17T07:28:50.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.454474+0000", 2024-09-17T07:28:50.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9402000000000001e-05 2024-09-17T07:28:50.955 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.955 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.955 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.956 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop_reply(client.4483.0:1158 3.7 e29/23 ondisk, result = 0)", 2024-09-17T07:28:50.956 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.450626+0000", 2024-09-17T07:28:50.956 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.440033396, 2024-09-17T07:28:50.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003779223, 2024-09-17T07:28:50.956 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.956 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.956 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:50.956 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.956 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450626+0000", 2024-09-17T07:28:50.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450626+0000", 2024-09-17T07:28:50.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450627+0000", 2024-09-17T07:28:50.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8100000000000002e-07 2024-09-17T07:28:50.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450631+0000", 2024-09-17T07:28:50.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.941e-06 2024-09-17T07:28:50.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450646+0000", 2024-09-17T07:28:50.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5293000000000001e-05 2024-09-17T07:28:50.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450654+0000", 2024-09-17T07:28:50.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7549999999999994e-06 2024-09-17T07:28:50.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.454390+0000", 2024-09-17T07:28:50.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037361959999999998 2024-09-17T07:28:50.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.454397+0000", 2024-09-17T07:28:50.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5130000000000002e-06 2024-09-17T07:28:50.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.454405+0000", 2024-09-17T07:28:50.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7439999999999996e-06 2024-09-17T07:28:50.960 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.960 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.960 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.960 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1181 3.8 3:1089ba96:::rbd_data.1183350c2d82.00000000000000e2:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:50.960 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.475326+0000", 2024-09-17T07:28:50.960 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.415333403, 2024-09-17T07:28:50.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.014074893, 2024-09-17T07:28:50.960 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.960 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.960 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.961 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.961 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.961 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1181 2024-09-17T07:28:50.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.961 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475326+0000", 2024-09-17T07:28:50.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475326+0000", 2024-09-17T07:28:50.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475327+0000", 2024-09-17T07:28:50.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9299999999999996e-07 2024-09-17T07:28:50.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475331+0000", 2024-09-17T07:28:50.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3189999999999996e-06 2024-09-17T07:28:50.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475343+0000", 2024-09-17T07:28:50.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1076e-05 2024-09-17T07:28:50.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475350+0000", 2024-09-17T07:28:50.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3549999999999999e-06 2024-09-17T07:28:50.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475403+0000", 2024-09-17T07:28:50.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3402999999999997e-05 2024-09-17T07:28:50.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475490+0000", 2024-09-17T07:28:50.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6638999999999995e-05 2024-09-17T07:28:50.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:28:50.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.477756+0000", 2024-09-17T07:28:50.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022665179999999999 2024-09-17T07:28:50.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:50.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.489269+0000", 2024-09-17T07:28:50.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011512872 2024-09-17T07:28:50.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:50.965 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.489291+0000", 2024-09-17T07:28:50.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1574999999999999e-05 2024-09-17T07:28:50.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.965 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.489349+0000", 2024-09-17T07:28:50.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8304000000000003e-05 2024-09-17T07:28:50.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.965 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.489401+0000", 2024-09-17T07:28:50.966 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1938999999999999e-05 2024-09-17T07:28:50.966 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.966 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.966 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.966 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1591 3.1e 3:7cba8a19:::rbd_data.11832b928878.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 103318~27754] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:50.966 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.660750+0000", 2024-09-17T07:28:50.966 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.22990939699999999, 2024-09-17T07:28:50.966 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0042241600000000002, 2024-09-17T07:28:50.966 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.966 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.966 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.967 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.967 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.967 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1591 2024-09-17T07:28:50.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.967 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.967 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660750+0000", 2024-09-17T07:28:50.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660750+0000", 2024-09-17T07:28:50.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660752+0000", 2024-09-17T07:28:50.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.204e-06 2024-09-17T07:28:50.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660790+0000", 2024-09-17T07:28:50.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7953000000000001e-05 2024-09-17T07:28:50.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660807+0000", 2024-09-17T07:28:50.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6719e-05 2024-09-17T07:28:50.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660814+0000", 2024-09-17T07:28:50.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1230000000000002e-06 2024-09-17T07:28:50.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660866+0000", 2024-09-17T07:28:50.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1307000000000002e-05 2024-09-17T07:28:50.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660951+0000", 2024-09-17T07:28:50.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5558999999999999e-05 2024-09-17T07:28:50.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-17T07:28:50.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.661049+0000", 2024-09-17T07:28:50.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7912999999999996e-05 2024-09-17T07:28:50.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:50.971 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.661729+0000", 2024-09-17T07:28:50.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00068000999999999999 2024-09-17T07:28:50.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:50.971 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.664912+0000", 2024-09-17T07:28:50.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031832560000000002 2024-09-17T07:28:50.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.971 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.664946+0000", 2024-09-17T07:28:50.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3253999999999997e-05 2024-09-17T07:28:50.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.972 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.664974+0000", 2024-09-17T07:28:50.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8861999999999999e-05 2024-09-17T07:28:50.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.972 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.972 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1739 3.b 3:d23e8048:::rbd_data.11832b928878.00000000000001b6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:50.972 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.788878+0000", 2024-09-17T07:28:50.972 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.101782014, 2024-09-17T07:28:50.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031386080000000002, 2024-09-17T07:28:50.972 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.972 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.973 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.973 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.973 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.973 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1739 2024-09-17T07:28:50.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.973 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.973 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.973 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.788878+0000", 2024-09-17T07:28:50.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.788878+0000", 2024-09-17T07:28:50.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.788880+0000", 2024-09-17T07:28:50.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9240000000000001e-06 2024-09-17T07:28:50.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.788890+0000", 2024-09-17T07:28:50.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0737000000000001e-05 2024-09-17T07:28:50.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.788909+0000", 2024-09-17T07:28:50.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.855e-05 2024-09-17T07:28:50.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.788919+0000", 2024-09-17T07:28:50.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0022e-05 2024-09-17T07:28:50.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.788980+0000", 2024-09-17T07:28:50.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0887000000000002e-05 2024-09-17T07:28:50.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.789080+0000", 2024-09-17T07:28:50.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000100331 2024-09-17T07:28:50.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:50.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.789218+0000", 2024-09-17T07:28:50.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013763499999999999 2024-09-17T07:28:50.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:50.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.789892+0000", 2024-09-17T07:28:50.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00067405399999999999 2024-09-17T07:28:50.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:50.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791937+0000", 2024-09-17T07:28:50.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020456210000000001 2024-09-17T07:28:50.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791988+0000", 2024-09-17T07:28:50.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1128999999999998e-05 2024-09-17T07:28:50.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.792016+0000", 2024-09-17T07:28:50.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7718000000000001e-05 2024-09-17T07:28:50.978 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.978 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.978 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.978 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1741 3.18 3:1eec3115:::rbd_data.11832b928878.00000000000001b8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:50.978 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.788939+0000", 2024-09-17T07:28:50.978 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.101720192, 2024-09-17T07:28:50.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031266470000000002, 2024-09-17T07:28:50.978 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.978 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:50.979 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.979 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.979 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.979 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1741 2024-09-17T07:28:50.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.979 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.788939+0000", 2024-09-17T07:28:50.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.788939+0000", 2024-09-17T07:28:50.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.788940+0000", 2024-09-17T07:28:50.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.89e-07 2024-09-17T07:28:50.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.788943+0000", 2024-09-17T07:28:50.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0649999999999999e-06 2024-09-17T07:28:50.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.788950+0000", 2024-09-17T07:28:50.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0430000000000002e-06 2024-09-17T07:28:50.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.788956+0000", 2024-09-17T07:28:50.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5729999999999998e-06 2024-09-17T07:28:50.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.789046+0000", 2024-09-17T07:28:50.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9957999999999995e-05 2024-09-17T07:28:50.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.789110+0000", 2024-09-17T07:28:50.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4938999999999996e-05 2024-09-17T07:28:50.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:50.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.789218+0000", 2024-09-17T07:28:50.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000107198 2024-09-17T07:28:50.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:50.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.790412+0000", 2024-09-17T07:28:50.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001194105 2024-09-17T07:28:50.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:50.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791997+0000", 2024-09-17T07:28:50.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015855769999999999 2024-09-17T07:28:50.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:50.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.792041+0000", 2024-09-17T07:28:50.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3881999999999998e-05 2024-09-17T07:28:50.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.984 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.792066+0000", 2024-09-17T07:28:50.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4817999999999999e-05 2024-09-17T07:28:50.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.984 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.984 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1796 3.2 3:45c0a2cc:::rbd_data.1183350c2d82.000000000000005e:4 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:50.984 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.795982+0000", 2024-09-17T07:28:50.984 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.094677688999999995, 2024-09-17T07:28:50.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0026707089999999998, 2024-09-17T07:28:50.984 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.984 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.985 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:50.985 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.985 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.985 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.985 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1796 2024-09-17T07:28:50.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.985 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.795982+0000", 2024-09-17T07:28:50.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.795982+0000", 2024-09-17T07:28:50.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.795983+0000", 2024-09-17T07:28:50.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0500000000000003e-07 2024-09-17T07:28:50.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.795986+0000", 2024-09-17T07:28:50.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.416e-06 2024-09-17T07:28:50.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.795998+0000", 2024-09-17T07:28:50.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2129e-05 2024-09-17T07:28:50.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796004+0000", 2024-09-17T07:28:50.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2879999999999998e-06 2024-09-17T07:28:50.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.798426+0000", 2024-09-17T07:28:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024217980000000002 2024-09-17T07:28:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.798511+0000", 2024-09-17T07:28:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5078000000000001e-05 2024-09-17T07:28:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.798653+0000", 2024-09-17T07:28:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014129499999999999 2024-09-17T07:28:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.988 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1797 3.2 3:45c0a2cc:::rbd_data.1183350c2d82.000000000000005e:4 [read 589824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.796015+0000", 2024-09-17T07:28:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.094644925000000005, 2024-09-17T07:28:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028894799999999998, 2024-09-17T07:28:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1797 2024-09-17T07:28:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.990 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796015+0000", 2024-09-17T07:28:50.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796015+0000", 2024-09-17T07:28:50.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796015+0000", 2024-09-17T07:28:50.991 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5400000000000002e-07 2024-09-17T07:28:50.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.991 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.991 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796018+0000", 2024-09-17T07:28:50.991 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.503e-06 2024-09-17T07:28:50.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.991 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.991 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796027+0000", 2024-09-17T07:28:50.991 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3670000000000003e-06 2024-09-17T07:28:50.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796033+0000", 2024-09-17T07:28:50.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4990000000000002e-06 2024-09-17T07:28:50.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.798709+0000", 2024-09-17T07:28:50.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0026766720000000002 2024-09-17T07:28:50.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.798797+0000", 2024-09-17T07:28:50.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7529999999999997e-05 2024-09-17T07:28:50.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.993 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.798904+0000", 2024-09-17T07:28:50.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010745499999999999 2024-09-17T07:28:50.993 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.993 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.993 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.993 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1803 3.2 3:45c0a2cc:::rbd_data.1183350c2d82.000000000000005e:4 [read 294912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:50.993 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.796066+0000", 2024-09-17T07:28:50.993 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.094593956000000007, 2024-09-17T07:28:50.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0030718540000000002, 2024-09-17T07:28:50.993 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.994 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.994 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:50.994 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.994 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.994 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.994 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1803 2024-09-17T07:28:50.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.994 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796066+0000", 2024-09-17T07:28:50.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796066+0000", 2024-09-17T07:28:50.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:50.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796066+0000", 2024-09-17T07:28:50.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5300000000000003e-07 2024-09-17T07:28:50.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:50.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796070+0000", 2024-09-17T07:28:50.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3110000000000001e-06 2024-09-17T07:28:50.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:50.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796078+0000", 2024-09-17T07:28:50.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3329999999999997e-06 2024-09-17T07:28:50.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:50.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796084+0000", 2024-09-17T07:28:50.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6300000000000003e-06 2024-09-17T07:28:50.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.997 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:50.997 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.798987+0000", 2024-09-17T07:28:50.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0029037720000000002 2024-09-17T07:28:50.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.997 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:50.997 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.799050+0000", 2024-09-17T07:28:50.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3003000000000006e-05 2024-09-17T07:28:50.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.997 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:50.997 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.799138+0000", 2024-09-17T07:28:50.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7052000000000006e-05 2024-09-17T07:28:50.997 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.998 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:50.998 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:50.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.998 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1804 3.2 3:45c0a2cc:::rbd_data.1183350c2d82.000000000000005e:4 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:50.998 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.796094+0000", 2024-09-17T07:28:50.998 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.094566070000000002, 2024-09-17T07:28:50.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0032921600000000001, 2024-09-17T07:28:50.998 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:50.998 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:50.998 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:50.998 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:50.998 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:50.998 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:50.999 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1804 2024-09-17T07:28:50.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.999 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:50.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:50.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796094+0000", 2024-09-17T07:28:50.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:50.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:50.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:50.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796094+0000", 2024-09-17T07:28:50.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:50.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:51.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796094+0000", 2024-09-17T07:28:51.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1099999999999996e-07 2024-09-17T07:28:51.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:51.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796097+0000", 2024-09-17T07:28:51.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:28:51.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:51.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796103+0000", 2024-09-17T07:28:51.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6669999999999999e-06 2024-09-17T07:28:51.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:51.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796108+0000", 2024-09-17T07:28:51.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:28:51.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:51.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.799186+0000", 2024-09-17T07:28:51.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00307764 2024-09-17T07:28:51.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.799271+0000", 2024-09-17T07:28:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5445999999999995e-05 2024-09-17T07:28:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.799386+0000", 2024-09-17T07:28:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011461 2024-09-17T07:28:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1805 3.2 3:45c0a2cc:::rbd_data.1183350c2d82.000000000000005e:4 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.796116+0000", 2024-09-17T07:28:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.094543903999999998, 2024-09-17T07:28:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0034149010000000001, 2024-09-17T07:28:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1805 2024-09-17T07:28:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:51.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796116+0000", 2024-09-17T07:28:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796116+0000", 2024-09-17T07:28:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796116+0000", 2024-09-17T07:28:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3799999999999998e-07 2024-09-17T07:28:51.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796118+0000", 2024-09-17T07:28:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:28:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796125+0000", 2024-09-17T07:28:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6069999999999996e-06 2024-09-17T07:28:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796130+0000", 2024-09-17T07:28:51.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.22e-06 2024-09-17T07:28:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.799416+0000", 2024-09-17T07:28:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0032860229999999999 2024-09-17T07:28:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.799461+0000", 2024-09-17T07:28:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5120000000000002e-05 2024-09-17T07:28:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:51.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.799531+0000", 2024-09-17T07:28:51.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9307000000000006e-05 2024-09-17T07:28:51.007 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:51.007 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:51.007 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:51.007 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:51.007 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:51.007 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:51.008 INFO:teuthology.orchestra.run.smithi045.stdout:348 read 0x9ab49bb thru 0x9abb275 (0x68bb bytes) 2024-09-17T07:28:51.008 INFO:teuthology.orchestra.run.smithi045.stdout:349 write 0xd2b1b49 thru 0xd2b84a9 (0x6961 bytes) 2024-09-17T07:28:51.008 INFO:teuthology.orchestra.run.smithi045.stdout:353 punch from 0x9ede2f9 to 0x9ee8639, (0xa340 bytes) 2024-09-17T07:28:51.008 INFO:teuthology.orchestra.run.smithi045.stdout:354 write 0x1395567 thru 0x13977c3 (0x225d bytes) 2024-09-17T07:28:51.009 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:28:51.012 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:51.012 INFO:teuthology.orchestra.run.smithi045.stdout:356 write 0xf85418 thru 0xf91df2 (0xc9db bytes) 2024-09-17T07:28:51.012 INFO:teuthology.orchestra.run.smithi045.stdout:357 read 0x729facf thru 0x729fcb1 (0x1e3 bytes) 2024-09-17T07:28:51.012 INFO:teuthology.orchestra.run.smithi045.stdout:362 write 0xc97dac4 thru 0xc9855c0 (0x7afd bytes) 2024-09-17T07:28:51.012 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:51.013 INFO:teuthology.orchestra.run.smithi045.stdout:364 write 0x1c81841 thru 0x1c8e577 (0xcd37 bytes) 2024-09-17T07:28:51.013 INFO:teuthology.orchestra.run.smithi045.stdout:365 read 0x81b1564 thru 0x81bc112 (0xabaf bytes) 2024-09-17T07:28:51.013 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:51.013 INFO:teuthology.orchestra.run.smithi045.stdout:368 read 0xa608c2d thru 0xa60f3dd (0x67b1 bytes) 2024-09-17T07:28:51.013 INFO:teuthology.orchestra.run.smithi045.stdout:371 punch from 0x78c6a2e to 0x78ce7df, (0x7db1 bytes) 2024-09-17T07:28:51.013 INFO:teuthology.orchestra.run.smithi045.stdout:372 writesame 0x7f91616 thru 0x7f94a2d data_size 0x4(0x3418 bytes) 2024-09-17T07:28:51.013 INFO:teuthology.orchestra.run.smithi045.stdout:375 writesame 0x6123777 thru 0x612c115 data_size 0x7(0x899f bytes) 2024-09-17T07:28:51.013 INFO:teuthology.orchestra.run.smithi045.stdout:376 punch from 0x8c386a9 to 0x8c4241d, (0x9d74 bytes) 2024-09-17T07:28:51.013 INFO:teuthology.orchestra.run.smithi045.stdout:377 write 0x3c042b3 thru 0x3c0bf6c (0x7cba bytes) 2024-09-17T07:28:51.013 INFO:teuthology.orchestra.run.smithi045.stdout:378 write 0xc4ba7ca thru 0xc4c4f6f (0xa7a6 bytes) 2024-09-17T07:28:51.013 INFO:teuthology.orchestra.run.smithi045.stdout:379 punch from 0x3ee7957 to 0x3eec1aa, (0x4853 bytes) 2024-09-17T07:28:51.013 INFO:teuthology.orchestra.run.smithi045.stdout:380 read 0x480421b thru 0x480a868 (0x664e bytes) 2024-09-17T07:28:51.013 INFO:teuthology.orchestra.run.smithi045.stdout:381 write 0x31ab176 thru 0x31b22b9 (0x7144 bytes) 2024-09-17T07:28:51.013 INFO:teuthology.orchestra.run.smithi045.stdout:382 writesame 0xbc084a thru 0xbcd49c data_size 0x1(0xcc53 bytes) 2024-09-17T07:28:51.014 INFO:teuthology.orchestra.run.smithi045.stdout:383 read 0x12cf01e thru 0x12da4ad (0xb490 bytes) 2024-09-17T07:28:51.014 INFO:teuthology.orchestra.run.smithi045.stdout:385 read 0x76e605 thru 0x7796be (0xb0ba bytes) 2024-09-17T07:28:51.014 INFO:teuthology.orchestra.run.smithi045.stdout:387 write 0x901ab6 thru 0x9083c2 (0x690d bytes) 2024-09-17T07:28:51.014 INFO:teuthology.orchestra.run.smithi045.stdout:388 writesame 0x1ac7b73 thru 0x1ac8b18 data_size 0x1(0xfa6 bytes) 2024-09-17T07:28:51.014 INFO:teuthology.orchestra.run.smithi045.stdout:389 trunc from 0xe5606bf to 0x3962be9 2024-09-17T07:28:51.014 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:51.014 INFO:teuthology.orchestra.run.smithi045.stdout:395 writesame 0x1f8185d thru 0x1f8bea6 data_size 0x2d(0xa64a bytes) 2024-09-17T07:28:51.014 INFO:teuthology.orchestra.run.smithi045.stdout:397 read 0x32df982 thru 0x32ed420 (0xda9f bytes) 2024-09-17T07:28:51.014 INFO:teuthology.orchestra.run.smithi045.stdout:398 writesame 0x6cbb27e thru 0x6cbeb40 data_size 0x1(0x38c3 bytes) 2024-09-17T07:28:51.014 INFO:teuthology.orchestra.run.smithi045.stdout:399 punch from 0x1da4f89 to 0x1db1e21, (0xce98 bytes) 2024-09-17T07:28:51.014 INFO:teuthology.orchestra.run.smithi045.stdout:400 writesame 0x4113932 thru 0x411a7f4 data_size 0x5(0x6ec3 bytes) 2024-09-17T07:28:51.015 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:51.008+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:51.017 INFO:teuthology.orchestra.run.smithi045.stdout:402 writesame 0xb246d92 thru 0xb255a95 data_size 0x9a(0xed04 bytes) 2024-09-17T07:28:51.017 INFO:teuthology.orchestra.run.smithi045.stdout:404 writesame 0x4ace22b thru 0x4adc76f data_size 0x1(0xe545 bytes) 2024-09-17T07:28:51.038 INFO:teuthology.orchestra.run.smithi045.stdout:405 read 0x334b02e thru 0x3356f1c (0xbeef bytes) 2024-09-17T07:28:51.038 INFO:teuthology.orchestra.run.smithi045.stdout:408 punch from 0x1afacec to 0x1b08ed4, (0xe1e8 bytes) 2024-09-17T07:28:51.038 INFO:teuthology.orchestra.run.smithi045.stdout:410 punch from 0x2dc85c to 0x2e763c, (0xade0 bytes) 2024-09-17T07:28:51.038 INFO:teuthology.orchestra.run.smithi045.stdout:411 writesame 0x4c720f7 thru 0x4c78e44 data_size 0x11(0x6d4e bytes) 2024-09-17T07:28:51.042 INFO:teuthology.orchestra.run.smithi045.stdout:412 write 0x2b43833 thru 0x2b44943 (0x1111 bytes) 2024-09-17T07:28:51.050 INFO:teuthology.orchestra.run.smithi045.stdout:413 write 0xc96d096 thru 0xc977f34 (0xae9f bytes) 2024-09-17T07:28:51.053 INFO:teuthology.orchestra.run.smithi045.stdout:414 write 0x526dff3 thru 0x526eae1 (0xaef bytes) 2024-09-17T07:28:51.056 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:51.057 INFO:teuthology.orchestra.run.smithi045.stdout:416 read 0x5592a10 thru 0x5598661 (0x5c52 bytes) 2024-09-17T07:28:51.057 INFO:teuthology.orchestra.run.smithi045.stdout:truncating to largest ever: 0xed1697a 2024-09-17T07:28:51.059 INFO:teuthology.orchestra.run.smithi045.stdout:417 trunc from 0xc977f35 to 0xed1697a 2024-09-17T07:28:51.065 INFO:teuthology.orchestra.run.smithi045.stdout:418 writesame 0x6083dc5 thru 0x60865c2 data_size 0x1(0x27fe bytes) 2024-09-17T07:28:51.072 INFO:teuthology.orchestra.run.smithi045.stdout:419 writesame 0xae50c93 thru 0xae5a0c3 data_size 0x1(0x9431 bytes) 2024-09-17T07:28:51.089 INFO:teuthology.orchestra.run.smithi045.stdout:421 read 0xc5b5dcc thru 0xc5c57df (0xfa14 bytes) 2024-09-17T07:28:51.089 INFO:teuthology.orchestra.run.smithi045.stdout:422 read 0xc29851f thru 0xc29ba89 (0x356b bytes) 2024-09-17T07:28:51.089 INFO:teuthology.orchestra.run.smithi045.stdout:426 read 0x6a9566a thru 0x6a990c1 (0x3a58 bytes) 2024-09-17T07:28:51.089 INFO:teuthology.orchestra.run.smithi045.stdout:427 read 0x666d30a thru 0x666e0b7 (0xdae bytes) 2024-09-17T07:28:51.089 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:51.098 INFO:teuthology.orchestra.run.smithi045.stdout:430 trunc from 0xed1697a to 0x6494eba 2024-09-17T07:28:51.109 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:51.108+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:51.116 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T07:28:51.117 INFO:teuthology.orchestra.run.smithi045.stdout:431 trunc from 0x6494eba to 0xcb31e3b 2024-09-17T07:28:51.125 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:28:51.125 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:28:51.125 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:51.134 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:28:51.136 INFO:teuthology.orchestra.run.smithi045.stdout:432 trunc from 0xcb31e3b to 0x1bf11ca 2024-09-17T07:28:51.162 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:28:51.173 INFO:teuthology.orchestra.run.smithi045.stdout:435 writesame 0x7d61b4f thru 0x7d6e765 data_size 0x1(0xcc17 bytes) 2024-09-17T07:28:51.173 INFO:teuthology.orchestra.run.smithi045.stdout:436 punch from 0xc4206a to 0xc4e36d, (0xc303 bytes) 2024-09-17T07:28:51.173 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:51.174 INFO:teuthology.orchestra.run.smithi045.stdout:438 read 0x7a1563c thru 0x7a21a6a (0xc42f bytes) 2024-09-17T07:28:51.174 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:51.174 INFO:teuthology.orchestra.run.smithi045.stdout:440 write 0x273d0ce thru 0x2744e3a (0x7d6d bytes) 2024-09-17T07:28:51.177 INFO:teuthology.orchestra.run.smithi045.stdout:441 write 0x20867f7 thru 0x20880f1 (0x18fb bytes) 2024-09-17T07:28:51.181 INFO:teuthology.orchestra.run.smithi045.stdout:442 read 0x166230c thru 0x166565e (0x3353 bytes) 2024-09-17T07:28:51.185 INFO:teuthology.orchestra.run.smithi045.stdout:443 trunc from 0x7d6e766 to 0xa2100a1 2024-09-17T07:28:51.190 INFO:teuthology.orchestra.run.smithi045.stdout:444 write 0x91a88b5 thru 0x91b2e9e (0xa5ea bytes) 2024-09-17T07:28:51.195 INFO:teuthology.orchestra.run.smithi045.stdout:445 writesame 0x79af5b9 thru 0x79b24a7 data_size 0x2d(0x2eef bytes) 2024-09-17T07:28:51.206 INFO:teuthology.orchestra.run.smithi045.stdout:446 writesame 0xade4889 thru 0xade97bb data_size 0x5(0x4f33 bytes) 2024-09-17T07:28:51.206 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:51.210 INFO:teuthology.orchestra.run.smithi045.stdout:449 trunc from 0xade97bc to 0x7b6ea60 2024-09-17T07:28:51.210 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:51.209+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:51.218 INFO:teuthology.orchestra.run.smithi045.stdout:450 write 0x778c691 thru 0x7798cbc (0xc62c bytes) 2024-09-17T07:28:51.223 INFO:teuthology.orchestra.run.smithi045.stdout:451 writesame 0x2c56e25 thru 0x2c64abe data_size 0x22(0xdc9a bytes) 2024-09-17T07:28:51.228 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:51.228 INFO:teuthology.orchestra.run.smithi045.stdout:458 read 0x549f252 thru 0x549fc7b (0xa2a bytes) 2024-09-17T07:28:51.228 INFO:teuthology.orchestra.run.smithi045.stdout:459 punch from 0x4e0572e to 0x4e0a312, (0x4be4 bytes) 2024-09-17T07:28:51.228 INFO:teuthology.orchestra.run.smithi045.stdout:461 read 0x664967 thru 0x673357 (0xe9f1 bytes) 2024-09-17T07:28:51.228 INFO:teuthology.orchestra.run.smithi045.stdout:462 writesame 0x109fb0a thru 0x10aa169 data_size 0x79(0xa660 bytes) 2024-09-17T07:28:51.233 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:51.247 INFO:teuthology.orchestra.run.smithi045.stdout:464 write 0xe258332 thru 0xe268331 (0x10000 bytes) 2024-09-17T07:28:51.252 INFO:teuthology.orchestra.run.smithi045.stdout:466 punch from 0x165aa2 to 0x16c2b7, (0x6815 bytes) 2024-09-17T07:28:51.252 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:51.252 INFO:teuthology.orchestra.run.smithi045.stdout:469 clone 3 order 23 su 2097152 sc 3 2024-09-17T07:28:51.286 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:28:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:28:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:28:51.286 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:51.295 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:28:51.300 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:28:51.301 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:28:51.318 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:28:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:28:51.334 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:28:51.334 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:28:51.339 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:51.339+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:51.350 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:28:51.351 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:28:51.368 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:28:51.368 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:28:51.384 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:28:51.385 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:28:51.399 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:51.399 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:51.402 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:28:51.402 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:28:51.408 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:28:51.417 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:28:51.417 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:28:51.449 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:51.448+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:51.516 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:51.513+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:51.551 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:51.552 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:51.560 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:28:51.614 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:51.613+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:51.703 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:51.703 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:51.711 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:28:51.755 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:51.755+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:51.815 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:51.815 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:51.816 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:51.815+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:51.824 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:28:51.916 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:51.916+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:51.968 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:51.968 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:51.977 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:28:52.020 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:52.020+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:52.064 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-17T07:28:52.083 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:52.083 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:52.092 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:28:52.123 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:52.122+0000 7f9a047b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33358) UID: 0 2024-09-17T07:28:52.134 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone2 from 0xe268332 (overlap 0x1bf11ca) to 0x138f2da 2024-09-17T07:28:52.220 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:52.218+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:52.242 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-17T07:28:52.252 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:52.252 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:52.261 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:28:52.287 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T07:28:52.288 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:28:52.288 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-17T07:28:52.288 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T07:28:52.288 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T07:28:52.288 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-17T07:28:52.320 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:52.320+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:52.367 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:52.367 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:52.376 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:28:52.404 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:52.405 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T07:28:52.405 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:52.460 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:52.459+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:52.489 INFO:teuthology.orchestra.run.smithi045.stdout:470 read 0x4c1544d thru 0x4c1ca2a (0x75de bytes) 2024-09-17T07:28:52.489 INFO:teuthology.orchestra.run.smithi045.stdout:473 read 0x5689521 thru 0x56968c0 (0xd3a0 bytes) 2024-09-17T07:28:52.490 INFO:teuthology.orchestra.run.smithi045.stdout:474 read 0x348a775 thru 0x348ff2a (0x57b6 bytes) 2024-09-17T07:28:52.491 INFO:teuthology.orchestra.run.smithi045.stdout:476 punch from 0x9085e59 to 0x908d92c, (0x7ad3 bytes) 2024-09-17T07:28:52.495 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:52.495 INFO:teuthology.orchestra.run.smithi045.stdout:480 read 0x148dbe9 thru 0x1498265 (0xa67d bytes) 2024-09-17T07:28:52.495 INFO:teuthology.orchestra.run.smithi045.stdout:481 read 0x9d39ca2 thru 0x9d4405c (0xa3bb bytes) 2024-09-17T07:28:52.495 INFO:teuthology.orchestra.run.smithi045.stdout:482 punch from 0xb0475b3 to 0xb055b78, (0xe5c5 bytes) 2024-09-17T07:28:52.495 INFO:teuthology.orchestra.run.smithi045.stdout:483 writesame 0xc313fad thru 0xc31e661 data_size 0x1(0xa6b5 bytes) 2024-09-17T07:28:52.517 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:52.517 INFO:teuthology.orchestra.run.smithi045.stdout:487 clone 4 order 19 su 8192 sc 8 2024-09-17T07:28:52.520 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:52.520 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:52.521 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:52.520+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:52.529 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:28:52.622 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:52.621+0000 7f9a047b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33358) UID: 0 2024-09-17T07:28:52.672 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:52.672 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:52.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:28:52.724 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:52.723+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:52.793 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:52.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:28:52.825 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:52.824+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:52.907 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:52.908 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:52.916 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:28:52.926 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:52.925+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:53.023 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:53.023 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:53.027 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:53.026+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:53.032 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:28:53.128 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:53.126+0000 7f9a047b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33358) UID: 0 2024-09-17T07:28:53.142 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:53.142 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:53.154 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:28:53.185 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone3 from 0xe268332 (overlap 0xe268332) to 0x0 2024-09-17T07:28:53.228 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:53.227+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:53.313 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:53.313 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:53.322 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-17T07:28:53.323 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:28:53.365 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:53.363+0000 7f9a047b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33358) UID: 0 2024-09-17T07:28:53.368 INFO:teuthology.orchestra.run.smithi045.stdout:490 trunc from 0xe268332 to 0x38c21dd 2024-09-17T07:28:53.399 INFO:teuthology.orchestra.run.smithi045.stdout:491 write 0x5cedbaf thru 0x5cfc612 (0xea64 bytes) 2024-09-17T07:28:53.407 INFO:teuthology.orchestra.run.smithi045.stdout:494 punch from 0x525af17 to 0x525c2c7, (0x13b0 bytes) 2024-09-17T07:28:53.407 INFO:teuthology.orchestra.run.smithi045.stdout:495 writesame 0x5a6e838 thru 0x5a74fcf data_size 0x22(0x6798 bytes) 2024-09-17T07:28:53.427 INFO:teuthology.orchestra.run.smithi045.stdout:496 write 0xe056c16 thru 0xe065f46 (0xf331 bytes) 2024-09-17T07:28:53.429 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:28:53.430 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:53.429+0000 7f9a047b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33358) UID: 0 2024-09-17T07:28:53.435 INFO:teuthology.orchestra.run.smithi045.stdout:497 read 0xa7ae975 thru 0xa7b3863 (0x4eef bytes) 2024-09-17T07:28:53.435 INFO:teuthology.orchestra.run.smithi045.stdout:498 read 0xb8808c9 thru 0xb8857c5 (0x4efd bytes) 2024-09-17T07:28:53.435 INFO:teuthology.orchestra.run.smithi045.stdout:500 write 0x4479efc thru 0x4480805 (0x690a bytes) 2024-09-17T07:28:53.442 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:53.442 INFO:teuthology.orchestra.run.smithi045.stdout:503 read 0x7e68af7 thru 0x7e726ae (0x9bb8 bytes) 2024-09-17T07:28:53.442 INFO:teuthology.orchestra.run.smithi045.stdout:505 punch from 0x8c42c00 to 0x8c51478, (0xe878 bytes) 2024-09-17T07:28:53.442 INFO:teuthology.orchestra.run.smithi045.stdout:506 write 0x910dbc1 thru 0x911a545 (0xc985 bytes) 2024-09-17T07:28:53.449 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:53.450 INFO:teuthology.orchestra.run.smithi045.stdout:509 writesame 0xb8ad01d thru 0xb8b484f data_size 0x27(0x7833 bytes) 2024-09-17T07:28:53.456 INFO:teuthology.orchestra.run.smithi045.stdout:510 write 0x9abad53 thru 0x9ac72c7 (0xc575 bytes) 2024-09-17T07:28:53.464 INFO:teuthology.orchestra.run.smithi045.stdout:512 clone 5 order 20 su 262144 sc 15 2024-09-17T07:28:53.491 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:53.491 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:53.500 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:28:53.541 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:53.540+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:53.561 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:28:53.562 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:28:53.579 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:28:53.579 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:28:53.596 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:28:53.596 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:28:53.612 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:28:53.613 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:28:53.630 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:28:53.630 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:28:53.631 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:53.630+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:53.646 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:28:53.646 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:28:53.650 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:53.650 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:53.650 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:28:53.650 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:53.659 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:28:53.663 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:28:53.664 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:28:53.681 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:28:53.681 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:28:53.732 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:53.732+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:53.771 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:53.771 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:53.771 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:28:53.771 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:28:53.771 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:53.781 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:28:53.833 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:53.831+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:53.889 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:53.889 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:28:53.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:28:53.889 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:28:53.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.889 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:759 3.4 e28/23 3:20e84768:::rbd_data.1183350c2d82.0000000000000075:head v 28'6, mlcod=28'4)", 2024-09-17T07:28:53.890 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:28:53.890 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.9629511470000001, 2024-09-17T07:28:53.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011641620000000001, 2024-09-17T07:28:53.890 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.890 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.890 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.890 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:28:53.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:28:53.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:28:53.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6100000000000002e-07 2024-09-17T07:28:53.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924866+0000", 2024-09-17T07:28:53.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:28:53.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924875+0000", 2024-09-17T07:28:53.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7909999999999993e-06 2024-09-17T07:28:53.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924882+0000", 2024-09-17T07:28:53.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.418e-06 2024-09-17T07:28:53.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924965+0000", 2024-09-17T07:28:53.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3029999999999996e-05 2024-09-17T07:28:53.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924987+0000", 2024-09-17T07:28:53.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1747e-05 2024-09-17T07:28:53.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.925980+0000", 2024-09-17T07:28:53.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00099299900000000005 2024-09-17T07:28:53.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.926025+0000", 2024-09-17T07:28:53.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4977e-05 2024-09-17T07:28:53.894 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.894 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.894 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.894 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:856 3.4 e28/23 3:220acdab:::rbd_data.1183350c2d82.0000000000000070:head v 28'7, mlcod=28'5)", 2024-09-17T07:28:53.894 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.042406+0000", 2024-09-17T07:28:53.894 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.8454063000000005, 2024-09-17T07:28:53.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00091025500000000001, 2024-09-17T07:28:53.894 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.894 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.894 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.894 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042406+0000", 2024-09-17T07:28:53.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042406+0000", 2024-09-17T07:28:53.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042407+0000", 2024-09-17T07:28:53.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4169999999999999e-06 2024-09-17T07:28:53.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042438+0000", 2024-09-17T07:28:53.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0409e-05 2024-09-17T07:28:53.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042455+0000", 2024-09-17T07:28:53.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6883000000000001e-05 2024-09-17T07:28:53.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042463+0000", 2024-09-17T07:28:53.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6470000000000004e-06 2024-09-17T07:28:53.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042532+0000", 2024-09-17T07:28:53.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8651000000000004e-05 2024-09-17T07:28:53.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.042554+0000", 2024-09-17T07:28:53.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1889e-05 2024-09-17T07:28:53.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.043272+0000", 2024-09-17T07:28:53.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00071802499999999996 2024-09-17T07:28:53.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.043316+0000", 2024-09-17T07:28:53.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4333999999999999e-05 2024-09-17T07:28:53.898 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.898 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.898 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.899 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:903 3.9 e29/23 3:97508d79:::rbd_header.1183350c2d82:head v 29'49, mlcod=28'47)", 2024-09-17T07:28:53.899 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.104471+0000", 2024-09-17T07:28:53.899 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.7833413589999996, 2024-09-17T07:28:53.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00090332699999999999, 2024-09-17T07:28:53.899 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.899 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.899 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.899 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.899 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104471+0000", 2024-09-17T07:28:53.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.900 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104471+0000", 2024-09-17T07:28:53.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.900 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104471+0000", 2024-09-17T07:28:53.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:28:53.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.900 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104490+0000", 2024-09-17T07:28:53.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8678e-05 2024-09-17T07:28:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104501+0000", 2024-09-17T07:28:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1559999999999999e-05 2024-09-17T07:28:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104510+0000", 2024-09-17T07:28:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0610000000000002e-06 2024-09-17T07:28:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104573+0000", 2024-09-17T07:28:53.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3071000000000007e-05 2024-09-17T07:28:53.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.104593+0000", 2024-09-17T07:28:53.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9371000000000001e-05 2024-09-17T07:28:53.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.105327+0000", 2024-09-17T07:28:53.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073368900000000004 2024-09-17T07:28:53.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.105374+0000", 2024-09-17T07:28:53.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7589999999999997e-05 2024-09-17T07:28:53.903 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.903 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.903 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.903 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:986 3.0 e29/23 3:04a2435f:::rbd_data.1183350c2d82.0000000000000017:head v 29'3, mlcod=28'1)", 2024-09-17T07:28:53.903 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.234074+0000", 2024-09-17T07:28:53.903 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.6537384729999998, 2024-09-17T07:28:53.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00094026099999999996, 2024-09-17T07:28:53.903 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.903 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.903 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.904 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234074+0000", 2024-09-17T07:28:53.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234074+0000", 2024-09-17T07:28:53.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234075+0000", 2024-09-17T07:28:53.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.344e-06 2024-09-17T07:28:53.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234117+0000", 2024-09-17T07:28:53.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1971000000000002e-05 2024-09-17T07:28:53.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234137+0000", 2024-09-17T07:28:53.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0432e-05 2024-09-17T07:28:53.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234145+0000", 2024-09-17T07:28:53.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8159999999999994e-06 2024-09-17T07:28:53.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234205+0000", 2024-09-17T07:28:53.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9979999999999998e-05 2024-09-17T07:28:53.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234230+0000", 2024-09-17T07:28:53.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4587e-05 2024-09-17T07:28:53.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.234965+0000", 2024-09-17T07:28:53.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073509300000000003 2024-09-17T07:28:53.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.235014+0000", 2024-09-17T07:28:53.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9038e-05 2024-09-17T07:28:53.907 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.907 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.907 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.908 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1018 3.4 e29/23 3:20e84768:::rbd_data.1183350c2d82.0000000000000075:head v 29'10, mlcod=28'8)", 2024-09-17T07:28:53.908 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.270607+0000", 2024-09-17T07:28:53.908 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.6172048129999999, 2024-09-17T07:28:53.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00091791799999999999, 2024-09-17T07:28:53.908 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.908 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.908 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.908 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.908 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270607+0000", 2024-09-17T07:28:53.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270607+0000", 2024-09-17T07:28:53.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270609+0000", 2024-09-17T07:28:53.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6619999999999999e-06 2024-09-17T07:28:53.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270620+0000", 2024-09-17T07:28:53.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0718e-05 2024-09-17T07:28:53.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270640+0000", 2024-09-17T07:28:53.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0760000000000001e-05 2024-09-17T07:28:53.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270650+0000", 2024-09-17T07:28:53.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6080000000000002e-06 2024-09-17T07:28:53.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270715+0000", 2024-09-17T07:28:53.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5265999999999995e-05 2024-09-17T07:28:53.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270740+0000", 2024-09-17T07:28:53.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4584e-05 2024-09-17T07:28:53.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271483+0000", 2024-09-17T07:28:53.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000742941 2024-09-17T07:28:53.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271525+0000", 2024-09-17T07:28:53.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2379000000000002e-05 2024-09-17T07:28:53.912 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.912 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.912 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.912 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1012 3.9 e29/23 3:93118134:::rbd_data.1183350c2d82.0000000000000067:head v 29'55, mlcod=29'53)", 2024-09-17T07:28:53.912 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.270691+0000", 2024-09-17T07:28:53.912 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.6171210829999998, 2024-09-17T07:28:53.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011919299999999999, 2024-09-17T07:28:53.912 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.912 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.912 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.913 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.913 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270691+0000", 2024-09-17T07:28:53.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.913 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270691+0000", 2024-09-17T07:28:53.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.913 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270692+0000", 2024-09-17T07:28:53.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9800000000000002e-07 2024-09-17T07:28:53.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.914 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270699+0000", 2024-09-17T07:28:53.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1679999999999998e-06 2024-09-17T07:28:53.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.914 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270717+0000", 2024-09-17T07:28:53.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8193e-05 2024-09-17T07:28:53.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.914 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270726+0000", 2024-09-17T07:28:53.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4519999999999997e-06 2024-09-17T07:28:53.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.915 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.915 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270779+0000", 2024-09-17T07:28:53.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3202000000000003e-05 2024-09-17T07:28:53.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.915 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.915 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270800+0000", 2024-09-17T07:28:53.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0846e-05 2024-09-17T07:28:53.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.915 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.916 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271841+0000", 2024-09-17T07:28:53.916 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001040921 2024-09-17T07:28:53.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.916 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.916 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271883+0000", 2024-09-17T07:28:53.916 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2150000000000001e-05 2024-09-17T07:28:53.916 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.916 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.916 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.916 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1058 3.9 e29/23 3:93118134:::rbd_data.1183350c2d82.0000000000000067:head v 29'59, mlcod=29'57)", 2024-09-17T07:28:53.916 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.303628+0000", 2024-09-17T07:28:53.917 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.5841843080000002, 2024-09-17T07:28:53.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001104257, 2024-09-17T07:28:53.917 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.917 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.917 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.917 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.917 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303628+0000", 2024-09-17T07:28:53.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.917 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303628+0000", 2024-09-17T07:28:53.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303629+0000", 2024-09-17T07:28:53.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.347e-06 2024-09-17T07:28:53.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303640+0000", 2024-09-17T07:28:53.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0387999999999999e-05 2024-09-17T07:28:53.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303661+0000", 2024-09-17T07:28:53.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1263999999999998e-05 2024-09-17T07:28:53.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303670+0000", 2024-09-17T07:28:53.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.55e-06 2024-09-17T07:28:53.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303739+0000", 2024-09-17T07:28:53.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8523999999999996e-05 2024-09-17T07:28:53.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.920 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.920 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303764+0000", 2024-09-17T07:28:53.920 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4800999999999999e-05 2024-09-17T07:28:53.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.920 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.920 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304674+0000", 2024-09-17T07:28:53.920 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00090985600000000001 2024-09-17T07:28:53.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.920 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.920 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304732+0000", 2024-09-17T07:28:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8526999999999997e-05 2024-09-17T07:28:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1054 3.0 e29/23 3:04a2435f:::rbd_data.1183350c2d82.0000000000000017:head v 29'4, mlcod=28'2)", 2024-09-17T07:28:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.304028+0000", 2024-09-17T07:28:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.5837840769999998, 2024-09-17T07:28:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010024789999999999, 2024-09-17T07:28:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304028+0000", 2024-09-17T07:28:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304028+0000", 2024-09-17T07:28:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304029+0000", 2024-09-17T07:28:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1429999999999999e-06 2024-09-17T07:28:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304038+0000", 2024-09-17T07:28:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2480000000000002e-06 2024-09-17T07:28:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304050+0000", 2024-09-17T07:28:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1756999999999999e-05 2024-09-17T07:28:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304058+0000", 2024-09-17T07:28:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7589999999999997e-06 2024-09-17T07:28:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304129+0000", 2024-09-17T07:28:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1318999999999999e-05 2024-09-17T07:28:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304149+0000", 2024-09-17T07:28:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0128000000000001e-05 2024-09-17T07:28:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304990+0000", 2024-09-17T07:28:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000840955 2024-09-17T07:28:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.305031+0000", 2024-09-17T07:28:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0170000000000003e-05 2024-09-17T07:28:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1159 3.0 e29/23 3:07b6f4cb:::rbd_data.1183350c2d82.0000000000000063:head v 29'5, mlcod=29'3)", 2024-09-17T07:28:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.450139+0000", 2024-09-17T07:28:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.4376736130000003, 2024-09-17T07:28:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00092069299999999999, 2024-09-17T07:28:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450139+0000", 2024-09-17T07:28:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450139+0000", 2024-09-17T07:28:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450140+0000", 2024-09-17T07:28:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.097e-06 2024-09-17T07:28:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450157+0000", 2024-09-17T07:28:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7041000000000002e-05 2024-09-17T07:28:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450172+0000", 2024-09-17T07:28:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5315000000000002e-05 2024-09-17T07:28:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450183+0000", 2024-09-17T07:28:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0964e-05 2024-09-17T07:28:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450247+0000", 2024-09-17T07:28:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4348000000000001e-05 2024-09-17T07:28:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450271+0000", 2024-09-17T07:28:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3244e-05 2024-09-17T07:28:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.451016+0000", 2024-09-17T07:28:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00074592899999999999 2024-09-17T07:28:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.451059+0000", 2024-09-17T07:28:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2755e-05 2024-09-17T07:28:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1531 3.4 e30/23 3:25f2f06a:::rbd_data.11832b928878.00000000000001bb:head v 30'11, mlcod=29'9)", 2024-09-17T07:28:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.599599+0000", 2024-09-17T07:28:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.288213389, 2024-09-17T07:28:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00091589899999999997, 2024-09-17T07:28:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.599599+0000", 2024-09-17T07:28:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.599599+0000", 2024-09-17T07:28:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.599600+0000", 2024-09-17T07:28:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0920000000000001e-06 2024-09-17T07:28:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.599606+0000", 2024-09-17T07:28:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2580000000000001e-06 2024-09-17T07:28:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.599621+0000", 2024-09-17T07:28:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4968e-05 2024-09-17T07:28:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.599628+0000", 2024-09-17T07:28:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0310000000000001e-06 2024-09-17T07:28:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.599699+0000", 2024-09-17T07:28:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1099000000000005e-05 2024-09-17T07:28:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.599725+0000", 2024-09-17T07:28:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6259e-05 2024-09-17T07:28:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.600448+0000", 2024-09-17T07:28:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072236299999999995 2024-09-17T07:28:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.600515+0000", 2024-09-17T07:28:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6828999999999993e-05 2024-09-17T07:28:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1610 3.0 e30/23 3:02fd7b4f:::rbd_data.11832b928878.00000000000000b4:head v 30'9, mlcod=29'7)", 2024-09-17T07:28:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.672629+0000", 2024-09-17T07:28:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.2151827470000001, 2024-09-17T07:28:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010181859999999999, 2024-09-17T07:28:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.672629+0000", 2024-09-17T07:28:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.672629+0000", 2024-09-17T07:28:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.672631+0000", 2024-09-17T07:28:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6500000000000001e-06 2024-09-17T07:28:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.672676+0000", 2024-09-17T07:28:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4719999999999999e-05 2024-09-17T07:28:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.672695+0000", 2024-09-17T07:28:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8938000000000001e-05 2024-09-17T07:28:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.672712+0000", 2024-09-17T07:28:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7404000000000001e-05 2024-09-17T07:28:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.672781+0000", 2024-09-17T07:28:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8380999999999995e-05 2024-09-17T07:28:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.672807+0000", 2024-09-17T07:28:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6809999999999999e-05 2024-09-17T07:28:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.673599+0000", 2024-09-17T07:28:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00079195899999999996 2024-09-17T07:28:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.673648+0000", 2024-09-17T07:28:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8324e-05 2024-09-17T07:28:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1649 3.9 e30/23 3:96c3871d:::rbd_data.11832b928878.000000000000037c:head v 30'79, mlcod=30'77)", 2024-09-17T07:28:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.712747+0000", 2024-09-17T07:28:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.175065472, 2024-09-17T07:28:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089255599999999997, 2024-09-17T07:28:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.712747+0000", 2024-09-17T07:28:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.712747+0000", 2024-09-17T07:28:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.712748+0000", 2024-09-17T07:28:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.48e-06 2024-09-17T07:28:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.712759+0000", 2024-09-17T07:28:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0943e-05 2024-09-17T07:28:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.712780+0000", 2024-09-17T07:28:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0564000000000002e-05 2024-09-17T07:28:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.712791+0000", 2024-09-17T07:28:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.099e-05 2024-09-17T07:28:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.712857+0000", 2024-09-17T07:28:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6249999999999998e-05 2024-09-17T07:28:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.712881+0000", 2024-09-17T07:28:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4576999999999999e-05 2024-09-17T07:28:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.713598+0000", 2024-09-17T07:28:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00071626100000000005 2024-09-17T07:28:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.713639+0000", 2024-09-17T07:28:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1491e-05 2024-09-17T07:28:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1738 3.9 e30/23 3:96a92ba0:::rbd_data.11832b928878.00000000000001b5:head v 30'80, mlcod=30'78)", 2024-09-17T07:28:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.789294+0000", 2024-09-17T07:28:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.09851769, 2024-09-17T07:28:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098913100000000008, 2024-09-17T07:28:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.789294+0000", 2024-09-17T07:28:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.789294+0000", 2024-09-17T07:28:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.789296+0000", 2024-09-17T07:28:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.311e-06 2024-09-17T07:28:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.789304+0000", 2024-09-17T07:28:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.704e-06 2024-09-17T07:28:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.789326+0000", 2024-09-17T07:28:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1126000000000001e-05 2024-09-17T07:28:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.789338+0000", 2024-09-17T07:28:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2825e-05 2024-09-17T07:28:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.789638+0000", 2024-09-17T07:28:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00029988500000000001 2024-09-17T07:28:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.789664+0000", 2024-09-17T07:28:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5684e-05 2024-09-17T07:28:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.790237+0000", 2024-09-17T07:28:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00057308599999999999 2024-09-17T07:28:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.790284+0000", 2024-09-17T07:28:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.651e-05 2024-09-17T07:28:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1744 3.4 e30/23 3:25f2f06a:::rbd_data.11832b928878.00000000000001bb:head v 30'13, mlcod=30'11)", 2024-09-17T07:28:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.790779+0000", 2024-09-17T07:28:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.0970332890000001, 2024-09-17T07:28:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001057466, 2024-09-17T07:28:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.790779+0000", 2024-09-17T07:28:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.790779+0000", 2024-09-17T07:28:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.790780+0000", 2024-09-17T07:28:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9500000000000001e-07 2024-09-17T07:28:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.790788+0000", 2024-09-17T07:28:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4849999999999992e-06 2024-09-17T07:28:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.790805+0000", 2024-09-17T07:28:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6532999999999999e-05 2024-09-17T07:28:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.790812+0000", 2024-09-17T07:28:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1600000000000001e-06 2024-09-17T07:28:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.790873+0000", 2024-09-17T07:28:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0738e-05 2024-09-17T07:28:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.790904+0000", 2024-09-17T07:28:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1545999999999999e-05 2024-09-17T07:28:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791794+0000", 2024-09-17T07:28:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00088975999999999996 2024-09-17T07:28:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791836+0000", 2024-09-17T07:28:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2449e-05 2024-09-17T07:28:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1746 3.4 e30/23 3:21509c5f:::rbd_data.11832b928878.00000000000001bd:head v 30'14, mlcod=30'11)", 2024-09-17T07:28:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.791256+0000", 2024-09-17T07:28:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.0965563469999999, 2024-09-17T07:28:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001583414, 2024-09-17T07:28:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791256+0000", 2024-09-17T07:28:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791256+0000", 2024-09-17T07:28:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791257+0000", 2024-09-17T07:28:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8599999999999997e-07 2024-09-17T07:28:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791290+0000", 2024-09-17T07:28:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3158999999999999e-05 2024-09-17T07:28:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791298+0000", 2024-09-17T07:28:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2700000000000004e-06 2024-09-17T07:28:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791305+0000", 2024-09-17T07:28:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8159999999999996e-06 2024-09-17T07:28:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791358+0000", 2024-09-17T07:28:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3300999999999997e-05 2024-09-17T07:28:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791377+0000", 2024-09-17T07:28:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9062000000000001e-05 2024-09-17T07:28:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.792791+0000", 2024-09-17T07:28:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014139160000000001 2024-09-17T07:28:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.792839+0000", 2024-09-17T07:28:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8103999999999999e-05 2024-09-17T07:28:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "pg_update_log_missing(3.4 epoch 30/23 rep_tid 52 entries 30'15 (0'0) error 3:21efcff5:::rbd_data.11832b928878.00000000000000ea:head by client.4483.0:1908 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'13)", 2024-09-17T07:28:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.843158+0000", 2024-09-17T07:28:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.044654156, 2024-09-17T07:28:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00092016299999999999, 2024-09-17T07:28:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:28:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.843158+0000", 2024-09-17T07:28:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.843158+0000", 2024-09-17T07:28:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.843159+0000", 2024-09-17T07:28:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.294e-06 2024-09-17T07:28:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.843173+0000", 2024-09-17T07:28:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4056000000000001e-05 2024-09-17T07:28:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.843203+0000", 2024-09-17T07:28:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9159999999999999e-05 2024-09-17T07:28:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.843219+0000", 2024-09-17T07:28:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6937999999999999e-05 2024-09-17T07:28:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.843311+0000", 2024-09-17T07:28:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1139999999999998e-05 2024-09-17T07:28:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.844078+0000", 2024-09-17T07:28:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076757500000000005 2024-09-17T07:28:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:2010 3.4 e30/23 3:221ed066:::rbd_data.11832b928878.0000000000000073:head v 30'17, mlcod=30'15)", 2024-09-17T07:28:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.914801+0000", 2024-09-17T07:28:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.9730115619999999, 2024-09-17T07:28:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00091329699999999998, 2024-09-17T07:28:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.914801+0000", 2024-09-17T07:28:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.914801+0000", 2024-09-17T07:28:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.914802+0000", 2024-09-17T07:28:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.331e-06 2024-09-17T07:28:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.914820+0000", 2024-09-17T07:28:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8267e-05 2024-09-17T07:28:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.914839+0000", 2024-09-17T07:28:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8499e-05 2024-09-17T07:28:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.914850+0000", 2024-09-17T07:28:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1219999999999999e-05 2024-09-17T07:28:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.914915+0000", 2024-09-17T07:28:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4677000000000004e-05 2024-09-17T07:28:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.914938+0000", 2024-09-17T07:28:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3564e-05 2024-09-17T07:28:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.915673+0000", 2024-09-17T07:28:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073485499999999997 2024-09-17T07:28:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.915714+0000", 2024-09-17T07:28:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0883999999999997e-05 2024-09-17T07:28:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:2183 3.4 e30/23 3:221ed066:::rbd_data.11832b928878.0000000000000073:head v 30'19, mlcod=30'17)", 2024-09-17T07:28:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:51.139568+0000", 2024-09-17T07:28:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.7482442640000002, 2024-09-17T07:28:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00124438, 2024-09-17T07:28:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139568+0000", 2024-09-17T07:28:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139568+0000", 2024-09-17T07:28:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139569+0000", 2024-09-17T07:28:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3769999999999999e-06 2024-09-17T07:28:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139580+0000", 2024-09-17T07:28:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0229e-05 2024-09-17T07:28:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139601+0000", 2024-09-17T07:28:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1818999999999999e-05 2024-09-17T07:28:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139611+0000", 2024-09-17T07:28:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0121e-05 2024-09-17T07:28:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139676+0000", 2024-09-17T07:28:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4900999999999994e-05 2024-09-17T07:28:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139706+0000", 2024-09-17T07:28:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9762000000000001e-05 2024-09-17T07:28:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.140765+0000", 2024-09-17T07:28:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001059019 2024-09-17T07:28:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.140812+0000", 2024-09-17T07:28:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7151999999999998e-05 2024-09-17T07:28:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:2190 3.0 e30/23 3:02fd7b4f:::rbd_data.11832b928878.00000000000000b4:head v 30'16, mlcod=30'14)", 2024-09-17T07:28:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:28:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.7478994370000001, 2024-09-17T07:28:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001228395, 2024-09-17T07:28:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:28:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:28:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:28:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0399999999999996e-07 2024-09-17T07:28:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139918+0000", 2024-09-17T07:28:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8330000000000001e-06 2024-09-17T07:28:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139926+0000", 2024-09-17T07:28:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1720000000000007e-06 2024-09-17T07:28:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139932+0000", 2024-09-17T07:28:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1789999999999996e-06 2024-09-17T07:28:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139988+0000", 2024-09-17T07:28:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5265e-05 2024-09-17T07:28:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.140004+0000", 2024-09-17T07:28:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6171000000000002e-05 2024-09-17T07:28:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.141100+0000", 2024-09-17T07:28:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010964519999999999 2024-09-17T07:28:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.141141+0000", 2024-09-17T07:28:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0818999999999997e-05 2024-09-17T07:28:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:2290 3.9 e30/23 3:939ef164:::rbd_data.11832b928878.00000000000001df:head v 30'82, mlcod=30'80)", 2024-09-17T07:28:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:51.221799+0000", 2024-09-17T07:28:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.666012958, 2024-09-17T07:28:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00092848200000000001, 2024-09-17T07:28:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.221799+0000", 2024-09-17T07:28:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.221799+0000", 2024-09-17T07:28:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.221801+0000", 2024-09-17T07:28:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5060000000000001e-06 2024-09-17T07:28:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.221834+0000", 2024-09-17T07:28:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2873000000000002e-05 2024-09-17T07:28:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.221855+0000", 2024-09-17T07:28:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1140000000000001e-05 2024-09-17T07:28:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.221865+0000", 2024-09-17T07:28:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9189999999999994e-06 2024-09-17T07:28:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.221933+0000", 2024-09-17T07:28:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8696999999999996e-05 2024-09-17T07:28:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.221950+0000", 2024-09-17T07:28:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6861999999999999e-05 2024-09-17T07:28:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.222671+0000", 2024-09-17T07:28:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072034099999999999 2024-09-17T07:28:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.222728+0000", 2024-09-17T07:28:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7144e-05 2024-09-17T07:28:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:53.982 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:53.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:28:53.984 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:53.983+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:54.083 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T07:28:54.084 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:54.083+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:54.136 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:54.136 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:54.136 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:28:54.136 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:54.146 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:28:54.185 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:54.184+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone4 from 0xe065f47 (overlap 0x38c21dd) to 0x0 2024-09-17T07:28:54.265 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:54.266 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:54.266 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:28:54.266 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:28:54.266 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:54.277 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:28:54.286 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:54.285+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:54.386 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:54.386+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:54.387 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:54.387 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:28:54.388 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:28:54.388 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:28:54.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.388 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:869 3.2 3:449b730c:::rbd_data.1183350c2d82.000000000000004d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:54.388 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.069352+0000", 2024-09-17T07:28:54.388 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.316941742, 2024-09-17T07:28:54.388 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036811930000000001, 2024-09-17T07:28:54.388 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.388 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.389 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.389 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.389 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.389 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.389 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 869 2024-09-17T07:28:54.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.389 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.389 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.389 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.389 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069352+0000", 2024-09-17T07:28:54.389 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.390 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.390 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069352+0000", 2024-09-17T07:28:54.390 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.390 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.390 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069352+0000", 2024-09-17T07:28:54.390 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6599999999999996e-07 2024-09-17T07:28:54.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.390 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.390 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069356+0000", 2024-09-17T07:28:54.390 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:28:54.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.391 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.391 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069367+0000", 2024-09-17T07:28:54.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1635e-05 2024-09-17T07:28:54.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.391 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.391 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069374+0000", 2024-09-17T07:28:54.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8399999999999997e-06 2024-09-17T07:28:54.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069432+0000", 2024-09-17T07:28:54.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8100000000000003e-05 2024-09-17T07:28:54.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069519+0000", 2024-09-17T07:28:54.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6459999999999996e-05 2024-09-17T07:28:54.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:28:54.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069627+0000", 2024-09-17T07:28:54.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010812299999999999 2024-09-17T07:28:54.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:54.393 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070811+0000", 2024-09-17T07:28:54.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011842179999999999 2024-09-17T07:28:54.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:54.393 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.072931+0000", 2024-09-17T07:28:54.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021195879999999999 2024-09-17T07:28:54.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:54.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.072988+0000", 2024-09-17T07:28:54.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7015000000000002e-05 2024-09-17T07:28:54.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.394 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073033+0000", 2024-09-17T07:28:54.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5380000000000003e-05 2024-09-17T07:28:54.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.394 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.394 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.394 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:874 3.2 3:42e15a00:::rbd_data.1183350c2d82.0000000000000082:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:54.395 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.069393+0000", 2024-09-17T07:28:54.395 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3169000439999996, 2024-09-17T07:28:54.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0063855170000000003, 2024-09-17T07:28:54.395 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.395 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.395 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.395 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.395 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 874 2024-09-17T07:28:54.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.395 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.396 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069393+0000", 2024-09-17T07:28:54.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.396 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069393+0000", 2024-09-17T07:28:54.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.396 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069394+0000", 2024-09-17T07:28:54.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.75e-07 2024-09-17T07:28:54.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.397 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.397 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069397+0000", 2024-09-17T07:28:54.397 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7020000000000001e-06 2024-09-17T07:28:54.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.397 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.397 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069405+0000", 2024-09-17T07:28:54.397 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.763e-06 2024-09-17T07:28:54.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.397 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.397 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069410+0000", 2024-09-17T07:28:54.398 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6620000000000002e-06 2024-09-17T07:28:54.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.398 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.398 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069903+0000", 2024-09-17T07:28:54.398 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00049267500000000004 2024-09-17T07:28:54.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.398 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.398 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069970+0000", 2024-09-17T07:28:54.398 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7469000000000001e-05 2024-09-17T07:28:54.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:28:54.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070074+0000", 2024-09-17T07:28:54.399 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000103305 2024-09-17T07:28:54.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:54.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.071385+0000", 2024-09-17T07:28:54.399 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013110050000000001 2024-09-17T07:28:54.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:54.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075665+0000", 2024-09-17T07:28:54.399 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0042803659999999999 2024-09-17T07:28:54.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:54.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075697+0000", 2024-09-17T07:28:54.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2181999999999998e-05 2024-09-17T07:28:54.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075779+0000", 2024-09-17T07:28:54.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1812999999999997e-05 2024-09-17T07:28:54.400 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.400 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.400 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.401 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:878 3.7 3:e4893214:::rbd_data.1183350c2d82.00000000000000b1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:54.401 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.070655+0000", 2024-09-17T07:28:54.401 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.315638292, 2024-09-17T07:28:54.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0052679559999999999, 2024-09-17T07:28:54.401 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.401 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.401 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.401 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.401 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.401 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 878 2024-09-17T07:28:54.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.402 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.402 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.402 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070655+0000", 2024-09-17T07:28:54.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.402 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.402 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070655+0000", 2024-09-17T07:28:54.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070656+0000", 2024-09-17T07:28:54.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7300000000000004e-07 2024-09-17T07:28:54.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070662+0000", 2024-09-17T07:28:54.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2989999999999996e-06 2024-09-17T07:28:54.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070676+0000", 2024-09-17T07:28:54.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4272e-05 2024-09-17T07:28:54.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.404 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070684+0000", 2024-09-17T07:28:54.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0269999999999995e-06 2024-09-17T07:28:54.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.404 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070799+0000", 2024-09-17T07:28:54.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000115123 2024-09-17T07:28:54.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070887+0000", 2024-09-17T07:28:54.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8426999999999999e-05 2024-09-17T07:28:54.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:28:54.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.071007+0000", 2024-09-17T07:28:54.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119677 2024-09-17T07:28:54.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:54.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.071596+0000", 2024-09-17T07:28:54.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00058881000000000005 2024-09-17T07:28:54.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:54.406 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075826+0000", 2024-09-17T07:28:54.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0042297740000000004 2024-09-17T07:28:54.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:54.406 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075879+0000", 2024-09-17T07:28:54.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3350999999999998e-05 2024-09-17T07:28:54.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.407 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075923+0000", 2024-09-17T07:28:54.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4223e-05 2024-09-17T07:28:54.407 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.407 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.407 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.407 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1052 3.7 3:e4777bd3:::rbd_data.1183350c2d82.0000000000000012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:54.407 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.303247+0000", 2024-09-17T07:28:54.407 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.0830467989999999, 2024-09-17T07:28:54.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003696857, 2024-09-17T07:28:54.407 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.408 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.408 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.408 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.408 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1052 2024-09-17T07:28:54.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.408 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.408 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.408 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303247+0000", 2024-09-17T07:28:54.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.409 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303247+0000", 2024-09-17T07:28:54.409 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.409 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303250+0000", 2024-09-17T07:28:54.409 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8270000000000002e-06 2024-09-17T07:28:54.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.409 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303259+0000", 2024-09-17T07:28:54.409 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3610000000000007e-06 2024-09-17T07:28:54.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.410 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303282+0000", 2024-09-17T07:28:54.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3190999999999999e-05 2024-09-17T07:28:54.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.410 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303293+0000", 2024-09-17T07:28:54.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0641e-05 2024-09-17T07:28:54.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.411 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303374+0000", 2024-09-17T07:28:54.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1137999999999995e-05 2024-09-17T07:28:54.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.411 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303476+0000", 2024-09-17T07:28:54.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010225699999999999 2024-09-17T07:28:54.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.411 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:28:54.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303612+0000", 2024-09-17T07:28:54.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013634500000000001 2024-09-17T07:28:54.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:54.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304357+0000", 2024-09-17T07:28:54.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00074472399999999999 2024-09-17T07:28:54.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:54.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.306886+0000", 2024-09-17T07:28:54.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025291519999999998 2024-09-17T07:28:54.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:54.413 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.306916+0000", 2024-09-17T07:28:54.413 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0077e-05 2024-09-17T07:28:54.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.413 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.413 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.306944+0000", 2024-09-17T07:28:54.413 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7144e-05 2024-09-17T07:28:54.413 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.413 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.413 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.413 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1109 3.1e 3:78e7e4bb:::rbd_data.1183350c2d82.000000000000002a:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:54.413 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.357457+0000", 2024-09-17T07:28:54.413 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.028836117, 2024-09-17T07:28:54.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.015696132000000002, 2024-09-17T07:28:54.414 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.414 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.414 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.414 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.414 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.414 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1109 2024-09-17T07:28:54.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.414 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.414 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357457+0000", 2024-09-17T07:28:54.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.415 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357457+0000", 2024-09-17T07:28:54.415 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.415 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357459+0000", 2024-09-17T07:28:54.415 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8810000000000001e-06 2024-09-17T07:28:54.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.415 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357467+0000", 2024-09-17T07:28:54.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3969999999999997e-06 2024-09-17T07:28:54.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.416 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357485+0000", 2024-09-17T07:28:54.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7898e-05 2024-09-17T07:28:54.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.416 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357494+0000", 2024-09-17T07:28:54.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3000000000000007e-06 2024-09-17T07:28:54.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357552+0000", 2024-09-17T07:28:54.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8044000000000002e-05 2024-09-17T07:28:54.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.357657+0000", 2024-09-17T07:28:54.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105209 2024-09-17T07:28:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-17T07:28:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.360368+0000", 2024-09-17T07:28:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00271119 2024-09-17T07:28:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.373026+0000", 2024-09-17T07:28:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.012657677000000001 2024-09-17T07:28:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:54.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.373076+0000", 2024-09-17T07:28:54.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0003999999999998e-05 2024-09-17T07:28:54.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:54.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.373115+0000", 2024-09-17T07:28:54.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9489000000000001e-05 2024-09-17T07:28:54.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.373154+0000", 2024-09-17T07:28:54.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8043000000000002e-05 2024-09-17T07:28:54.419 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.420 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.420 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.420 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop_reply(client.4483.0:1109 3.1e e29/23 ondisk, result = 0)", 2024-09-17T07:28:54.420 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.367839+0000", 2024-09-17T07:28:54.420 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.0184540279999998, 2024-09-17T07:28:54.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0053228360000000001, 2024-09-17T07:28:54.420 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.420 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.420 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:54.420 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.421 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.367839+0000", 2024-09-17T07:28:54.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.421 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.367839+0000", 2024-09-17T07:28:54.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.367841+0000", 2024-09-17T07:28:54.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.319e-06 2024-09-17T07:28:54.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.367846+0000", 2024-09-17T07:28:54.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4149999999999998e-06 2024-09-17T07:28:54.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.367863+0000", 2024-09-17T07:28:54.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6359e-05 2024-09-17T07:28:54.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.367877+0000", 2024-09-17T07:28:54.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4691e-05 2024-09-17T07:28:54.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.373064+0000", 2024-09-17T07:28:54.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0051870120000000004 2024-09-17T07:28:54.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.373072+0000", 2024-09-17T07:28:54.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5549999999999997e-06 2024-09-17T07:28:54.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.373162+0000", 2024-09-17T07:28:54.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0484999999999998e-05 2024-09-17T07:28:54.424 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.424 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.425 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1158 3.7 3:e1042ee7:::rbd_data.1183350c2d82.0000000000000062:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:54.425 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.445922+0000", 2024-09-17T07:28:54.425 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.9403711880000003, 2024-09-17T07:28:54.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0085511760000000006, 2024-09-17T07:28:54.425 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.425 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.425 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.425 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.425 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.426 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.426 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1158 2024-09-17T07:28:54.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.426 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.426 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.426 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.445922+0000", 2024-09-17T07:28:54.426 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.426 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.426 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.445922+0000", 2024-09-17T07:28:54.426 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.427 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.427 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.445925+0000", 2024-09-17T07:28:54.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6970000000000002e-06 2024-09-17T07:28:54.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.427 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.427 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.445938+0000", 2024-09-17T07:28:54.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2907e-05 2024-09-17T07:28:54.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.427 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.445969+0000", 2024-09-17T07:28:54.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0815999999999998e-05 2024-09-17T07:28:54.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.445989+0000", 2024-09-17T07:28:54.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0205000000000001e-05 2024-09-17T07:28:54.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.446052+0000", 2024-09-17T07:28:54.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2578999999999998e-05 2024-09-17T07:28:54.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.446105+0000", 2024-09-17T07:28:54.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3947999999999999e-05 2024-09-17T07:28:54.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:28:54.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.447592+0000", 2024-09-17T07:28:54.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001486235 2024-09-17T07:28:54.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:54.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.454400+0000", 2024-09-17T07:28:54.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0068086730000000003 2024-09-17T07:28:54.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:54.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.454411+0000", 2024-09-17T07:28:54.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0166999999999999e-05 2024-09-17T07:28:54.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:54.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.454444+0000", 2024-09-17T07:28:54.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3547000000000003e-05 2024-09-17T07:28:54.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.431 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.431 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.454474+0000", 2024-09-17T07:28:54.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9402000000000001e-05 2024-09-17T07:28:54.431 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.431 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.431 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.431 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop_reply(client.4483.0:1158 3.7 e29/23 ondisk, result = 0)", 2024-09-17T07:28:54.431 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.450626+0000", 2024-09-17T07:28:54.431 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.9356673009999996, 2024-09-17T07:28:54.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003779223, 2024-09-17T07:28:54.432 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.432 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:54.432 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450626+0000", 2024-09-17T07:28:54.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450626+0000", 2024-09-17T07:28:54.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450627+0000", 2024-09-17T07:28:54.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8100000000000002e-07 2024-09-17T07:28:54.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450631+0000", 2024-09-17T07:28:54.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.941e-06 2024-09-17T07:28:54.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450646+0000", 2024-09-17T07:28:54.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5293000000000001e-05 2024-09-17T07:28:54.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.450654+0000", 2024-09-17T07:28:54.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7549999999999994e-06 2024-09-17T07:28:54.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.454390+0000", 2024-09-17T07:28:54.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037361959999999998 2024-09-17T07:28:54.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.454397+0000", 2024-09-17T07:28:54.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5130000000000002e-06 2024-09-17T07:28:54.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.454405+0000", 2024-09-17T07:28:54.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7439999999999996e-06 2024-09-17T07:28:54.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.436 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.436 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1181 3.8 3:1089ba96:::rbd_data.1183350c2d82.00000000000000e2:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:54.437 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.475326+0000", 2024-09-17T07:28:54.437 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.910967308, 2024-09-17T07:28:54.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.014074893, 2024-09-17T07:28:54.437 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.437 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.437 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.437 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.437 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1181 2024-09-17T07:28:54.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.437 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475326+0000", 2024-09-17T07:28:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475326+0000", 2024-09-17T07:28:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475327+0000", 2024-09-17T07:28:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9299999999999996e-07 2024-09-17T07:28:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475331+0000", 2024-09-17T07:28:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3189999999999996e-06 2024-09-17T07:28:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475343+0000", 2024-09-17T07:28:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1076e-05 2024-09-17T07:28:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475350+0000", 2024-09-17T07:28:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3549999999999999e-06 2024-09-17T07:28:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475403+0000", 2024-09-17T07:28:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3402999999999997e-05 2024-09-17T07:28:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475490+0000", 2024-09-17T07:28:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6638999999999995e-05 2024-09-17T07:28:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:28:54.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.477756+0000", 2024-09-17T07:28:54.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022665179999999999 2024-09-17T07:28:54.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:54.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.489269+0000", 2024-09-17T07:28:54.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011512872 2024-09-17T07:28:54.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:54.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.489291+0000", 2024-09-17T07:28:54.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1574999999999999e-05 2024-09-17T07:28:54.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:54.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.489349+0000", 2024-09-17T07:28:54.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8304000000000003e-05 2024-09-17T07:28:54.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.489401+0000", 2024-09-17T07:28:54.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1938999999999999e-05 2024-09-17T07:28:54.442 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.442 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.442 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.442 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1591 3.1e 3:7cba8a19:::rbd_data.11832b928878.00000000000002eb:head [set-alloc-hint object_size 262144 write_size 262144,write 103318~27754] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:54.442 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.660750+0000", 2024-09-17T07:28:54.443 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.7255433020000002, 2024-09-17T07:28:54.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0042241600000000002, 2024-09-17T07:28:54.443 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.443 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.443 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.443 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.443 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.443 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1591 2024-09-17T07:28:54.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.443 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660750+0000", 2024-09-17T07:28:54.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.444 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660750+0000", 2024-09-17T07:28:54.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.444 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660752+0000", 2024-09-17T07:28:54.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.204e-06 2024-09-17T07:28:54.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.444 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660790+0000", 2024-09-17T07:28:54.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7953000000000001e-05 2024-09-17T07:28:54.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660807+0000", 2024-09-17T07:28:54.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6719e-05 2024-09-17T07:28:54.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660814+0000", 2024-09-17T07:28:54.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1230000000000002e-06 2024-09-17T07:28:54.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660866+0000", 2024-09-17T07:28:54.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1307000000000002e-05 2024-09-17T07:28:54.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.660951+0000", 2024-09-17T07:28:54.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5558999999999999e-05 2024-09-17T07:28:54.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-17T07:28:54.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.661049+0000", 2024-09-17T07:28:54.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7912999999999996e-05 2024-09-17T07:28:54.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:54.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.661729+0000", 2024-09-17T07:28:54.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00068000999999999999 2024-09-17T07:28:54.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:54.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.664912+0000", 2024-09-17T07:28:54.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031832560000000002 2024-09-17T07:28:54.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:54.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.664946+0000", 2024-09-17T07:28:54.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3253999999999997e-05 2024-09-17T07:28:54.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.664974+0000", 2024-09-17T07:28:54.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8861999999999999e-05 2024-09-17T07:28:54.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.448 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.448 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1804 3.2 3:45c0a2cc:::rbd_data.1183350c2d82.000000000000005e:4 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:54.449 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.796094+0000", 2024-09-17T07:28:54.449 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.590199975, 2024-09-17T07:28:54.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0032921600000000001, 2024-09-17T07:28:54.449 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.449 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:54.449 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.449 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.449 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.449 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1804 2024-09-17T07:28:54.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.449 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796094+0000", 2024-09-17T07:28:54.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796094+0000", 2024-09-17T07:28:54.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796094+0000", 2024-09-17T07:28:54.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1099999999999996e-07 2024-09-17T07:28:54.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796097+0000", 2024-09-17T07:28:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:28:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796103+0000", 2024-09-17T07:28:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6669999999999999e-06 2024-09-17T07:28:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796108+0000", 2024-09-17T07:28:54.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:28:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.799186+0000", 2024-09-17T07:28:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00307764 2024-09-17T07:28:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.799271+0000", 2024-09-17T07:28:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5445999999999995e-05 2024-09-17T07:28:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.799386+0000", 2024-09-17T07:28:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011461 2024-09-17T07:28:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1805 3.2 3:45c0a2cc:::rbd_data.1183350c2d82.000000000000005e:4 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.796116+0000", 2024-09-17T07:28:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.5901778090000001, 2024-09-17T07:28:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0034149010000000001, 2024-09-17T07:28:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.453 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1805 2024-09-17T07:28:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796116+0000", 2024-09-17T07:28:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796116+0000", 2024-09-17T07:28:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796116+0000", 2024-09-17T07:28:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3799999999999998e-07 2024-09-17T07:28:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796118+0000", 2024-09-17T07:28:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:28:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796125+0000", 2024-09-17T07:28:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6069999999999996e-06 2024-09-17T07:28:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.796130+0000", 2024-09-17T07:28:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.22e-06 2024-09-17T07:28:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.799416+0000", 2024-09-17T07:28:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0032860229999999999 2024-09-17T07:28:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.799461+0000", 2024-09-17T07:28:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5120000000000002e-05 2024-09-17T07:28:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.799531+0000", 2024-09-17T07:28:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9307000000000006e-05 2024-09-17T07:28:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.457 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:2119 3.7 3:e4d85cad:::rbd_data.11832b928878.0000000000000126:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:51.020066+0000", 2024-09-17T07:28:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.36622763, 2024-09-17T07:28:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.018189549999999999, 2024-09-17T07:28:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 2119 2024-09-17T07:28:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.020066+0000", 2024-09-17T07:28:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.020066+0000", 2024-09-17T07:28:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.020067+0000", 2024-09-17T07:28:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.55e-06 2024-09-17T07:28:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.020075+0000", 2024-09-17T07:28:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.374e-06 2024-09-17T07:28:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.020089+0000", 2024-09-17T07:28:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4345000000000001e-05 2024-09-17T07:28:54.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.020096+0000", 2024-09-17T07:28:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8480000000000003e-06 2024-09-17T07:28:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.020148+0000", 2024-09-17T07:28:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2235999999999999e-05 2024-09-17T07:28:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.020246+0000", 2024-09-17T07:28:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8022000000000004e-05 2024-09-17T07:28:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:28:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.023435+0000", 2024-09-17T07:28:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003188841 2024-09-17T07:28:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.038183+0000", 2024-09-17T07:28:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.014747547 2024-09-17T07:28:54.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.038193+0000", 2024-09-17T07:28:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0414999999999999e-05 2024-09-17T07:28:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.038226+0000", 2024-09-17T07:28:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2994999999999998e-05 2024-09-17T07:28:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.038255+0000", 2024-09-17T07:28:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9377e-05 2024-09-17T07:28:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:2120 3.8 3:15efcdbd:::rbd_data.11832b928878.0000000000000127:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:51.021611+0000", 2024-09-17T07:28:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.3646821870000001, 2024-09-17T07:28:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010540898999999999, 2024-09-17T07:28:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 2120 2024-09-17T07:28:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.021611+0000", 2024-09-17T07:28:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.021611+0000", 2024-09-17T07:28:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.021612+0000", 2024-09-17T07:28:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5700000000000002e-07 2024-09-17T07:28:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.021617+0000", 2024-09-17T07:28:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4100000000000001e-06 2024-09-17T07:28:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.021628+0000", 2024-09-17T07:28:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1019000000000001e-05 2024-09-17T07:28:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.021632+0000", 2024-09-17T07:28:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9479999999999999e-06 2024-09-17T07:28:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.021682+0000", 2024-09-17T07:28:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9552999999999998e-05 2024-09-17T07:28:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.021793+0000", 2024-09-17T07:28:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111128 2024-09-17T07:28:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:28:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.023844+0000", 2024-09-17T07:28:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020505319999999999 2024-09-17T07:28:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.032019+0000", 2024-09-17T07:28:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0081756490000000001 2024-09-17T07:28:54.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.032040+0000", 2024-09-17T07:28:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0188999999999999e-05 2024-09-17T07:28:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.032099+0000", 2024-09-17T07:28:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9895000000000002e-05 2024-09-17T07:28:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.032152+0000", 2024-09-17T07:28:54.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2818999999999997e-05 2024-09-17T07:28:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop_reply(client.4483.0:2120 3.8 e30/23 ondisk, result = 0)", 2024-09-17T07:28:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:51.028160+0000", 2024-09-17T07:28:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.358133703, 2024-09-17T07:28:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0038698970000000002, 2024-09-17T07:28:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:54.470 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.028160+0000", 2024-09-17T07:28:54.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.028160+0000", 2024-09-17T07:28:54.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.028161+0000", 2024-09-17T07:28:54.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9800000000000002e-07 2024-09-17T07:28:54.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.028165+0000", 2024-09-17T07:28:54.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9389999999999999e-06 2024-09-17T07:28:54.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.028175+0000", 2024-09-17T07:28:54.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.049e-05 2024-09-17T07:28:54.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.028181+0000", 2024-09-17T07:28:54.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9399999999999999e-06 2024-09-17T07:28:54.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.032001+0000", 2024-09-17T07:28:54.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0038201620000000002 2024-09-17T07:28:54.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.032011+0000", 2024-09-17T07:28:54.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0209e-05 2024-09-17T07:28:54.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.474 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.474 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.032030+0000", 2024-09-17T07:28:54.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8258999999999999e-05 2024-09-17T07:28:54.474 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.474 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.474 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.474 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop_reply(client.4483.0:2119 3.7 e30/23 ondisk, result = 0)", 2024-09-17T07:28:54.474 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:51.032728+0000", 2024-09-17T07:28:54.474 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.3535655009999998, 2024-09-17T07:28:54.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0054597020000000003, 2024-09-17T07:28:54.474 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.032728+0000", 2024-09-17T07:28:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.032728+0000", 2024-09-17T07:28:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.032729+0000", 2024-09-17T07:28:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0500000000000003e-07 2024-09-17T07:28:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.032733+0000", 2024-09-17T07:28:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0790000000000002e-06 2024-09-17T07:28:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.032743+0000", 2024-09-17T07:28:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8989999999999996e-06 2024-09-17T07:28:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.032748+0000", 2024-09-17T07:28:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4060000000000004e-06 2024-09-17T07:28:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.038171+0000", 2024-09-17T07:28:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0054230550000000004 2024-09-17T07:28:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.038179+0000", 2024-09-17T07:28:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3880000000000002e-06 2024-09-17T07:28:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.038188+0000", 2024-09-17T07:28:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1700000000000003e-06 2024-09-17T07:28:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:4200 3.2 3:41bd54bc:::rbd_data.1183cff49e3a.0000000000000006:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:28:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.211567+0000", 2024-09-17T07:28:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.17472639000000001, 2024-09-17T07:28:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041988110000000002, 2024-09-17T07:28:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:28:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 4200 2024-09-17T07:28:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.211567+0000", 2024-09-17T07:28:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.211567+0000", 2024-09-17T07:28:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.211568+0000", 2024-09-17T07:28:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4600000000000004e-07 2024-09-17T07:28:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.211572+0000", 2024-09-17T07:28:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9090000000000002e-06 2024-09-17T07:28:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.211582+0000", 2024-09-17T07:28:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0161e-05 2024-09-17T07:28:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.211588+0000", 2024-09-17T07:28:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1079999999999998e-06 2024-09-17T07:28:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.211838+0000", 2024-09-17T07:28:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00025037799999999999 2024-09-17T07:28:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.211930+0000", 2024-09-17T07:28:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1365000000000004e-05 2024-09-17T07:28:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.215766+0000", 2024-09-17T07:28:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0038362439999999999 2024-09-17T07:28:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:4916 3.1e 3:7ddfe06e:::rbd_data.1183cff49e3a.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:28:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.249645+0000", 2024-09-17T07:28:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.136648558, 2024-09-17T07:28:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0032154570000000001, 2024-09-17T07:28:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 4916 2024-09-17T07:28:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.249645+0000", 2024-09-17T07:28:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.249645+0000", 2024-09-17T07:28:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.249647+0000", 2024-09-17T07:28:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0159999999999998e-06 2024-09-17T07:28:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.249992+0000", 2024-09-17T07:28:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00034521399999999999 2024-09-17T07:28:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.250010+0000", 2024-09-17T07:28:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8275999999999999e-05 2024-09-17T07:28:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.250019+0000", 2024-09-17T07:28:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7830000000000004e-06 2024-09-17T07:28:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.250083+0000", 2024-09-17T07:28:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3837000000000006e-05 2024-09-17T07:28:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.250157+0000", 2024-09-17T07:28:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3932999999999998e-05 2024-09-17T07:28:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-17T07:28:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.250311+0000", 2024-09-17T07:28:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015411299999999999 2024-09-17T07:28:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.252110+0000", 2024-09-17T07:28:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017993289999999999 2024-09-17T07:28:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.252796+0000", 2024-09-17T07:28:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00068575899999999998 2024-09-17T07:28:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.252831+0000", 2024-09-17T07:28:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4918e-05 2024-09-17T07:28:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.252860+0000", 2024-09-17T07:28:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9278999999999999e-05 2024-09-17T07:28:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:4984 3.2 3:40263d16:::rbd_data.1183cff49e3a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:28:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.260614+0000", 2024-09-17T07:28:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.125679611, 2024-09-17T07:28:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0034804900000000001, 2024-09-17T07:28:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 4984 2024-09-17T07:28:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.260614+0000", 2024-09-17T07:28:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.260614+0000", 2024-09-17T07:28:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.260616+0000", 2024-09-17T07:28:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:28:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.260790+0000", 2024-09-17T07:28:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017472700000000001 2024-09-17T07:28:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.260807+0000", 2024-09-17T07:28:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6518e-05 2024-09-17T07:28:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.260816+0000", 2024-09-17T07:28:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4870000000000008e-06 2024-09-17T07:28:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.260888+0000", 2024-09-17T07:28:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1792999999999995e-05 2024-09-17T07:28:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.260950+0000", 2024-09-17T07:28:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1644999999999997e-05 2024-09-17T07:28:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:28:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.261119+0000", 2024-09-17T07:28:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00016934799999999999 2024-09-17T07:28:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.263087+0000", 2024-09-17T07:28:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019677879999999998 2024-09-17T07:28:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.263985+0000", 2024-09-17T07:28:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089783500000000004 2024-09-17T07:28:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.264046+0000", 2024-09-17T07:28:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1338000000000001e-05 2024-09-17T07:28:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.264094+0000", 2024-09-17T07:28:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8318000000000001e-05 2024-09-17T07:28:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:4985 3.2 3:40263d16:::rbd_data.1183cff49e3a.0000000000000000:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:28:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.260893+0000", 2024-09-17T07:28:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.12540024299999999, 2024-09-17T07:28:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0049469869999999999, 2024-09-17T07:28:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 4985 2024-09-17T07:28:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.260893+0000", 2024-09-17T07:28:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.260893+0000", 2024-09-17T07:28:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.260894+0000", 2024-09-17T07:28:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.048e-06 2024-09-17T07:28:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.260899+0000", 2024-09-17T07:28:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9710000000000003e-06 2024-09-17T07:28:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.260911+0000", 2024-09-17T07:28:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1838000000000001e-05 2024-09-17T07:28:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.260920+0000", 2024-09-17T07:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9190000000000005e-06 2024-09-17T07:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.261850+0000", 2024-09-17T07:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093013000000000004 2024-09-17T07:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.261977+0000", 2024-09-17T07:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000127121 2024-09-17T07:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:28:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.262172+0000", 2024-09-17T07:28:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00019479999999999999 2024-09-17T07:28:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.263428+0000", 2024-09-17T07:28:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001256339 2024-09-17T07:28:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.265750+0000", 2024-09-17T07:28:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023217709999999998 2024-09-17T07:28:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.265800+0000", 2024-09-17T07:28:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0059999999999998e-05 2024-09-17T07:28:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.265840+0000", 2024-09-17T07:28:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9990000000000002e-05 2024-09-17T07:28:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:54.500 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:54.502 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:28:54.502 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:54.500+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:54.603 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:54.602+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:54.633 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:54.634 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:28:54.634 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:28:54.634 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:54.643 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:28:54.703 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:54.702+0000 7f9a047b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33358) UID: 0 2024-09-17T07:28:54.804 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:28:54.803+0000 7f9a047b6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33358) UID: 0 2024-09-17T07:28:54.820 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:54.820 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:28:54.820 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:28:54.820 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:28:54.820 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:54.832 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:28:54.905 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:54.904+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:54.962 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:28:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:28:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:28:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-17T07:28:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:45.078157+0000", 2024-09-17T07:28:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8821590829999995, 2024-09-17T07:28:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046274630000000001, 2024-09-17T07:28:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 9 2024-09-17T07:28:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:45.078157+0000", 2024-09-17T07:28:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:45.078157+0000", 2024-09-17T07:28:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:45.078159+0000", 2024-09-17T07:28:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.637e-06 2024-09-17T07:28:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:45.078165+0000", 2024-09-17T07:28:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2129999999999996e-06 2024-09-17T07:28:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:45.078180+0000", 2024-09-17T07:28:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4997000000000001e-05 2024-09-17T07:28:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:45.078188+0000", 2024-09-17T07:28:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8509999999999999e-06 2024-09-17T07:28:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:45.078244+0000", 2024-09-17T07:28:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6718999999999997e-05 2024-09-17T07:28:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-17T07:28:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:45.078251+0000", 2024-09-17T07:28:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8310000000000003e-06 2024-09-17T07:28:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:45.080139+0000", 2024-09-17T07:28:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001887512 2024-09-17T07:28:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:45.080193+0000", 2024-09-17T07:28:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4054000000000001e-05 2024-09-17T07:28:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-17T07:28:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:45.080278+0000", 2024-09-17T07:28:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5284999999999994e-05 2024-09-17T07:28:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:45.081416+0000", 2024-09-17T07:28:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011380489999999999 2024-09-17T07:28:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:45.082675+0000", 2024-09-17T07:28:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0012587080000000001 2024-09-17T07:28:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:45.082736+0000", 2024-09-17T07:28:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1568000000000004e-05 2024-09-17T07:28:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:45.082784+0000", 2024-09-17T07:28:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8038999999999999e-05 2024-09-17T07:28:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4483.0:942 3.13 e29/23 3:cd1016d0:::rbd_object_map.1183350c2d82:head v 29'176, mlcod=29'173)", 2024-09-17T07:28:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:49.155719+0000", 2024-09-17T07:28:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8045972990000001, 2024-09-17T07:28:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.005070995, 2024-09-17T07:28:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.155719+0000", 2024-09-17T07:28:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.155719+0000", 2024-09-17T07:28:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.155719+0000", 2024-09-17T07:28:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5300000000000004e-07 2024-09-17T07:28:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.155724+0000", 2024-09-17T07:28:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.206e-06 2024-09-17T07:28:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.155735+0000", 2024-09-17T07:28:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1894e-05 2024-09-17T07:28:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.155742+0000", 2024-09-17T07:28:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4150000000000004e-06 2024-09-17T07:28:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.155778+0000", 2024-09-17T07:28:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6433000000000002e-05 2024-09-17T07:28:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.155796+0000", 2024-09-17T07:28:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7839999999999999e-05 2024-09-17T07:28:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.160757+0000", 2024-09-17T07:28:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049610349999999999 2024-09-17T07:28:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.160790+0000", 2024-09-17T07:28:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2619000000000001e-05 2024-09-17T07:28:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4483.0:1151 3.13 e29/23 3:cd1016d0:::rbd_object_map.1183350c2d82:head v 29'230, mlcod=29'227)", 2024-09-17T07:28:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:49.435699+0000", 2024-09-17T07:28:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.5246166370000003, 2024-09-17T07:28:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.005345252, 2024-09-17T07:28:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.435699+0000", 2024-09-17T07:28:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.435699+0000", 2024-09-17T07:28:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.435700+0000", 2024-09-17T07:28:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.37e-07 2024-09-17T07:28:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.435706+0000", 2024-09-17T07:28:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7200000000000003e-06 2024-09-17T07:28:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.435717+0000", 2024-09-17T07:28:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1182999999999999e-05 2024-09-17T07:28:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.435725+0000", 2024-09-17T07:28:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0050000000000004e-06 2024-09-17T07:28:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.435765+0000", 2024-09-17T07:28:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0263999999999999e-05 2024-09-17T07:28:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.435777+0000", 2024-09-17T07:28:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.173e-05 2024-09-17T07:28:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.441022+0000", 2024-09-17T07:28:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052453930000000001 2024-09-17T07:28:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.441045+0000", 2024-09-17T07:28:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2220000000000001e-05 2024-09-17T07:28:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4483.0:1152 3.13 e29/23 3:cd1016d0:::rbd_object_map.1183350c2d82:head v 29'231, mlcod=29'227)", 2024-09-17T07:28:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:49.436281+0000", 2024-09-17T07:28:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.5240353799999999, 2024-09-17T07:28:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047827939999999999, 2024-09-17T07:28:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.436281+0000", 2024-09-17T07:28:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.436281+0000", 2024-09-17T07:28:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.436281+0000", 2024-09-17T07:28:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.92e-07 2024-09-17T07:28:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.436286+0000", 2024-09-17T07:28:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5859999999999998e-06 2024-09-17T07:28:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.436296+0000", 2024-09-17T07:28:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6930000000000002e-06 2024-09-17T07:28:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.436300+0000", 2024-09-17T07:28:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4399999999999998e-06 2024-09-17T07:28:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.436340+0000", 2024-09-17T07:28:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9570000000000002e-05 2024-09-17T07:28:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.436357+0000", 2024-09-17T07:28:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7357e-05 2024-09-17T07:28:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.441046+0000", 2024-09-17T07:28:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046885759999999999 2024-09-17T07:28:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.441063+0000", 2024-09-17T07:28:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7880000000000002e-05 2024-09-17T07:28:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:1505 3.9 3:912d8610:::rbd_data.11832b928878.000000000000023c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:50.580828+0000", 2024-09-17T07:28:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.3794877799999998, 2024-09-17T07:28:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.005055337, 2024-09-17T07:28:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1505 2024-09-17T07:28:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580828+0000", 2024-09-17T07:28:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580828+0000", 2024-09-17T07:28:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580829+0000", 2024-09-17T07:28:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7199999999999997e-07 2024-09-17T07:28:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580834+0000", 2024-09-17T07:28:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2690000000000001e-06 2024-09-17T07:28:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580847+0000", 2024-09-17T07:28:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3210000000000001e-05 2024-09-17T07:28:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580854+0000", 2024-09-17T07:28:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4000000000000003e-06 2024-09-17T07:28:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580907+0000", 2024-09-17T07:28:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3341999999999999e-05 2024-09-17T07:28:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580997+0000", 2024-09-17T07:28:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9482999999999997e-05 2024-09-17T07:28:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-17T07:28:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.581102+0000", 2024-09-17T07:28:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00010466700000000001 2024-09-17T07:28:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.582045+0000", 2024-09-17T07:28:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00094332499999999996 2024-09-17T07:28:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.585793+0000", 2024-09-17T07:28:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037478020000000002 2024-09-17T07:28:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.585843+0000", 2024-09-17T07:28:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0609999999999998e-05 2024-09-17T07:28:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.585884+0000", 2024-09-17T07:28:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0256999999999998e-05 2024-09-17T07:28:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4483.0:1503 3.1d e30/23 3:bb76ae33:::rbd_data.11832b928878.000000000000023a:head v 30'14, mlcod=29'12)", 2024-09-17T07:28:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:50.581059+0000", 2024-09-17T07:28:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.379256614, 2024-09-17T07:28:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047450909999999999, 2024-09-17T07:28:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.581059+0000", 2024-09-17T07:28:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.581059+0000", 2024-09-17T07:28:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.581060+0000", 2024-09-17T07:28:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5199999999999997e-07 2024-09-17T07:28:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.581064+0000", 2024-09-17T07:28:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.104e-06 2024-09-17T07:28:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.581074+0000", 2024-09-17T07:28:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5200000000000003e-06 2024-09-17T07:28:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.581080+0000", 2024-09-17T07:28:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0379999999999996e-06 2024-09-17T07:28:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.581127+0000", 2024-09-17T07:28:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7094e-05 2024-09-17T07:28:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.581145+0000", 2024-09-17T07:28:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8150000000000001e-05 2024-09-17T07:28:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.585774+0000", 2024-09-17T07:28:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046293009999999997 2024-09-17T07:28:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.585805+0000", 2024-09-17T07:28:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0332e-05 2024-09-17T07:28:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:1578 3.3 3:c31e86cc:::rbd_data.11832b928878.00000000000001a3:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:50.639745+0000", 2024-09-17T07:28:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.320571138, 2024-09-17T07:28:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072301709999999996, 2024-09-17T07:28:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1578 2024-09-17T07:28:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639745+0000", 2024-09-17T07:28:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639745+0000", 2024-09-17T07:28:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639746+0000", 2024-09-17T07:28:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9500000000000001e-07 2024-09-17T07:28:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639750+0000", 2024-09-17T07:28:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9840000000000003e-06 2024-09-17T07:28:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639760+0000", 2024-09-17T07:28:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7829999999999993e-06 2024-09-17T07:28:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639765+0000", 2024-09-17T07:28:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.977e-06 2024-09-17T07:28:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639805+0000", 2024-09-17T07:28:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0414999999999998e-05 2024-09-17T07:28:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639892+0000", 2024-09-17T07:28:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6717000000000003e-05 2024-09-17T07:28:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:28:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.641224+0000", 2024-09-17T07:28:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00133264 2024-09-17T07:28:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.646861+0000", 2024-09-17T07:28:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0056365310000000002 2024-09-17T07:28:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.646892+0000", 2024-09-17T07:28:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0760999999999999e-05 2024-09-17T07:28:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.646929+0000", 2024-09-17T07:28:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6909999999999997e-05 2024-09-17T07:28:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.646975+0000", 2024-09-17T07:28:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6557999999999997e-05 2024-09-17T07:28:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:1628 3.9 3:96c3871d:::rbd_data.11832b928878.000000000000037c:head [set-alloc-hint object_size 262144 write_size 262144,write 104010~27062] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:50.692084+0000", 2024-09-17T07:28:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.2682321349999999, 2024-09-17T07:28:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0048456769999999996, 2024-09-17T07:28:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1628 2024-09-17T07:28:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.692084+0000", 2024-09-17T07:28:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.692084+0000", 2024-09-17T07:28:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.692085+0000", 2024-09-17T07:28:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4900000000000005e-07 2024-09-17T07:28:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.692101+0000", 2024-09-17T07:28:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6232e-05 2024-09-17T07:28:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.692113+0000", 2024-09-17T07:28:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1983e-05 2024-09-17T07:28:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.692118+0000", 2024-09-17T07:28:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0660000000000003e-06 2024-09-17T07:28:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.692160+0000", 2024-09-17T07:28:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2341e-05 2024-09-17T07:28:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.692239+0000", 2024-09-17T07:28:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8579999999999996e-05 2024-09-17T07:28:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-17T07:28:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.692321+0000", 2024-09-17T07:28:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2038000000000003e-05 2024-09-17T07:28:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.693402+0000", 2024-09-17T07:28:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010810419999999999 2024-09-17T07:28:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.696871+0000", 2024-09-17T07:28:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034693430000000002 2024-09-17T07:28:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.696905+0000", 2024-09-17T07:28:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188999999999997e-05 2024-09-17T07:28:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.696930+0000", 2024-09-17T07:28:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5014000000000001e-05 2024-09-17T07:28:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4483.0:1648 3.1c e30/23 3:3fc76693:::rbd_data.11832b928878.00000000000002ed:head v 30'16, mlcod=30'14)", 2024-09-17T07:28:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:50.712769+0000", 2024-09-17T07:28:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.2475472329999997, 2024-09-17T07:28:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046650210000000001, 2024-09-17T07:28:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712769+0000", 2024-09-17T07:28:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712769+0000", 2024-09-17T07:28:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712770+0000", 2024-09-17T07:28:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2799999999999995e-07 2024-09-17T07:28:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712777+0000", 2024-09-17T07:28:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:28:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712787+0000", 2024-09-17T07:28:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0149000000000001e-05 2024-09-17T07:28:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712794+0000", 2024-09-17T07:28:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7410000000000003e-06 2024-09-17T07:28:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712848+0000", 2024-09-17T07:28:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3414000000000001e-05 2024-09-17T07:28:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712866+0000", 2024-09-17T07:28:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8241999999999999e-05 2024-09-17T07:28:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.717391+0000", 2024-09-17T07:28:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045256869999999996 2024-09-17T07:28:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.717434+0000", 2024-09-17T07:28:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410000000000002e-05 2024-09-17T07:28:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4483.0:1647 3.f e30/23 3:f6f7612d:::rbd_data.11832b928878.00000000000002ec:head v 30'2, mlcod=0'0)", 2024-09-17T07:28:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:50.712822+0000", 2024-09-17T07:28:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.2474936090000002, 2024-09-17T07:28:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046231479999999997, 2024-09-17T07:28:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712822+0000", 2024-09-17T07:28:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712822+0000", 2024-09-17T07:28:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712823+0000", 2024-09-17T07:28:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1900000000000002e-07 2024-09-17T07:28:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712828+0000", 2024-09-17T07:28:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:28:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712836+0000", 2024-09-17T07:28:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3890000000000005e-06 2024-09-17T07:28:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712841+0000", 2024-09-17T07:28:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0780000000000003e-06 2024-09-17T07:28:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712896+0000", 2024-09-17T07:28:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4273e-05 2024-09-17T07:28:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712915+0000", 2024-09-17T07:28:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9592000000000001e-05 2024-09-17T07:28:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.717405+0000", 2024-09-17T07:28:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044898730000000001 2024-09-17T07:28:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.717446+0000", 2024-09-17T07:28:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0521e-05 2024-09-17T07:28:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4483.0:1688 3.11 e30/23 3:8d3d6fbc:::rbd_object_map.11832b928878:head v 30'99, mlcod=30'95)", 2024-09-17T07:28:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:50.728948+0000", 2024-09-17T07:28:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.2313677050000003, 2024-09-17T07:28:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0048479100000000004, 2024-09-17T07:28:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.728948+0000", 2024-09-17T07:28:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.728948+0000", 2024-09-17T07:28:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.728949+0000", 2024-09-17T07:28:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5899999999999996e-07 2024-09-17T07:28:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.728955+0000", 2024-09-17T07:28:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8449999999999999e-06 2024-09-17T07:28:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.728963+0000", 2024-09-17T07:28:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.225e-06 2024-09-17T07:28:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.728968+0000", 2024-09-17T07:28:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.074e-06 2024-09-17T07:28:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.729014+0000", 2024-09-17T07:28:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5911000000000001e-05 2024-09-17T07:28:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.729034+0000", 2024-09-17T07:28:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9564000000000001e-05 2024-09-17T07:28:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.733767+0000", 2024-09-17T07:28:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0047338809999999997 2024-09-17T07:28:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.733796+0000", 2024-09-17T07:28:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8850999999999999e-05 2024-09-17T07:28:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:1694 3.1a 3:5801d5f8:::rbd_data.11832b928878.00000000000001f4:head [set-alloc-hint object_size 262144 write_size 262144,write 175228~21380] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:50.737176+0000", 2024-09-17T07:28:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.2231398240000004, 2024-09-17T07:28:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045461859999999998, 2024-09-17T07:28:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1694 2024-09-17T07:28:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.737176+0000", 2024-09-17T07:28:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.737176+0000", 2024-09-17T07:28:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.737182+0000", 2024-09-17T07:28:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4949999999999999e-06 2024-09-17T07:28:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.737200+0000", 2024-09-17T07:28:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8019000000000001e-05 2024-09-17T07:28:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.737211+0000", 2024-09-17T07:28:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1633e-05 2024-09-17T07:28:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.737216+0000", 2024-09-17T07:28:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8770000000000001e-06 2024-09-17T07:28:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.737260+0000", 2024-09-17T07:28:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3377999999999997e-05 2024-09-17T07:28:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.737336+0000", 2024-09-17T07:28:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6429999999999998e-05 2024-09-17T07:28:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-17T07:28:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.737419+0000", 2024-09-17T07:28:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3207000000000005e-05 2024-09-17T07:28:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.738416+0000", 2024-09-17T07:28:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00099653299999999997 2024-09-17T07:28:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.741664+0000", 2024-09-17T07:28:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0032483429999999999 2024-09-17T07:28:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.741698+0000", 2024-09-17T07:28:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3674000000000003e-05 2024-09-17T07:28:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.741722+0000", 2024-09-17T07:28:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4596999999999998e-05 2024-09-17T07:28:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4483.0:1695 3.1 e30/23 3:87cc7aab:::rbd_data.11832b928878.00000000000001f5:head v 30'8, mlcod=30'6)", 2024-09-17T07:28:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:50.739683+0000", 2024-09-17T07:28:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.2206332900000003, 2024-09-17T07:28:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0048221740000000003, 2024-09-17T07:28:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.739683+0000", 2024-09-17T07:28:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.739683+0000", 2024-09-17T07:28:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.739683+0000", 2024-09-17T07:28:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3900000000000004e-07 2024-09-17T07:28:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.739696+0000", 2024-09-17T07:28:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2451e-05 2024-09-17T07:28:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.739705+0000", 2024-09-17T07:28:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4690000000000003e-06 2024-09-17T07:28:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.739710+0000", 2024-09-17T07:28:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4120000000000003e-06 2024-09-17T07:28:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.739752+0000", 2024-09-17T07:28:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2667999999999999e-05 2024-09-17T07:28:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.739766+0000", 2024-09-17T07:28:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3608000000000001e-05 2024-09-17T07:28:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.744391+0000", 2024-09-17T07:28:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046251529999999999 2024-09-17T07:28:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.744505+0000", 2024-09-17T07:28:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113774 2024-09-17T07:28:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4483.0:1898 3.11 e30/23 3:8d3d6fbc:::rbd_object_map.11832b928878:head v 30'134, mlcod=30'132)", 2024-09-17T07:28:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:50.843210+0000", 2024-09-17T07:28:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.1171058130000002, 2024-09-17T07:28:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045484480000000001, 2024-09-17T07:28:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.843210+0000", 2024-09-17T07:28:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.843210+0000", 2024-09-17T07:28:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.843211+0000", 2024-09-17T07:28:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000005e-07 2024-09-17T07:28:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.843217+0000", 2024-09-17T07:28:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5609999999999998e-06 2024-09-17T07:28:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.843228+0000", 2024-09-17T07:28:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1018e-05 2024-09-17T07:28:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.843234+0000", 2024-09-17T07:28:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5089999999999999e-06 2024-09-17T07:28:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.843317+0000", 2024-09-17T07:28:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2538000000000002e-05 2024-09-17T07:28:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.843333+0000", 2024-09-17T07:28:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6563e-05 2024-09-17T07:28:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.847711+0000", 2024-09-17T07:28:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0043778480000000002 2024-09-17T07:28:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.847759+0000", 2024-09-17T07:28:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7661999999999999e-05 2024-09-17T07:28:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4483.0:2206 3.1b e30/23 3:dda19077:::rbd_data.11832b928878.0000000000000131:head v 30'80, mlcod=30'77)", 2024-09-17T07:28:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:51.143670+0000", 2024-09-17T07:28:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.8166456260000001, 2024-09-17T07:28:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046124820000000002, 2024-09-17T07:28:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.143670+0000", 2024-09-17T07:28:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.143670+0000", 2024-09-17T07:28:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.143671+0000", 2024-09-17T07:28:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0299999999999998e-07 2024-09-17T07:28:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.143676+0000", 2024-09-17T07:28:55.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9490000000000004e-06 2024-09-17T07:28:55.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.143684+0000", 2024-09-17T07:28:55.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1410000000000005e-06 2024-09-17T07:28:55.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.143690+0000", 2024-09-17T07:28:55.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.879e-06 2024-09-17T07:28:55.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.143741+0000", 2024-09-17T07:28:55.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1369000000000002e-05 2024-09-17T07:28:55.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.143760+0000", 2024-09-17T07:28:55.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8822999999999999e-05 2024-09-17T07:28:55.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.148217+0000", 2024-09-17T07:28:55.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044562109999999999 2024-09-17T07:28:55.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.148283+0000", 2024-09-17T07:28:55.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6407000000000003e-05 2024-09-17T07:28:55.057 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.057 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.057 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.057 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:6683 3.9 3:97ddc208:::rbd_data.1183cff49e3a.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:28:55.057 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.763526+0000", 2024-09-17T07:28:55.058 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.19679041799999999, 2024-09-17T07:28:55.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032433677000000001, 2024-09-17T07:28:55.058 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.058 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.058 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.058 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6683 2024-09-17T07:28:55.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.058 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.763526+0000", 2024-09-17T07:28:55.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.763526+0000", 2024-09-17T07:28:55.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.763527+0000", 2024-09-17T07:28:55.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.78e-07 2024-09-17T07:28:55.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764048+0000", 2024-09-17T07:28:55.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00052137799999999997 2024-09-17T07:28:55.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764060+0000", 2024-09-17T07:28:55.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2014000000000001e-05 2024-09-17T07:28:55.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764072+0000", 2024-09-17T07:28:55.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1762e-05 2024-09-17T07:28:55.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764130+0000", 2024-09-17T07:28:55.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7759e-05 2024-09-17T07:28:55.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764194+0000", 2024-09-17T07:28:55.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4189999999999994e-05 2024-09-17T07:28:55.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-17T07:28:55.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764332+0000", 2024-09-17T07:28:55.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013821300000000001 2024-09-17T07:28:55.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795822+0000", 2024-09-17T07:28:55.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031489847000000001 2024-09-17T07:28:55.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795858+0000", 2024-09-17T07:28:55.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6452999999999998e-05 2024-09-17T07:28:55.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795917+0000", 2024-09-17T07:28:55.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8851999999999998e-05 2024-09-17T07:28:55.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795959+0000", 2024-09-17T07:28:55.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2231000000000003e-05 2024-09-17T07:28:55.063 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.063 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.063 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.063 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:6684 3.9 3:97ddc208:::rbd_data.1183cff49e3a.000000000000001e:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:28:55.063 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.764091+0000", 2024-09-17T07:28:55.063 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.196225235, 2024-09-17T07:28:55.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032300207999999997, 2024-09-17T07:28:55.063 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.064 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.064 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.064 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.064 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.064 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.064 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6684 2024-09-17T07:28:55.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.064 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764091+0000", 2024-09-17T07:28:55.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764091+0000", 2024-09-17T07:28:55.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764092+0000", 2024-09-17T07:28:55.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8700000000000005e-07 2024-09-17T07:28:55.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764094+0000", 2024-09-17T07:28:55.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9189999999999999e-06 2024-09-17T07:28:55.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764102+0000", 2024-09-17T07:28:55.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2710000000000003e-06 2024-09-17T07:28:55.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764118+0000", 2024-09-17T07:28:55.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6231000000000001e-05 2024-09-17T07:28:55.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764995+0000", 2024-09-17T07:28:55.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00087753699999999996 2024-09-17T07:28:55.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.765094+0000", 2024-09-17T07:28:55.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8027000000000002e-05 2024-09-17T07:28:55.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-17T07:28:55.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794998+0000", 2024-09-17T07:28:55.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029904612000000001 2024-09-17T07:28:55.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796267+0000", 2024-09-17T07:28:55.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001268911 2024-09-17T07:28:55.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796328+0000", 2024-09-17T07:28:55.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0594000000000003e-05 2024-09-17T07:28:55.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796356+0000", 2024-09-17T07:28:55.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8787999999999999e-05 2024-09-17T07:28:55.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796391+0000", 2024-09-17T07:28:55.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4631000000000001e-05 2024-09-17T07:28:55.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.069 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.069 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4483.0:6683 3.9 e33/23 ondisk, result = 0)", 2024-09-17T07:28:55.069 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:28:55.069 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.19407743899999999, 2024-09-17T07:28:55.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029728505999999998, 2024-09-17T07:28:55.069 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.069 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.070 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:28:55.070 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:28:55.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:28:55.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:28:55.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9400000000000004e-07 2024-09-17T07:28:55.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766243+0000", 2024-09-17T07:28:55.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9419999999999997e-06 2024-09-17T07:28:55.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794812+0000", 2024-09-17T07:28:55.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028568324999999999 2024-09-17T07:28:55.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794822+0000", 2024-09-17T07:28:55.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0786e-05 2024-09-17T07:28:55.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795847+0000", 2024-09-17T07:28:55.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010248919999999999 2024-09-17T07:28:55.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795854+0000", 2024-09-17T07:28:55.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4919999999999998e-06 2024-09-17T07:28:55.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795967+0000", 2024-09-17T07:28:55.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113275 2024-09-17T07:28:55.073 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.073 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.073 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.073 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:6685 3.14 3:2a16c27c:::rbd_object_map.1183cff49e3a.0000000000000007:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:28:55.073 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.770642+0000", 2024-09-17T07:28:55.073 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.189674538, 2024-09-17T07:28:55.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027608490999999999, 2024-09-17T07:28:55.073 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.074 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.074 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.074 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6685 2024-09-17T07:28:55.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.074 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770642+0000", 2024-09-17T07:28:55.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770642+0000", 2024-09-17T07:28:55.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770643+0000", 2024-09-17T07:28:55.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.824e-06 2024-09-17T07:28:55.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770651+0000", 2024-09-17T07:28:55.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9680000000000006e-06 2024-09-17T07:28:55.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794795+0000", 2024-09-17T07:28:55.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024143286 2024-09-17T07:28:55.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794811+0000", 2024-09-17T07:28:55.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6112000000000001e-05 2024-09-17T07:28:55.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794961+0000", 2024-09-17T07:28:55.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014987 2024-09-17T07:28:55.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795048+0000", 2024-09-17T07:28:55.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7740999999999999e-05 2024-09-17T07:28:55.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:28:55.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795317+0000", 2024-09-17T07:28:55.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026868500000000001 2024-09-17T07:28:55.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.797274+0000", 2024-09-17T07:28:55.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019571660000000002 2024-09-17T07:28:55.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.798166+0000", 2024-09-17T07:28:55.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00089202400000000005 2024-09-17T07:28:55.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.798212+0000", 2024-09-17T07:28:55.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.596e-05 2024-09-17T07:28:55.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.798250+0000", 2024-09-17T07:28:55.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7855000000000003e-05 2024-09-17T07:28:55.079 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.079 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.079 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.079 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:6688 3.14 3:2a16c27c:::rbd_object_map.1183cff49e3a.0000000000000007:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:28:55.079 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.795364+0000", 2024-09-17T07:28:55.079 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.16495178399999999, 2024-09-17T07:28:55.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057475419999999996, 2024-09-17T07:28:55.079 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.079 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.079 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.080 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.080 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.080 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6688 2024-09-17T07:28:55.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.080 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795364+0000", 2024-09-17T07:28:55.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795364+0000", 2024-09-17T07:28:55.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795365+0000", 2024-09-17T07:28:55.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.23e-07 2024-09-17T07:28:55.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795371+0000", 2024-09-17T07:28:55.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5260000000000001e-06 2024-09-17T07:28:55.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795382+0000", 2024-09-17T07:28:55.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1565e-05 2024-09-17T07:28:55.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795389+0000", 2024-09-17T07:28:55.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0360000000000001e-06 2024-09-17T07:28:55.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.797456+0000", 2024-09-17T07:28:55.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002066825 2024-09-17T07:28:55.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.797532+0000", 2024-09-17T07:28:55.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6417999999999999e-05 2024-09-17T07:28:55.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:28:55.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.797823+0000", 2024-09-17T07:28:55.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029098299999999998 2024-09-17T07:28:55.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.800617+0000", 2024-09-17T07:28:55.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0027933599999999999 2024-09-17T07:28:55.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.801044+0000", 2024-09-17T07:28:55.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00042737299999999999 2024-09-17T07:28:55.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.801075+0000", 2024-09-17T07:28:55.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1260999999999998e-05 2024-09-17T07:28:55.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.801112+0000", 2024-09-17T07:28:55.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6371999999999997e-05 2024-09-17T07:28:55.084 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.084 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.084 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.085 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.085 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.085 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:55.086 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:28:55.087 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:55.084+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-17T07:28:55.186 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:55.186+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:55.189 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:55.189 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:28:55.190 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:28:55.190 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:55.198 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:28:55.287 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:55.287+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:55.314 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:55.314 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:28:55.314 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:28:55.314 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:28:55.315 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:55.327 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:28:55.388 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:55.387+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:55.445 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-17T07:28:55.446 INFO:teuthology.orchestra.run.smithi045.stdout:514 write 0xaf82889 thru 0xaf92683 (0xfdfb bytes) 2024-09-17T07:28:55.452 INFO:teuthology.orchestra.run.smithi045.stdout:515 read 0x15113fe thru 0x15142a8 (0x2eab bytes) 2024-09-17T07:28:55.453 INFO:teuthology.orchestra.run.smithi045.stdout:518 punch from 0x14f6cf4 to 0x14f9e87, (0x3193 bytes) 2024-09-17T07:28:55.453 INFO:teuthology.orchestra.run.smithi045.stdout:519 read 0xda4d256 thru 0xda4e384 (0x112f bytes) 2024-09-17T07:28:55.453 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:55.458 INFO:teuthology.orchestra.run.smithi045.stdout:522 writesame 0xe741cb3 thru 0xe749b9a data_size 0x3e(0x7ee8 bytes) 2024-09-17T07:28:55.459 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:55.459 INFO:teuthology.orchestra.run.smithi045.stdout:524 read 0x7db7ad0 thru 0x7dba96a (0x2e9b bytes) 2024-09-17T07:28:55.459 INFO:teuthology.orchestra.run.smithi045.stdout:525 write 0x96fe1cb thru 0x97077fd (0x9633 bytes) 2024-09-17T07:28:55.464 INFO:teuthology.orchestra.run.smithi045.stdout:526 write 0xcd9c013 thru 0xcda2b44 (0x6b32 bytes) 2024-09-17T07:28:55.467 INFO:teuthology.orchestra.run.smithi045.stdout:527 read 0x40ea4fe thru 0x40f27cd (0x82d0 bytes) 2024-09-17T07:28:55.467 INFO:teuthology.orchestra.run.smithi045.stdout:528 punch from 0x34f2cbc to 0x34f87d6, (0x5b1a bytes) 2024-09-17T07:28:55.467 INFO:teuthology.orchestra.run.smithi045.stdout:529 read 0x467af6 thru 0x476f62 (0xf46d bytes) 2024-09-17T07:28:55.467 INFO:teuthology.orchestra.run.smithi045.stdout:531 write 0x96f0d76 thru 0x96fce88 (0xc113 bytes) 2024-09-17T07:28:55.469 INFO:teuthology.orchestra.run.smithi045.stdout:533 write 0x11a13ac thru 0x11a7a77 (0x66cc bytes) 2024-09-17T07:28:55.475 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:55.475 INFO:teuthology.orchestra.run.smithi045.stdout:truncating to largest ever: 0xee5aa98 2024-09-17T07:28:55.476 INFO:teuthology.orchestra.run.smithi045.stdout:535 trunc from 0xe749b9b to 0xee5aa98 2024-09-17T07:28:55.480 INFO:teuthology.orchestra.run.smithi045.stdout:536 writesame 0xd147685 thru 0xd1546ec data_size 0xc(0xd068 bytes) 2024-09-17T07:28:55.482 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:55.482 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:28:55.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:28:55.482 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:28:55.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.482 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4226.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-17T07:28:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:31.356276+0000", 2024-09-17T07:28:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.124272034000001, 2024-09-17T07:28:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053924159999999997, 2024-09-17T07:28:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4226", 2024-09-17T07:28:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/1373512931", 2024-09-17T07:28:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:28:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356276+0000", 2024-09-17T07:28:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356276+0000", 2024-09-17T07:28:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356280+0000", 2024-09-17T07:28:55.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.019e-06 2024-09-17T07:28:55.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356287+0000", 2024-09-17T07:28:55.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.267e-06 2024-09-17T07:28:55.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356297+0000", 2024-09-17T07:28:55.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5699999999999999e-06 2024-09-17T07:28:55.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356305+0000", 2024-09-17T07:28:55.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5040000000000002e-06 2024-09-17T07:28:55.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356392+0000", 2024-09-17T07:28:55.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7012e-05 2024-09-17T07:28:55.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-17T07:28:55.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.356402+0000", 2024-09-17T07:28:55.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9650000000000003e-06 2024-09-17T07:28:55.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.358226+0000", 2024-09-17T07:28:55.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018234620000000001 2024-09-17T07:28:55.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:28:55.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.358232+0000", 2024-09-17T07:28:55.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2820000000000002e-06 2024-09-17T07:28:55.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.360426+0000", 2024-09-17T07:28:55.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021943890000000001 2024-09-17T07:28:55.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.360556+0000", 2024-09-17T07:28:55.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012972 2024-09-17T07:28:55.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-17T07:28:55.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.360649+0000", 2024-09-17T07:28:55.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2942000000000006e-05 2024-09-17T07:28:55.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361494+0000", 2024-09-17T07:28:55.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00084479200000000003 2024-09-17T07:28:55.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361559+0000", 2024-09-17T07:28:55.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5347999999999998e-05 2024-09-17T07:28:55.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361616+0000", 2024-09-17T07:28:55.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6836000000000003e-05 2024-09-17T07:28:55.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:31.361668+0000", 2024-09-17T07:28:55.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2308000000000001e-05 2024-09-17T07:28:55.491 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.491 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.491 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.491 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:958 3.1c 3:3c42a6c1:::rbd_data.1183350c2d82.000000000000000f:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:55.491 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:49.182586+0000", 2024-09-17T07:28:55.491 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.2979618899999998, 2024-09-17T07:28:55.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.013048837000000001, 2024-09-17T07:28:55.491 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.491 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.492 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.492 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.492 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.492 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.492 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 958 2024-09-17T07:28:55.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.492 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.182586+0000", 2024-09-17T07:28:55.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.182586+0000", 2024-09-17T07:28:55.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.182587+0000", 2024-09-17T07:28:55.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7899999999999997e-07 2024-09-17T07:28:55.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.182591+0000", 2024-09-17T07:28:55.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5260000000000004e-06 2024-09-17T07:28:55.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.182601+0000", 2024-09-17T07:28:55.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0129000000000001e-05 2024-09-17T07:28:55.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.182607+0000", 2024-09-17T07:28:55.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:28:55.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.182660+0000", 2024-09-17T07:28:55.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2945999999999998e-05 2024-09-17T07:28:55.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.182782+0000", 2024-09-17T07:28:55.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012294199999999999 2024-09-17T07:28:55.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:55.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.184809+0000", 2024-09-17T07:28:55.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002026581 2024-09-17T07:28:55.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.195535+0000", 2024-09-17T07:28:55.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010725816000000001 2024-09-17T07:28:55.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.195566+0000", 2024-09-17T07:28:55.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0942000000000003e-05 2024-09-17T07:28:55.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.195604+0000", 2024-09-17T07:28:55.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8581999999999998e-05 2024-09-17T07:28:55.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:49.195635+0000", 2024-09-17T07:28:55.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0357000000000001e-05 2024-09-17T07:28:55.498 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.498 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.498 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.498 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:1503 3.1d 3:bb76ae33:::rbd_data.11832b928878.000000000000023a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:55.498 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:50.580741+0000", 2024-09-17T07:28:55.498 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.8998069360000001, 2024-09-17T07:28:55.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052196079999999997, 2024-09-17T07:28:55.499 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.499 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.499 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.499 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.499 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.499 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.499 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1503 2024-09-17T07:28:55.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.499 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580741+0000", 2024-09-17T07:28:55.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580741+0000", 2024-09-17T07:28:55.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580742+0000", 2024-09-17T07:28:55.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9299999999999996e-07 2024-09-17T07:28:55.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580747+0000", 2024-09-17T07:28:55.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8520000000000003e-06 2024-09-17T07:28:55.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580760+0000", 2024-09-17T07:28:55.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3577000000000001e-05 2024-09-17T07:28:55.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580768+0000", 2024-09-17T07:28:55.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8089999999999993e-06 2024-09-17T07:28:55.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580812+0000", 2024-09-17T07:28:55.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3529999999999998e-05 2024-09-17T07:28:55.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.580883+0000", 2024-09-17T07:28:55.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1582999999999995e-05 2024-09-17T07:28:55.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:55.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.581000+0000", 2024-09-17T07:28:55.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117222 2024-09-17T07:28:55.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.582972+0000", 2024-09-17T07:28:55.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001972017 2024-09-17T07:28:55.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.585890+0000", 2024-09-17T07:28:55.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0029174230000000002 2024-09-17T07:28:55.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.585927+0000", 2024-09-17T07:28:55.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7660999999999997e-05 2024-09-17T07:28:55.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.585960+0000", 2024-09-17T07:28:55.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3040999999999998e-05 2024-09-17T07:28:55.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.506 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:1577 3.5 3:a318d77c:::rbd_data.11832b928878.00000000000001a2:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:55.506 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:50.639384+0000", 2024-09-17T07:28:55.506 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.841163313, 2024-09-17T07:28:55.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012105382, 2024-09-17T07:28:55.506 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.506 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.506 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.506 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1577 2024-09-17T07:28:55.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.507 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639384+0000", 2024-09-17T07:28:55.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639384+0000", 2024-09-17T07:28:55.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639386+0000", 2024-09-17T07:28:55.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.207e-06 2024-09-17T07:28:55.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639392+0000", 2024-09-17T07:28:55.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8710000000000002e-06 2024-09-17T07:28:55.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639404+0000", 2024-09-17T07:28:55.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2687999999999999e-05 2024-09-17T07:28:55.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639417+0000", 2024-09-17T07:28:55.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2264e-05 2024-09-17T07:28:55.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639483+0000", 2024-09-17T07:28:55.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6656999999999996e-05 2024-09-17T07:28:55.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.639615+0000", 2024-09-17T07:28:55.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131781 2024-09-17T07:28:55.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-09-17T07:28:55.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.641755+0000", 2024-09-17T07:28:55.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021397830000000001 2024-09-17T07:28:55.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.651413+0000", 2024-09-17T07:28:55.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0096580290000000003 2024-09-17T07:28:55.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.651423+0000", 2024-09-17T07:28:55.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8619999999999998e-06 2024-09-17T07:28:55.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.651459+0000", 2024-09-17T07:28:55.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6389e-05 2024-09-17T07:28:55.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.651490+0000", 2024-09-17T07:28:55.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0851e-05 2024-09-17T07:28:55.511 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.511 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.511 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.512 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:1648 3.1c 3:3fc76693:::rbd_data.11832b928878.00000000000002ed:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:55.512 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:50.712400+0000", 2024-09-17T07:28:55.512 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.7681473949999997, 2024-09-17T07:28:55.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052211810000000001, 2024-09-17T07:28:55.512 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.512 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.512 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.512 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1648 2024-09-17T07:28:55.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.513 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712400+0000", 2024-09-17T07:28:55.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712400+0000", 2024-09-17T07:28:55.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712402+0000", 2024-09-17T07:28:55.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2419999999999999e-06 2024-09-17T07:28:55.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712406+0000", 2024-09-17T07:28:55.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8609999999999997e-06 2024-09-17T07:28:55.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712418+0000", 2024-09-17T07:28:55.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:28:55.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712424+0000", 2024-09-17T07:28:55.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.553e-06 2024-09-17T07:28:55.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712502+0000", 2024-09-17T07:28:55.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8201999999999994e-05 2024-09-17T07:28:55.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712589+0000", 2024-09-17T07:28:55.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7379999999999993e-05 2024-09-17T07:28:55.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:55.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.712696+0000", 2024-09-17T07:28:55.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000106829 2024-09-17T07:28:55.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.713217+0000", 2024-09-17T07:28:55.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00052112100000000004 2024-09-17T07:28:55.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.717566+0000", 2024-09-17T07:28:55.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0043485290000000003 2024-09-17T07:28:55.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.717596+0000", 2024-09-17T07:28:55.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0728000000000001e-05 2024-09-17T07:28:55.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.717622+0000", 2024-09-17T07:28:55.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5162999999999999e-05 2024-09-17T07:28:55.518 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.518 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.518 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.518 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:1903 3.1c 3:3f254970:::rbd_data.11832b928878.00000000000000e6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:55.518 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:50.842809+0000", 2024-09-17T07:28:55.518 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.637738562, 2024-09-17T07:28:55.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051225840000000003, 2024-09-17T07:28:55.518 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:28:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1903 2024-09-17T07:28:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.842809+0000", 2024-09-17T07:28:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.842809+0000", 2024-09-17T07:28:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.842810+0000", 2024-09-17T07:28:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7599999999999998e-07 2024-09-17T07:28:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.842811+0000", 2024-09-17T07:28:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.674e-06 2024-09-17T07:28:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.842816+0000", 2024-09-17T07:28:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9590000000000003e-06 2024-09-17T07:28:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.842821+0000", 2024-09-17T07:28:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8250000000000004e-06 2024-09-17T07:28:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.842889+0000", 2024-09-17T07:28:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7829000000000004e-05 2024-09-17T07:28:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.843007+0000", 2024-09-17T07:28:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011784499999999999 2024-09-17T07:28:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.847932+0000", 2024-09-17T07:28:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049250759999999996 2024-09-17T07:28:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:1953 3.1d 3:bdd9a6f7:::rbd_data.11832b928878.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 65536~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:50.859319+0000", 2024-09-17T07:28:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.6212292560000003, 2024-09-17T07:28:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050716880000000004, 2024-09-17T07:28:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1953 2024-09-17T07:28:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.859319+0000", 2024-09-17T07:28:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.859319+0000", 2024-09-17T07:28:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.859320+0000", 2024-09-17T07:28:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.575e-06 2024-09-17T07:28:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.859342+0000", 2024-09-17T07:28:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:28:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.859356+0000", 2024-09-17T07:28:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3563e-05 2024-09-17T07:28:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.859363+0000", 2024-09-17T07:28:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4209999999999998e-06 2024-09-17T07:28:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.859408+0000", 2024-09-17T07:28:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4483000000000001e-05 2024-09-17T07:28:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.859539+0000", 2024-09-17T07:28:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000131092 2024-09-17T07:28:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.859753+0000", 2024-09-17T07:28:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021433199999999999 2024-09-17T07:28:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.864067+0000", 2024-09-17T07:28:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0043137360000000003 2024-09-17T07:28:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.864316+0000", 2024-09-17T07:28:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024939100000000001 2024-09-17T07:28:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.864349+0000", 2024-09-17T07:28:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2552000000000003e-05 2024-09-17T07:28:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:50.864390+0000", 2024-09-17T07:28:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1381000000000002e-05 2024-09-17T07:28:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:2155 3.5 3:a751bffa:::rbd_data.11832b928878.00000000000002b4:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:51.074808+0000", 2024-09-17T07:28:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4057398699999997, 2024-09-17T07:28:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.014018765000000001, 2024-09-17T07:28:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2155 2024-09-17T07:28:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.074808+0000", 2024-09-17T07:28:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.074808+0000", 2024-09-17T07:28:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.074810+0000", 2024-09-17T07:28:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7850000000000001e-06 2024-09-17T07:28:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.074817+0000", 2024-09-17T07:28:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7060000000000004e-06 2024-09-17T07:28:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.074834+0000", 2024-09-17T07:28:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6294999999999999e-05 2024-09-17T07:28:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.074843+0000", 2024-09-17T07:28:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8559999999999996e-06 2024-09-17T07:28:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.074899+0000", 2024-09-17T07:28:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6789000000000001e-05 2024-09-17T07:28:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.074992+0000", 2024-09-17T07:28:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2417999999999995e-05 2024-09-17T07:28:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-09-17T07:28:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.077091+0000", 2024-09-17T07:28:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020991909999999998 2024-09-17T07:28:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.088748+0000", 2024-09-17T07:28:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011656794999999999 2024-09-17T07:28:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.088755+0000", 2024-09-17T07:28:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7340000000000008e-06 2024-09-17T07:28:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.088786+0000", 2024-09-17T07:28:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0885e-05 2024-09-17T07:28:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.088827+0000", 2024-09-17T07:28:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0311000000000001e-05 2024-09-17T07:28:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:2156 3.c 3:31bc5ae3:::rbd_data.11832b928878.00000000000002b5:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:28:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:51.077345+0000", 2024-09-17T07:28:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.4032024500000002, 2024-09-17T07:28:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075015309999999997, 2024-09-17T07:28:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2156 2024-09-17T07:28:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.077345+0000", 2024-09-17T07:28:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.077345+0000", 2024-09-17T07:28:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.077347+0000", 2024-09-17T07:28:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.68e-06 2024-09-17T07:28:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.077354+0000", 2024-09-17T07:28:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6189999999999996e-06 2024-09-17T07:28:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.077369+0000", 2024-09-17T07:28:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5509999999999999e-05 2024-09-17T07:28:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.077380+0000", 2024-09-17T07:28:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.101e-05 2024-09-17T07:28:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.077450+0000", 2024-09-17T07:28:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9424000000000005e-05 2024-09-17T07:28:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.077590+0000", 2024-09-17T07:28:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:28:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-09-17T07:28:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.078513+0000", 2024-09-17T07:28:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00092306899999999999 2024-09-17T07:28:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.082568+0000", 2024-09-17T07:28:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040545099999999999 2024-09-17T07:28:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.084720+0000", 2024-09-17T07:28:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0021525200000000002 2024-09-17T07:28:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.084792+0000", 2024-09-17T07:28:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1358000000000004e-05 2024-09-17T07:28:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:51.084847+0000", 2024-09-17T07:28:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5092e-05 2024-09-17T07:28:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:5325 3.1d 3:bb8dc19a:::rbd_data.1183cff49e3a.0000000000000009:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:28:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.508850+0000", 2024-09-17T07:28:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.97169811500000003, 2024-09-17T07:28:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049986500000000003, 2024-09-17T07:28:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5325 2024-09-17T07:28:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.508850+0000", 2024-09-17T07:28:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.508850+0000", 2024-09-17T07:28:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.508850+0000", 2024-09-17T07:28:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6499999999999999e-07 2024-09-17T07:28:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.508853+0000", 2024-09-17T07:28:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1130000000000001e-06 2024-09-17T07:28:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.508862+0000", 2024-09-17T07:28:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3110000000000006e-06 2024-09-17T07:28:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.508868+0000", 2024-09-17T07:28:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1999999999999999e-06 2024-09-17T07:28:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.509979+0000", 2024-09-17T07:28:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011109099999999999 2024-09-17T07:28:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.510056+0000", 2024-09-17T07:28:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6847999999999993e-05 2024-09-17T07:28:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.510220+0000", 2024-09-17T07:28:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016443799999999999 2024-09-17T07:28:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.513601+0000", 2024-09-17T07:28:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033805179999999999 2024-09-17T07:28:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.513751+0000", 2024-09-17T07:28:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000150097 2024-09-17T07:28:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.513805+0000", 2024-09-17T07:28:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4348999999999998e-05 2024-09-17T07:28:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.513848+0000", 2024-09-17T07:28:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3201000000000002e-05 2024-09-17T07:28:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7918 3.16 3:6f261c84:::rbd_data.1183cff49e3a.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:28:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.994807+0000", 2024-09-17T07:28:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.48574055999999999, 2024-09-17T07:28:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057723469999999997, 2024-09-17T07:28:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7918 2024-09-17T07:28:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.994807+0000", 2024-09-17T07:28:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.994807+0000", 2024-09-17T07:28:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.994816+0000", 2024-09-17T07:28:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1030000000000008e-06 2024-09-17T07:28:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.995348+0000", 2024-09-17T07:28:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00053160100000000004 2024-09-17T07:28:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.995360+0000", 2024-09-17T07:28:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.242e-05 2024-09-17T07:28:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.995366+0000", 2024-09-17T07:28:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3669999999999996e-06 2024-09-17T07:28:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.995409+0000", 2024-09-17T07:28:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3056000000000002e-05 2024-09-17T07:28:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.995471+0000", 2024-09-17T07:28:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2069000000000005e-05 2024-09-17T07:28:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:28:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.995612+0000", 2024-09-17T07:28:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014140999999999999 2024-09-17T07:28:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997179+0000", 2024-09-17T07:28:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015671140000000001 2024-09-17T07:28:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.000521+0000", 2024-09-17T07:28:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033420059999999998 2024-09-17T07:28:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.000554+0000", 2024-09-17T07:28:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2187999999999997e-05 2024-09-17T07:28:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.000580+0000", 2024-09-17T07:28:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6013e-05 2024-09-17T07:28:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7919 3.16 3:6f261c84:::rbd_data.1183cff49e3a.000000000000002f:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:28:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.995380+0000", 2024-09-17T07:28:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.48516743899999998, 2024-09-17T07:28:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052723170000000003, 2024-09-17T07:28:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7919 2024-09-17T07:28:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.995380+0000", 2024-09-17T07:28:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.995380+0000", 2024-09-17T07:28:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.995381+0000", 2024-09-17T07:28:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:28:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.995384+0000", 2024-09-17T07:28:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8710000000000001e-06 2024-09-17T07:28:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.995389+0000", 2024-09-17T07:28:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4280000000000004e-06 2024-09-17T07:28:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.995399+0000", 2024-09-17T07:28:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0210000000000001e-05 2024-09-17T07:28:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996252+0000", 2024-09-17T07:28:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00085292199999999999 2024-09-17T07:28:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996351+0000", 2024-09-17T07:28:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8592999999999996e-05 2024-09-17T07:28:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:28:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996482+0000", 2024-09-17T07:28:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013074500000000001 2024-09-17T07:28:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997544+0000", 2024-09-17T07:28:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001061914 2024-09-17T07:28:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.000607+0000", 2024-09-17T07:28:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030635240000000002 2024-09-17T07:28:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.000632+0000", 2024-09-17T07:28:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5196999999999999e-05 2024-09-17T07:28:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.000653+0000", 2024-09-17T07:28:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0333e-05 2024-09-17T07:28:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4483.0:8901 3.1f e34/23 3:ffa28dc9:::rbd_data.1183cff49e3a.000000000000005c:head v 34'40, mlcod=33'36)", 2024-09-17T07:28:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:55.342769+0000", 2024-09-17T07:28:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.137779076, 2024-09-17T07:28:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063251840000000002, 2024-09-17T07:28:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.342769+0000", 2024-09-17T07:28:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.342769+0000", 2024-09-17T07:28:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.342769+0000", 2024-09-17T07:28:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5899999999999996e-07 2024-09-17T07:28:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.342774+0000", 2024-09-17T07:28:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0509999999999996e-06 2024-09-17T07:28:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.342783+0000", 2024-09-17T07:28:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0289999999999994e-06 2024-09-17T07:28:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.342789+0000", 2024-09-17T07:28:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9279999999999998e-06 2024-09-17T07:28:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.342908+0000", 2024-09-17T07:28:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011862 2024-09-17T07:28:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.342927+0000", 2024-09-17T07:28:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8810000000000001e-05 2024-09-17T07:28:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.349053+0000", 2024-09-17T07:28:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061259670000000004 2024-09-17T07:28:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.349094+0000", 2024-09-17T07:28:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1119999999999999e-05 2024-09-17T07:28:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:8905 3.c 3:37340ed2:::rbd_data.1183cff49e3a.000000000000006e:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T07:28:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:55.344006+0000", 2024-09-17T07:28:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.136541471, 2024-09-17T07:28:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051462920000000002, 2024-09-17T07:28:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:28:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 8905 2024-09-17T07:28:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.344006+0000", 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.344006+0000", 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.344007+0000", 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.13e-06 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.344015+0000", 2024-09-17T07:28:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:28:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.344028+0000", 2024-09-17T07:28:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622e-05 2024-09-17T07:28:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.344036+0000", 2024-09-17T07:28:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3289999999999994e-06 2024-09-17T07:28:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.344087+0000", 2024-09-17T07:28:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1048999999999999e-05 2024-09-17T07:28:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.344212+0000", 2024-09-17T07:28:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000124864 2024-09-17T07:28:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.349153+0000", 2024-09-17T07:28:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049407160000000004 2024-09-17T07:28:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:8917 3.c 3:37340ed2:::rbd_data.1183cff49e3a.000000000000006e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T07:28:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:55.350091+0000", 2024-09-17T07:28:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.13045707500000001, 2024-09-17T07:28:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0068124020000000004, 2024-09-17T07:28:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 8917 2024-09-17T07:28:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.350091+0000", 2024-09-17T07:28:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.350091+0000", 2024-09-17T07:28:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.350092+0000", 2024-09-17T07:28:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0499999999999999e-06 2024-09-17T07:28:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.350099+0000", 2024-09-17T07:28:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.447e-06 2024-09-17T07:28:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.350111+0000", 2024-09-17T07:28:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1469e-05 2024-09-17T07:28:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.350118+0000", 2024-09-17T07:28:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9330000000000004e-06 2024-09-17T07:28:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.350770+0000", 2024-09-17T07:28:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00065238399999999999 2024-09-17T07:28:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.350841+0000", 2024-09-17T07:28:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1372999999999996e-05 2024-09-17T07:28:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-09-17T07:28:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.351009+0000", 2024-09-17T07:28:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016802000000000001 2024-09-17T07:28:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.351807+0000", 2024-09-17T07:28:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00079757000000000005 2024-09-17T07:28:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.356818+0000", 2024-09-17T07:28:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050106450000000002 2024-09-17T07:28:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.356871+0000", 2024-09-17T07:28:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3415000000000003e-05 2024-09-17T07:28:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.356903+0000", 2024-09-17T07:28:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2095999999999999e-05 2024-09-17T07:28:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4483.0:8922 3.1f e34/23 3:fbdaace4:::rbd_data.1183cff49e3a.000000000000005e:head v 34'43, mlcod=34'38)", 2024-09-17T07:28:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:55.356214+0000", 2024-09-17T07:28:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.124333872, 2024-09-17T07:28:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063063679999999997, 2024-09-17T07:28:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.356214+0000", 2024-09-17T07:28:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.356214+0000", 2024-09-17T07:28:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.356215+0000", 2024-09-17T07:28:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3499999999999995e-07 2024-09-17T07:28:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.356221+0000", 2024-09-17T07:28:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:28:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.356233+0000", 2024-09-17T07:28:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1595e-05 2024-09-17T07:28:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.356239+0000", 2024-09-17T07:28:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:28:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.356830+0000", 2024-09-17T07:28:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00059087899999999999 2024-09-17T07:28:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.356852+0000", 2024-09-17T07:28:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1503999999999999e-05 2024-09-17T07:28:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.362484+0000", 2024-09-17T07:28:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0056322949999999998 2024-09-17T07:28:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.362520+0000", 2024-09-17T07:28:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6171000000000003e-05 2024-09-17T07:28:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:8941 3.1d 3:bf8c80f4:::rbd_data.1183cff49e3a.00000000000000bc:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T07:28:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:55.368483+0000", 2024-09-17T07:28:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.112064988, 2024-09-17T07:28:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0052806700000000003, 2024-09-17T07:28:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 8941 2024-09-17T07:28:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.368483+0000", 2024-09-17T07:28:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.368483+0000", 2024-09-17T07:28:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.368484+0000", 2024-09-17T07:28:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6500000000000008e-07 2024-09-17T07:28:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.368488+0000", 2024-09-17T07:28:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4939999999999997e-06 2024-09-17T07:28:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.368499+0000", 2024-09-17T07:28:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0288e-05 2024-09-17T07:28:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.368503+0000", 2024-09-17T07:28:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.938e-06 2024-09-17T07:28:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.368553+0000", 2024-09-17T07:28:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9663000000000002e-05 2024-09-17T07:28:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.368632+0000", 2024-09-17T07:28:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9297999999999999e-05 2024-09-17T07:28:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.368783+0000", 2024-09-17T07:28:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000150118 2024-09-17T07:28:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.370816+0000", 2024-09-17T07:28:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020337100000000002 2024-09-17T07:28:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.373667+0000", 2024-09-17T07:28:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028511299999999999 2024-09-17T07:28:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.373716+0000", 2024-09-17T07:28:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9011000000000002e-05 2024-09-17T07:28:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.373763+0000", 2024-09-17T07:28:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7055000000000003e-05 2024-09-17T07:28:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:8947 3.1c 3:3c222b09:::rbd_data.1183cff49e3a.0000000000000122:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T07:28:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:55.369497+0000", 2024-09-17T07:28:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.11105073, 2024-09-17T07:28:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070282540000000003, 2024-09-17T07:28:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 8947 2024-09-17T07:28:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.369497+0000", 2024-09-17T07:28:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.369497+0000", 2024-09-17T07:28:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.369498+0000", 2024-09-17T07:28:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0219999999999999e-06 2024-09-17T07:28:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.369503+0000", 2024-09-17T07:28:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8420000000000004e-06 2024-09-17T07:28:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.369515+0000", 2024-09-17T07:28:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2334000000000001e-05 2024-09-17T07:28:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.369523+0000", 2024-09-17T07:28:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.379e-06 2024-09-17T07:28:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.369583+0000", 2024-09-17T07:28:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0102999999999997e-05 2024-09-17T07:28:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.369633+0000", 2024-09-17T07:28:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0340000000000003e-05 2024-09-17T07:28:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.369765+0000", 2024-09-17T07:28:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000132146 2024-09-17T07:28:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.373632+0000", 2024-09-17T07:28:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0038671090000000001 2024-09-17T07:28:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.376432+0000", 2024-09-17T07:28:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002799817 2024-09-17T07:28:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.376485+0000", 2024-09-17T07:28:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2970999999999998e-05 2024-09-17T07:28:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.376525+0000", 2024-09-17T07:28:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0191000000000002e-05 2024-09-17T07:28:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop(client.4483.0:8950 3.1f e34/23 3:ff4b6120:::rbd_data.1183cff49e3a.0000000000000125:head v 34'47, mlcod=34'43)", 2024-09-17T07:28:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:55.371286+0000", 2024-09-17T07:28:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.109262102, 2024-09-17T07:28:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007400907, 2024-09-17T07:28:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.371286+0000", 2024-09-17T07:28:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.371286+0000", 2024-09-17T07:28:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.371287+0000", 2024-09-17T07:28:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.269e-06 2024-09-17T07:28:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.371293+0000", 2024-09-17T07:28:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3579999999999999e-06 2024-09-17T07:28:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.371307+0000", 2024-09-17T07:28:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3621999999999999e-05 2024-09-17T07:28:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.371315+0000", 2024-09-17T07:28:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7800000000000001e-06 2024-09-17T07:28:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.371362+0000", 2024-09-17T07:28:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6885999999999998e-05 2024-09-17T07:28:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.371376+0000", 2024-09-17T07:28:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4096e-05 2024-09-17T07:28:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.378644+0000", 2024-09-17T07:28:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007268783 2024-09-17T07:28:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.378687+0000", 2024-09-17T07:28:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2113000000000002e-05 2024-09-17T07:28:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:8954 3.1d 3:b9776cb6:::rbd_data.1183cff49e3a.0000000000000131:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T07:28:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:55.373956+0000", 2024-09-17T07:28:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.106591963, 2024-09-17T07:28:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0075146939999999997, 2024-09-17T07:28:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:28:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:28:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:28:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:28:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 8954 2024-09-17T07:28:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:28:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:28:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.373956+0000", 2024-09-17T07:28:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:28:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.373956+0000", 2024-09-17T07:28:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:28:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:28:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.373957+0000", 2024-09-17T07:28:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9999999999999996e-07 2024-09-17T07:28:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:28:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.373961+0000", 2024-09-17T07:28:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0539999999999996e-06 2024-09-17T07:28:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:28:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.373970+0000", 2024-09-17T07:28:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5470000000000002e-06 2024-09-17T07:28:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:28:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.373975+0000", 2024-09-17T07:28:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:28:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:28:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.374010+0000", 2024-09-17T07:28:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5317000000000001e-05 2024-09-17T07:28:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:28:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.374087+0000", 2024-09-17T07:28:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6939000000000003e-05 2024-09-17T07:28:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.374264+0000", 2024-09-17T07:28:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017696500000000001 2024-09-17T07:28:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.378885+0000", 2024-09-17T07:28:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046201660000000002 2024-09-17T07:28:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:28:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.381374+0000", 2024-09-17T07:28:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0024891259999999999 2024-09-17T07:28:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:28:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.381431+0000", 2024-09-17T07:28:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7484000000000001e-05 2024-09-17T07:28:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:28:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:28:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:28:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.381471+0000", 2024-09-17T07:28:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9393e-05 2024-09-17T07:28:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:28:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:28:55.603 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:55.604 INFO:teuthology.orchestra.run.smithi045.stdout:537 writesame 0x3f244b9 thru 0x3f24c0f data_size 0x1(0x757 bytes) 2024-09-17T07:28:55.604 INFO:teuthology.orchestra.run.smithi045.stdout:538 write 0x98f5362 thru 0x98f80fc (0x2d9b bytes) 2024-09-17T07:28:55.605 INFO:teuthology.orchestra.run.smithi045.stdout:540 trunc from 0xee5aa98 to 0x951d913 2024-09-17T07:28:55.605 INFO:teuthology.orchestra.run.smithi045.stdout:541 clone 6 order 24 su 1048576 sc 4 2024-09-17T07:28:55.608 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:55.603+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:55.609 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:28:55.693 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:28:55.705 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:55.704+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:55.712 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:55.712 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:28:55.712 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:28:55.713 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:55.721 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:28:55.806 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:55.806+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:55.818 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:28:55.818 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:28:55.835 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:28:55.836 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:28:55.852 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:28:55.852 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:28:55.864 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:28:55.864 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:28:55.865 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:28:55.865 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:28:55.865 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:28:55.868 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:28:55.868 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:28:55.873 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:28:55.885 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:28:55.885 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:28:55.903 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:28:55.903 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:28:55.913 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:55.913+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:55.920 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:28:55.920 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:28:55.937 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:28:55.937 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:28:56.008 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:56.006+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:56.016 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:56.016 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:56.025 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:28:56.109 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:56.108+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:56.130 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:56.130 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:56.141 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:28:56.210 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:56.208+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:56.213 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone5 from 0x951d913 (overlap 0x951d913) to 0x0 2024-09-17T07:28:56.312 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:56.311+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:56.340 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:56.340 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:56.353 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:28:56.413 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:56.411+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:56.507 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-17T07:28:56.508 INFO:teuthology.orchestra.run.smithi045.stdout:544 read 0x7836330 thru 0x783b70b (0x53dc bytes) 2024-09-17T07:28:56.508 INFO:teuthology.orchestra.run.smithi045.stdout:546 writesame 0x204e8ad thru 0x2057386 data_size 0x7(0x8ada bytes) 2024-09-17T07:28:56.514 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:56.513+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:56.515 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:56.516 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:56.528 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:28:56.573 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:56.574 INFO:teuthology.orchestra.run.smithi045.stdout:549 punch from 0x2b07737 to 0x2b0cd6d, (0x5636 bytes) 2024-09-17T07:28:56.574 INFO:teuthology.orchestra.run.smithi045.stdout:552 punch from 0x8d29384 to 0x8d2d037, (0x3cb3 bytes) 2024-09-17T07:28:56.574 INFO:teuthology.orchestra.run.smithi045.stdout:554 read 0x5216c2f thru 0x521b36a (0x473c bytes) 2024-09-17T07:28:56.574 INFO:teuthology.orchestra.run.smithi045.stdout:555 punch from 0x50ff42f to 0x5107bfe, (0x87cf bytes) 2024-09-17T07:28:56.574 INFO:teuthology.orchestra.run.smithi045.stdout:556 read 0x839e91c thru 0x83a926e (0xa953 bytes) 2024-09-17T07:28:56.574 INFO:teuthology.orchestra.run.smithi045.stdout:560 punch from 0x3e4ed9c to 0x3e5188b, (0x2aef bytes) 2024-09-17T07:28:56.585 INFO:teuthology.orchestra.run.smithi045.stdout:562 write 0xec3179c thru 0xec3e8b3 (0xd118 bytes) 2024-09-17T07:28:56.588 INFO:teuthology.orchestra.run.smithi045.stdout:563 punch from 0x7f62f60 to 0x7f68f4e, (0x5fee bytes) 2024-09-17T07:28:56.588 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:56.588 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:56.588 INFO:teuthology.orchestra.run.smithi045.stdout:567 writesame 0xa935d2d thru 0xa94570b data_size 0x1(0xf9df bytes) 2024-09-17T07:28:56.615 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:56.613+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:56.637 INFO:teuthology.orchestra.run.smithi045.stdout:568 trunc from 0xec3e8b4 to 0x6a23c47 2024-09-17T07:28:56.643 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:56.643 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:56.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:28:56.688 INFO:teuthology.orchestra.run.smithi045.stdout:571 write 0x53e6577 thru 0x53f22f1 (0xbd7b bytes) 2024-09-17T07:28:56.690 INFO:teuthology.orchestra.run.smithi045.stdout:572 read 0x18e5c29 thru 0x18f429c (0xe674 bytes) 2024-09-17T07:28:56.690 INFO:teuthology.orchestra.run.smithi045.stdout:573 read 0x13129ca thru 0x131b362 (0x8999 bytes) 2024-09-17T07:28:56.691 INFO:teuthology.orchestra.run.smithi045.stdout:575 write 0x2b56d32 thru 0x2b618eb (0xabba bytes) 2024-09-17T07:28:56.715 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:56.714+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:56.764 INFO:teuthology.orchestra.run.smithi045.stdout:576 trunc from 0x6a23c47 to 0xe0b3371 2024-09-17T07:28:56.776 INFO:teuthology.orchestra.run.smithi045.stdout:578 write 0xeaa1132 thru 0xeaa16dd (0x5ac bytes) 2024-09-17T07:28:56.779 INFO:teuthology.orchestra.run.smithi045.stdout:579 read 0xcd90b9d thru 0xcd9f0a6 (0xe50a bytes) 2024-09-17T07:28:56.779 INFO:teuthology.orchestra.run.smithi045.stdout:581 write 0xc2c333a thru 0xc2cd5af (0xa276 bytes) 2024-09-17T07:28:56.780 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:56.780 INFO:teuthology.orchestra.run.smithi045.stdout:584 read 0xca7156f thru 0xca72fc8 (0x1a5a bytes) 2024-09-17T07:28:56.794 INFO:teuthology.orchestra.run.smithi045.stdout:585 trunc from 0xeaa16de to 0x25c58f0 2024-09-17T07:28:56.811 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:56.811 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:56.816 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:56.816+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:56.827 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:28:56.872 INFO:teuthology.orchestra.run.smithi045.stdout:586 trunc from 0x25c58f0 to 0x430c24a 2024-09-17T07:28:56.890 INFO:teuthology.orchestra.run.smithi045.stdout:587 write 0xc0dba74 thru 0xc0dd4b9 (0x1a46 bytes) 2024-09-17T07:28:56.893 INFO:teuthology.orchestra.run.smithi045.stdout:588 writesame 0x214c89a thru 0x2153d35 data_size 0x22(0x749c bytes) 2024-09-17T07:28:56.895 INFO:teuthology.orchestra.run.smithi045.stdout:589 punch from 0x7bb25c6 to 0x7bb8450, (0x5e8a bytes) 2024-09-17T07:28:56.895 INFO:teuthology.orchestra.run.smithi045.stdout:591 read 0x162d805 thru 0x1635b78 (0x8374 bytes) 2024-09-17T07:28:56.896 INFO:teuthology.orchestra.run.smithi045.stdout:592 write 0x5dde401 thru 0x5de6009 (0x7c09 bytes) 2024-09-17T07:28:56.899 INFO:teuthology.orchestra.run.smithi045.stdout:593 write 0x6ffffc8 thru 0x700a78e (0xa7c7 bytes) 2024-09-17T07:28:56.902 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:56.910 INFO:teuthology.orchestra.run.smithi045.stdout:595 write 0xe46531a thru 0xe467a2e (0x2715 bytes) 2024-09-17T07:28:56.912 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:56.912 INFO:teuthology.orchestra.run.smithi045.stdout:598 read 0x7bfc8d1 thru 0x7c0b2ad (0xe9dd bytes) 2024-09-17T07:28:56.913 INFO:teuthology.orchestra.run.smithi045.stdout:599 clone 7 order 23 su 262144 sc 3 2024-09-17T07:28:56.917 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:56.917+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:56.982 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:56.982 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:56.990 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:28:57.019 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:57.018+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:57.119 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:57.118+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:57.133 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T07:28:57.163 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:57.164 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:57.173 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:28:57.220 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:57.219+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:57.225 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone6 from 0xe467a2f (overlap 0x25c58f0) to 0x21fe9a4 2024-09-17T07:28:57.321 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:57.320+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:57.339 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-17T07:28:57.341 INFO:teuthology.orchestra.run.smithi045.stdout:600 write 0x398e45d thru 0x399192c (0x34d0 bytes) 2024-09-17T07:28:57.352 INFO:teuthology.orchestra.run.smithi045.stdout:601 write 0x2dc6c51 thru 0x2dcfe1e (0x91ce bytes) 2024-09-17T07:28:57.359 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:57.359 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:57.372 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:28:57.373 INFO:teuthology.orchestra.run.smithi045.stdout:602 read 0x5f765e7 thru 0x5f85979 (0xf393 bytes) 2024-09-17T07:28:57.373 INFO:teuthology.orchestra.run.smithi045.stdout:603 write 0x9ef34de thru 0x9ef709d (0x3bc0 bytes) 2024-09-17T07:28:57.376 INFO:teuthology.orchestra.run.smithi045.stdout:604 read 0xaf7c018 thru 0xaf85b6e (0x9b57 bytes) 2024-09-17T07:28:57.376 INFO:teuthology.orchestra.run.smithi045.stdout:605 read 0xabc2eb thru 0xabde4c (0x1b62 bytes) 2024-09-17T07:28:57.376 INFO:teuthology.orchestra.run.smithi045.stdout:606 read 0xed0e9e thru 0xed90f9 (0x825c bytes) 2024-09-17T07:28:57.377 INFO:teuthology.orchestra.run.smithi045.stdout:607 read 0x687991b thru 0x687b3d8 (0x1abe bytes) 2024-09-17T07:28:57.377 INFO:teuthology.orchestra.run.smithi045.stdout:608 write 0x12c6a6d thru 0x12d2628 (0xbbbc bytes) 2024-09-17T07:28:57.410 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:57.410 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:57.410 INFO:teuthology.orchestra.run.smithi045.stdout:614 write 0xa99b9f4 thru 0xa99f325 (0x3932 bytes) 2024-09-17T07:28:57.413 INFO:teuthology.orchestra.run.smithi045.stdout:615 read 0x785c6cb thru 0x78629dd (0x6313 bytes) 2024-09-17T07:28:57.414 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T07:28:57.414 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:28:57.415 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T07:28:57.415 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T07:28:57.415 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T07:28:57.425 INFO:teuthology.orchestra.run.smithi045.stdout:617 trunc from 0xe467a2f to 0x5a94aa2 2024-09-17T07:28:57.428 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:57.426+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:57.449 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T07:28:57.449 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.3 2024-09-17T07:28:57.449 INFO:tasks.thrashosds.thrasher:remote for osd 3 is ubuntu@smithi045.front.sepia.ceph.com 2024-09-17T07:28:57.449 DEBUG:teuthology.orchestra.run.smithi045:> type ceph-objectstore-tool 2024-09-17T07:28:57.495 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:57.495 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:57.504 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:28:57.509 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T07:28:57.509 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op list-pgs 2024-09-17T07:28:57.522 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:57.522+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:57.622 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:57.622 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:57.622 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:57.621+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:57.630 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:28:57.723 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:57.722+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:57.782 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:57.782 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:57.792 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:28:57.837 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:57.835+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:57.898 INFO:teuthology.orchestra.run.smithi045.stdout:3.c 2024-09-17T07:28:57.898 INFO:teuthology.orchestra.run.smithi045.stdout:3.7 2024-09-17T07:28:57.898 INFO:teuthology.orchestra.run.smithi045.stdout:3.2 2024-09-17T07:28:57.898 INFO:teuthology.orchestra.run.smithi045.stdout:3.8 2024-09-17T07:28:57.898 INFO:teuthology.orchestra.run.smithi045.stdout:3.1e 2024-09-17T07:28:57.898 INFO:teuthology.orchestra.run.smithi045.stdout:3.b 2024-09-17T07:28:57.898 INFO:teuthology.orchestra.run.smithi045.stdout:3.5 2024-09-17T07:28:57.899 INFO:teuthology.orchestra.run.smithi045.stdout:3.18 2024-09-17T07:28:57.899 INFO:teuthology.orchestra.run.smithi045.stdout:3.15 2024-09-17T07:28:57.899 INFO:teuthology.orchestra.run.smithi045.stdout:3.19 2024-09-17T07:28:57.899 INFO:teuthology.orchestra.run.smithi045.stdout:2.1 2024-09-17T07:28:57.901 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:57.901 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:57.910 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:28:57.949 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:28:57.955 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:57.955+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:58.015 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:58.016 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:58.025 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:28:58.025 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:58.024+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:58.075 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:28:58.076 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:28:58.093 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:28:58.093 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:28:58.109 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:28:58.109 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:28:58.126 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:58.125+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:58.131 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-17T07:28:58.174 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op export --pgid 3.7 --file /var/log/ceph/exp.3.7.3 2024-09-17T07:28:58.175 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:58.175 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:58.183 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:28:58.226 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:58.226+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:58.327 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:58.327 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:58.327 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:58.327+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:58.336 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:28:58.428 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:58.428+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:58.440 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:58.440 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:58.449 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:28:58.533 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:28:58.533+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:28:58.592 INFO:teuthology.orchestra.run.smithi045.stderr:Exporting 3.7 info 3.7( v 35'55 (0'0,35'55] local-lis/les=23/24 n=23 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) 2024-09-17T07:28:58.593 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:58.593 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:58.594 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e0c12f65:::rbd_data.1183cff49e3a.0000000000000077:7# 2024-09-17T07:28:58.594 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e0c12f65:::rbd_data.1183cff49e3a.0000000000000077:head# 2024-09-17T07:28:58.594 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e1042ee7:::rbd_data.1183350c2d82.0000000000000062:head# 2024-09-17T07:28:58.594 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e16017d4:::rbd_data.11833dd73a5d.000000000000006a:head# 2024-09-17T07:28:58.594 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e1790928:::rbd_data.11833dd73a5d.0000000000000003:8# 2024-09-17T07:28:58.597 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e1790928:::rbd_data.11833dd73a5d.0000000000000003:head# 2024-09-17T07:28:58.597 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e1b78bc5:::rbd_data.1183cff49e3a.0000000000000130:7# 2024-09-17T07:28:58.597 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e1b78bc5:::rbd_data.1183cff49e3a.0000000000000130:head# 2024-09-17T07:28:58.597 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e36c193a:::rbd_data.118312a5715.0000000000000017:head# 2024-09-17T07:28:58.597 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e3c941af:::rbd_data.11833dd73a5d.0000000000000075:head# 2024-09-17T07:28:58.597 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e3e93c9d:::rbd_data.1183cff49e3a.0000000000000038:head# 2024-09-17T07:28:58.597 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e4c7175c:::rbd_data.11833dd73a5d.0000000000000008:8# 2024-09-17T07:28:58.600 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e4c7175c:::rbd_data.11833dd73a5d.0000000000000008:head# 2024-09-17T07:28:58.601 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e4ea5cb5:::rbd_data.1183350c2d82.0000000000000002:head# 2024-09-17T07:28:58.601 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e51adc2a:::rbd_object_map.11832b928878.0000000000000005:head# 2024-09-17T07:28:58.601 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e550acae:::rbd_data.11833dd73a5d.000000000000007e:head# 2024-09-17T07:28:58.601 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e5a4793c:::rbd_data.11833dd73a5d.000000000000002e:head# 2024-09-17T07:28:58.601 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e5d57cd4:::rbd_data.1183cff49e3a.00000000000000ba:7# 2024-09-17T07:28:58.601 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e5d57cd4:::rbd_data.1183cff49e3a.00000000000000ba:head# 2024-09-17T07:28:58.601 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e6553d3f:::rbd_data.11833dd73a5d.0000000000000046:head# 2024-09-17T07:28:58.601 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e723d14d:::rbd_data.1183cff49e3a.0000000000000072:7# 2024-09-17T07:28:58.601 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e723d14d:::rbd_data.1183cff49e3a.0000000000000072:head# 2024-09-17T07:28:58.601 INFO:teuthology.orchestra.run.smithi045.stderr:Read #3:e7e9ecdf:::rbd_data.11833dd73a5d.0000000000000038:head# 2024-09-17T07:28:58.601 INFO:teuthology.orchestra.run.smithi045.stderr:Export successful 2024-09-17T07:28:58.603 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:28:58.644 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:58.643+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:58.731 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:58.731+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:58.746 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:58.746 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:58.754 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:28:58.832 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:58.831+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:58.868 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --force --op remove --pgid 3.7 2024-09-17T07:28:58.898 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:58.898 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:58.907 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:28:58.951 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:58.950+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:59.009 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:59.009 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:59.018 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:28:59.061 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:28:59.061+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:28:59.121 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:59.122 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:59.130 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:28:59.135 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:28:59.133+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:28:59.164 INFO:teuthology.orchestra.run.smithi045.stdout:618 write 0xc4180a3 thru 0xc4183e6 (0x344 bytes) 2024-09-17T07:28:59.167 INFO:teuthology.orchestra.run.smithi045.stdout:623 read 0x859af9e thru 0x859c2c8 (0x132b bytes) 2024-09-17T07:28:59.167 INFO:teuthology.orchestra.run.smithi045.stdout:629 read 0x9be80f2 thru 0x9bf43ee (0xc2fd bytes) 2024-09-17T07:28:59.167 INFO:teuthology.orchestra.run.smithi045.stdout:630 write 0x18b1446 thru 0x18b69a7 (0x5562 bytes) 2024-09-17T07:28:59.191 INFO:teuthology.orchestra.run.smithi045.stdout:631 punch from 0xc06ea4 to 0xc15bb2, (0xed0e bytes) 2024-09-17T07:28:59.192 INFO:teuthology.orchestra.run.smithi045.stdout:633 write 0x9e4a33 thru 0x9f1eb6 (0xd484 bytes) 2024-09-17T07:28:59.193 INFO:teuthology.orchestra.run.smithi045.stdout:634 write 0xa483b27 thru 0xa488594 (0x4a6e bytes) 2024-09-17T07:28:59.196 INFO:teuthology.orchestra.run.smithi045.stdout:635 punch from 0x357cc1e to 0x3582089, (0x546b bytes) 2024-09-17T07:28:59.196 INFO:teuthology.orchestra.run.smithi045.stdout:636 write 0x1123bfa thru 0x113088b (0xcc92 bytes) 2024-09-17T07:28:59.229 INFO:teuthology.orchestra.run.smithi045.stdout:637 writesame 0x27a2c7d thru 0x27aa543 data_size 0x7(0x78c7 bytes) 2024-09-17T07:28:59.232 INFO:teuthology.orchestra.run.smithi045.stdout:639 read 0x7839cab thru 0x784668b (0xc9e1 bytes) 2024-09-17T07:28:59.232 INFO:teuthology.orchestra.run.smithi045.stdout:640 read 0x2bb64bf thru 0x2bc231d (0xbe5f bytes) 2024-09-17T07:28:59.232 INFO:teuthology.orchestra.run.smithi045.stdout:641 punch from 0x26f98ea to 0x26fb9c2, (0x20d8 bytes) 2024-09-17T07:28:59.233 INFO:teuthology.orchestra.run.smithi045.stdout:642 trunc from 0xc4183e7 to 0xb870f5e 2024-09-17T07:28:59.235 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:59.235+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:59.243 INFO:teuthology.orchestra.run.smithi045.stdout:643 punch from 0x829ca52 to 0x82a4899, (0x7e47 bytes) 2024-09-17T07:28:59.250 INFO:teuthology.orchestra.run.smithi045.stdout:644 trunc from 0xb870f5e to 0x5716fbd 2024-09-17T07:28:59.261 INFO:teuthology.orchestra.run.smithi045.stdout:646 read 0x45c74f8 thru 0x45c930a (0x1e13 bytes) 2024-09-17T07:28:59.276 INFO:teuthology.orchestra.run.smithi045.stdout:647 write 0xdb1fb1f thru 0xdb2462d (0x4b0f bytes) 2024-09-17T07:28:59.280 INFO:teuthology.orchestra.run.smithi045.stdout:650 read 0x6d87b5b thru 0x6d95e6f (0xe315 bytes) 2024-09-17T07:28:59.280 INFO:teuthology.orchestra.run.smithi045.stdout:652 write 0x20a78b thru 0x2179c6 (0xd23c bytes) 2024-09-17T07:28:59.280 INFO:teuthology.orchestra.run.smithi045.stdout:653 write 0x1efc9c6 thru 0x1efd273 (0x8ae bytes) 2024-09-17T07:28:59.282 INFO:teuthology.orchestra.run.smithi045.stdout:654 trunc from 0xdb2462e to 0xcbf134f 2024-09-17T07:28:59.291 INFO:teuthology.orchestra.run.smithi045.stdout:655 read 0x3dbac90 thru 0x3dc2f9e (0x830f bytes) 2024-09-17T07:28:59.291 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:59.292 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:59.294 INFO:teuthology.orchestra.run.smithi045.stdout:657 trunc from 0xcbf134f to 0xa9ff2ab 2024-09-17T07:28:59.298 INFO:teuthology.orchestra.run.smithi045.stdout:658 write 0xa7575a0 thru 0xa761f62 (0xa9c3 bytes) 2024-09-17T07:28:59.301 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:28:59.302 INFO:teuthology.orchestra.run.smithi045.stdout:660 writesame 0x5372c4a thru 0x53774ab data_size 0x55(0x4862 bytes) 2024-09-17T07:28:59.305 INFO:teuthology.orchestra.run.smithi045.stdout:662 trunc from 0xa9ff2ab to 0xc45427c 2024-09-17T07:28:59.310 INFO:teuthology.orchestra.run.smithi045.stdout:663 writesame 0x4b24364 thru 0x4b273c4 data_size 0x1(0x3061 bytes) 2024-09-17T07:28:59.316 INFO:teuthology.orchestra.run.smithi045.stdout:664 write 0xbcda58a thru 0xbce7bf6 (0xd66d bytes) 2024-09-17T07:28:59.318 INFO:teuthology.orchestra.run.smithi045.stdout:665 write 0x1be20e0 thru 0x1bf030b (0xe22c bytes) 2024-09-17T07:28:59.320 INFO:teuthology.orchestra.run.smithi045.stdout:666 write 0x47bc000 thru 0x47bde51 (0x1e52 bytes) 2024-09-17T07:28:59.334 INFO:teuthology.orchestra.run.smithi045.stdout:667 trunc from 0xc45427c to 0x356cde8 2024-09-17T07:28:59.336 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:28:59.335+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:28:59.370 INFO:teuthology.orchestra.run.smithi045.stdout:668 writesame 0xc8668af thru 0xc86efca data_size 0x2(0x871c bytes) 2024-09-17T07:28:59.370 INFO:teuthology.orchestra.run.smithi045.stdout:669 read 0x8ad4f32 thru 0x8ad7676 (0x2745 bytes) 2024-09-17T07:28:59.370 INFO:teuthology.orchestra.run.smithi045.stdout:670 write 0xc8567ba thru 0xc865b7f (0xf3c6 bytes) 2024-09-17T07:28:59.373 INFO:teuthology.orchestra.run.smithi045.stdout:671 write 0xacf2ed2 thru 0xacfdfcd (0xb0fc bytes) 2024-09-17T07:28:59.375 INFO:teuthology.orchestra.run.smithi045.stdout:672 write 0x43f9596 thru 0x43f97b0 (0x21b bytes) 2024-09-17T07:28:59.382 INFO:teuthology.orchestra.run.smithi045.stdout:673 write 0xc93343a thru 0xc9388dd (0x54a4 bytes) 2024-09-17T07:28:59.383 INFO:teuthology.orchestra.run.smithi045.stdout:674 punch from 0x9bbb1ff to 0x9bbc727, (0x1528 bytes) 2024-09-17T07:28:59.383 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:59.383 INFO:teuthology.orchestra.run.smithi045.stdout:676 writesame 0x1f70295 thru 0x1f77fcc data_size 0x4(0x7d38 bytes) 2024-09-17T07:28:59.397 INFO:teuthology.orchestra.run.smithi045.stdout:678 trunc from 0xc9388de to 0x48ef9eb 2024-09-17T07:28:59.407 INFO:teuthology.orchestra.run.smithi045.stdout:679 read 0x2e56049 thru 0x2e610a6 (0xb05e bytes) 2024-09-17T07:28:59.413 INFO:teuthology.orchestra.run.smithi045.stdout:680 write 0x5744b4f thru 0x574cdda (0x828c bytes) 2024-09-17T07:28:59.430 INFO:teuthology.orchestra.run.smithi045.stdout:683 write 0xbdc28ca thru 0xbdcdb90 (0xb2c7 bytes) 2024-09-17T07:28:59.433 INFO:teuthology.orchestra.run.smithi045.stdout:684 writesame 0x11657c9 thru 0x116c815 data_size 0x25(0x704d bytes) 2024-09-17T07:28:59.434 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:28:59.435 INFO:teuthology.orchestra.run.smithi045.stdout:686 writesame 0x2cae944 thru 0x2cb555d data_size 0x2(0x6c1a bytes) 2024-09-17T07:28:59.438 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:59.437+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:59.451 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:59.451 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:59.460 INFO:teuthology.orchestra.run.smithi045.stdout:687 write 0xd5631c2 thru 0xd567cf4 (0x4b33 bytes) 2024-09-17T07:28:59.462 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:28:59.470 INFO:teuthology.orchestra.run.smithi045.stdout:689 write 0xe1cbd80 thru 0xe1d878b (0xca0c bytes) 2024-09-17T07:28:59.472 INFO:teuthology.orchestra.run.smithi045.stdout:690 punch from 0x7ca62c6 to 0x7cae8a3, (0x85dd bytes) 2024-09-17T07:28:59.473 INFO:teuthology.orchestra.run.smithi045.stdout:691 write 0x31ed1fe thru 0x31eda07 (0x80a bytes) 2024-09-17T07:28:59.474 INFO:teuthology.orchestra.run.smithi045.stdout:692 read 0x71ce46d thru 0x71cf319 (0xead bytes) 2024-09-17T07:28:59.474 INFO:teuthology.orchestra.run.smithi045.stdout:693 write 0xa11887 thru 0xa20d29 (0xf4a3 bytes) 2024-09-17T07:28:59.518 INFO:teuthology.orchestra.run.smithi045.stdout:696 trunc from 0xe1d878c to 0x2ee712c 2024-09-17T07:28:59.534 INFO:teuthology.orchestra.run.smithi045.stdout:697 write 0x353f15a thru 0x3545774 (0x661b bytes) 2024-09-17T07:28:59.538 INFO:teuthology.orchestra.run.smithi045.stdout:699 punch from 0xb9fcc0 to 0xbaa0ac, (0xa3ec bytes) 2024-09-17T07:28:59.538 INFO:teuthology.orchestra.run.smithi045.stdout:700 punch from 0x1cdb40b to 0x1ce5b01, (0xa6f6 bytes) 2024-09-17T07:28:59.538 INFO:teuthology.orchestra.run.smithi045.stdout:701 read 0x2dd04ec thru 0x2dd88d7 (0x83ec bytes) 2024-09-17T07:28:59.539 INFO:teuthology.orchestra.run.smithi045.stdout:703 punch from 0x1600a94 to 0x160b1a7, (0xa713 bytes) 2024-09-17T07:28:59.539 INFO:teuthology.orchestra.run.smithi045.stdout:704 punch from 0x208d48c to 0x2099ac7, (0xc63b bytes) 2024-09-17T07:28:59.540 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:59.539+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:59.546 INFO:teuthology.orchestra.run.smithi045.stdout:706 writesame 0x64c88e8 thru 0x64caa5c data_size 0x5(0x2175 bytes) 2024-09-17T07:28:59.561 INFO:teuthology.orchestra.run.smithi045.stdout:707 write 0xe4e7e51 thru 0xe4f7dba (0xff6a bytes) 2024-09-17T07:28:59.564 INFO:teuthology.orchestra.run.smithi045.stdout:708 punch from 0x9165eae to 0x9173ac2, (0xdc14 bytes) 2024-09-17T07:28:59.564 INFO:teuthology.orchestra.run.smithi045.stdout:709 read 0xd35a8d5 thru 0xd365dca (0xb4f6 bytes) 2024-09-17T07:28:59.564 INFO:teuthology.orchestra.run.smithi045.stdout:710 write 0x1656ef4 thru 0x165f511 (0x861e bytes) 2024-09-17T07:28:59.565 INFO:teuthology.orchestra.run.smithi045.stdout:711 clone 8 order 24 su 262144 sc 13 2024-09-17T07:28:59.577 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:28:59.577 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:28:59.585 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:28:59.640 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:28:59.639+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:28:59.693 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:28:59.694 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:59.702 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:28:59.742 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:59.741+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:59.842 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:28:59.842+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:28:59.853 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:59.853 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:28:59.853 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:28:59.853 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:28:59.854 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:28:59.863 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:28:59.943 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:28:59.943+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:28:59.977 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:28:59.977 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:28:59.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:28:59.977 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:28:59.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.978 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:818 3.12 3:48316f5c:::rbd_data.1183350c2d82.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:59.978 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.985133+0000", 2024-09-17T07:28:59.978 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 10.990167971, 2024-09-17T07:28:59.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.012417920000000001, 2024-09-17T07:28:59.978 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:59.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:59.978 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:59.978 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:59.978 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:59.978 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:59.978 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 818 2024-09-17T07:28:59.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.978 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:59.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:59.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985133+0000", 2024-09-17T07:28:59.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:59.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:59.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985133+0000", 2024-09-17T07:28:59.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:59.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:59.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985134+0000", 2024-09-17T07:28:59.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3209999999999999e-06 2024-09-17T07:28:59.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:59.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985141+0000", 2024-09-17T07:28:59.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.443e-06 2024-09-17T07:28:59.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:59.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985154+0000", 2024-09-17T07:28:59.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3302000000000001e-05 2024-09-17T07:28:59.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:59.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985162+0000", 2024-09-17T07:28:59.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.644e-06 2024-09-17T07:28:59.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:59.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985226+0000", 2024-09-17T07:28:59.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4620000000000001e-05 2024-09-17T07:28:59.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:59.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.985333+0000", 2024-09-17T07:28:59.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010641099999999999 2024-09-17T07:28:59.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:28:59.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.987842+0000", 2024-09-17T07:28:59.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025096720000000001 2024-09-17T07:28:59.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:59.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.997364+0000", 2024-09-17T07:28:59.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.009521346 2024-09-17T07:28:59.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:59.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.997424+0000", 2024-09-17T07:28:59.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0282000000000003e-05 2024-09-17T07:28:59.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:59.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.997495+0000", 2024-09-17T07:28:59.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1334000000000005e-05 2024-09-17T07:28:59.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:59.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.997551+0000", 2024-09-17T07:28:59.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5544999999999998e-05 2024-09-17T07:28:59.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:59.984 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:59.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:59.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.984 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:877 3.17 3:edbcd0af:::rbd_data.1183350c2d82.00000000000000a7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:59.984 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.069283+0000", 2024-09-17T07:28:59.984 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 10.906017577, 2024-09-17T07:28:59.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0065074670000000003, 2024-09-17T07:28:59.984 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:59.984 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:59.984 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:59.984 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:59.984 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:59.985 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:59.985 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 877 2024-09-17T07:28:59.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.985 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:59.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:59.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069283+0000", 2024-09-17T07:28:59.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:59.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:59.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069283+0000", 2024-09-17T07:28:59.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:59.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:59.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069286+0000", 2024-09-17T07:28:59.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:28:59.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:59.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069293+0000", 2024-09-17T07:28:59.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7030000000000006e-06 2024-09-17T07:28:59.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069314+0000", 2024-09-17T07:28:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.033e-05 2024-09-17T07:28:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069328+0000", 2024-09-17T07:28:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4044e-05 2024-09-17T07:28:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069400+0000", 2024-09-17T07:28:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2360999999999994e-05 2024-09-17T07:28:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:59.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069496+0000", 2024-09-17T07:28:59.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6267999999999994e-05 2024-09-17T07:28:59.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:28:59.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.069619+0000", 2024-09-17T07:28:59.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012247199999999999 2024-09-17T07:28:59.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:59.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.070568+0000", 2024-09-17T07:28:59.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00094935899999999999 2024-09-17T07:28:59.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:59.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075682+0000", 2024-09-17T07:28:59.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.005114164 2024-09-17T07:28:59.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:59.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075743+0000", 2024-09-17T07:28:59.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0421000000000003e-05 2024-09-17T07:28:59.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:59.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.075791+0000", 2024-09-17T07:28:59.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8087999999999998e-05 2024-09-17T07:28:59.990 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:59.990 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:59.990 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:59.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.990 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:882 3.1 3:871a2656:::rbd_data.1183350c2d82.00000000000000c5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-17T07:28:59.990 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.073104+0000", 2024-09-17T07:28:59.990 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 10.902197469000001, 2024-09-17T07:28:59.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00561532, 2024-09-17T07:28:59.990 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:59.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:59.991 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:59.991 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:59.991 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:59.991 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:59.991 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 882 2024-09-17T07:28:59.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.991 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:59.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.991 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:59.991 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073104+0000", 2024-09-17T07:28:59.991 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:59.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.991 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:59.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073104+0000", 2024-09-17T07:28:59.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:59.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:59.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073105+0000", 2024-09-17T07:28:59.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.263e-06 2024-09-17T07:28:59.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:59.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073110+0000", 2024-09-17T07:28:59.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-06 2024-09-17T07:28:59.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:59.993 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073125+0000", 2024-09-17T07:28:59.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5509999999999999e-05 2024-09-17T07:28:59.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:59.993 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073133+0000", 2024-09-17T07:28:59.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0600000000000008e-06 2024-09-17T07:28:59.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:28:59.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073204+0000", 2024-09-17T07:28:59.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0223999999999997e-05 2024-09-17T07:28:59.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:28:59.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073297+0000", 2024-09-17T07:28:59.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3134999999999995e-05 2024-09-17T07:28:59.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:28:59.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073416+0000", 2024-09-17T07:28:59.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011952599999999999 2024-09-17T07:28:59.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:28:59.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.073903+0000", 2024-09-17T07:28:59.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000486417 2024-09-17T07:28:59.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:28:59.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.078659+0000", 2024-09-17T07:28:59.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0047562400000000001 2024-09-17T07:28:59.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:28:59.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.078689+0000", 2024-09-17T07:28:59.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0266000000000001e-05 2024-09-17T07:28:59.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:28:59.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.078719+0000", 2024-09-17T07:28:59.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9719000000000001e-05 2024-09-17T07:28:59.996 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:59.996 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:28:59.996 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:28:59.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.996 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1060 3.12 3:4b321d73:::rbd_data.1183350c2d82.0000000000000069:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:28:59.996 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.303259+0000", 2024-09-17T07:28:59.997 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 10.672041570999999, 2024-09-17T07:28:59.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0063303309999999998, 2024-09-17T07:28:59.997 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:28:59.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:28:59.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:28:59.997 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:28:59.997 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:28:59.997 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:28:59.997 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1060 2024-09-17T07:28:59.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.997 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:28:59.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.997 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:28:59.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303259+0000", 2024-09-17T07:28:59.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:59.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:28:59.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303259+0000", 2024-09-17T07:28:59.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:28:59.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:28:59.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303261+0000", 2024-09-17T07:28:59.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8530000000000001e-06 2024-09-17T07:28:59.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:28:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303269+0000", 2024-09-17T07:28:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.255e-06 2024-09-17T07:28:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:28:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303287+0000", 2024-09-17T07:28:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7960000000000001e-05 2024-09-17T07:28:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:28:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:28:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303295+0000", 2024-09-17T07:28:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1589999999999993e-06 2024-09-17T07:28:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:28:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303376+0000", 2024-09-17T07:29:00.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1132999999999997e-05 2024-09-17T07:29:00.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303478+0000", 2024-09-17T07:29:00.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000102515 2024-09-17T07:29:00.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:00.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.303628+0000", 2024-09-17T07:29:00.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014999699999999999 2024-09-17T07:29:00.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.304599+0000", 2024-09-17T07:29:00.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097082099999999997 2024-09-17T07:29:00.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.309516+0000", 2024-09-17T07:29:00.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0049165099999999998 2024-09-17T07:29:00.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.309548+0000", 2024-09-17T07:29:00.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2718000000000001e-05 2024-09-17T07:29:00.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.309590+0000", 2024-09-17T07:29:00.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1409999999999998e-05 2024-09-17T07:29:00.002 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.002 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.002 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.002 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1180 3.12 3:49b4e8b1:::rbd_data.1183350c2d82.00000000000000e1:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T07:29:00.002 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.474967+0000", 2024-09-17T07:29:00.003 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 10.500334285999999, 2024-09-17T07:29:00.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.016363323999999999, 2024-09-17T07:29:00.003 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.003 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:00.003 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.003 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.003 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.003 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1180 2024-09-17T07:29:00.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.003 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.003 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.003 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.474967+0000", 2024-09-17T07:29:00.004 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.004 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.004 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.474967+0000", 2024-09-17T07:29:00.004 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.004 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.004 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.474967+0000", 2024-09-17T07:29:00.004 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2399999999999998e-07 2024-09-17T07:29:00.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.004 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.474972+0000", 2024-09-17T07:29:00.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5770000000000004e-06 2024-09-17T07:29:00.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.474979+0000", 2024-09-17T07:29:00.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6730000000000003e-06 2024-09-17T07:29:00.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.474983+0000", 2024-09-17T07:29:00.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1180000000000002e-06 2024-09-17T07:29:00.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475028+0000", 2024-09-17T07:29:00.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5354e-05 2024-09-17T07:29:00.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.475071+0000", 2024-09-17T07:29:00.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2998999999999999e-05 2024-09-17T07:29:00.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:00.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.478091+0000", 2024-09-17T07:29:00.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003020193 2024-09-17T07:29:00.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.491213+0000", 2024-09-17T07:29:00.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013121226 2024-09-17T07:29:00.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.491263+0000", 2024-09-17T07:29:00.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0821e-05 2024-09-17T07:29:00.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.491300+0000", 2024-09-17T07:29:00.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6650000000000003e-05 2024-09-17T07:29:00.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.491330+0000", 2024-09-17T07:29:00.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0088999999999999e-05 2024-09-17T07:29:00.008 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.008 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.008 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.008 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1644 3.1 3:807c7886:::rbd_data.11832b928878.00000000000002d6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:29:00.009 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.712491+0000", 2024-09-17T07:29:00.009 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.2628104499999999, 2024-09-17T07:29:00.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0053083990000000001, 2024-09-17T07:29:00.009 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.009 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.009 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:00.009 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.009 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.009 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.009 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1644 2024-09-17T07:29:00.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.009 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.010 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.712491+0000", 2024-09-17T07:29:00.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.010 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.010 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.712491+0000", 2024-09-17T07:29:00.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.010 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.010 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.712491+0000", 2024-09-17T07:29:00.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2799999999999996e-07 2024-09-17T07:29:00.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.712496+0000", 2024-09-17T07:29:00.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6349999999999997e-06 2024-09-17T07:29:00.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.712513+0000", 2024-09-17T07:29:00.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7410000000000001e-05 2024-09-17T07:29:00.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.712519+0000", 2024-09-17T07:29:00.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9109999999999998e-06 2024-09-17T07:29:00.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.713090+0000", 2024-09-17T07:29:00.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00057126599999999996 2024-09-17T07:29:00.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.713174+0000", 2024-09-17T07:29:00.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3397000000000001e-05 2024-09-17T07:29:00.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:29:00.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.713286+0000", 2024-09-17T07:29:00.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112637 2024-09-17T07:29:00.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.713808+0000", 2024-09-17T07:29:00.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00052124199999999999 2024-09-17T07:29:00.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.717689+0000", 2024-09-17T07:29:00.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0038810369999999999 2024-09-17T07:29:00.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.717754+0000", 2024-09-17T07:29:00.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5171000000000003e-05 2024-09-17T07:29:00.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.717799+0000", 2024-09-17T07:29:00.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5164999999999999e-05 2024-09-17T07:29:00.014 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.014 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.014 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:1695 3.1 3:87cc7aab:::rbd_data.11832b928878.00000000000001f5:head [set-alloc-hint object_size 262144 write_size 262144,write 163840~20075] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:29:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.739319+0000", 2024-09-17T07:29:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.2359818120000003, 2024-09-17T07:29:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0054410830000000002, 2024-09-17T07:29:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 1695 2024-09-17T07:29:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.739319+0000", 2024-09-17T07:29:00.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.739319+0000", 2024-09-17T07:29:00.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.739320+0000", 2024-09-17T07:29:00.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6099999999999999e-07 2024-09-17T07:29:00.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.739330+0000", 2024-09-17T07:29:00.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0117e-05 2024-09-17T07:29:00.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.739341+0000", 2024-09-17T07:29:00.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0723000000000001e-05 2024-09-17T07:29:00.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.739346+0000", 2024-09-17T07:29:00.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8300000000000003e-06 2024-09-17T07:29:00.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.739390+0000", 2024-09-17T07:29:00.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4002999999999999e-05 2024-09-17T07:29:00.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.739480+0000", 2024-09-17T07:29:00.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0239000000000001e-05 2024-09-17T07:29:00.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:29:00.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.739569+0000", 2024-09-17T07:29:00.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9408999999999997e-05 2024-09-17T07:29:00.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.740149+0000", 2024-09-17T07:29:00.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00057936399999999996 2024-09-17T07:29:00.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.744693+0000", 2024-09-17T07:29:00.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045446599999999998 2024-09-17T07:29:00.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.744734+0000", 2024-09-17T07:29:00.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0904e-05 2024-09-17T07:29:00.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.744760+0000", 2024-09-17T07:29:00.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5973e-05 2024-09-17T07:29:00.020 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.020 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.020 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.021 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:2206 3.1b 3:dda19077:::rbd_data.11832b928878.0000000000000131:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T07:29:00.021 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:51.143134+0000", 2024-09-17T07:29:00.021 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 8.8321674730000002, 2024-09-17T07:29:00.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0053003379999999999, 2024-09-17T07:29:00.021 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.021 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.021 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:00.021 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.021 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.021 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.021 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 2206 2024-09-17T07:29:00.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.022 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.143134+0000", 2024-09-17T07:29:00.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.143134+0000", 2024-09-17T07:29:00.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.143135+0000", 2024-09-17T07:29:00.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1510000000000001e-06 2024-09-17T07:29:00.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.143142+0000", 2024-09-17T07:29:00.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1910000000000003e-06 2024-09-17T07:29:00.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.143173+0000", 2024-09-17T07:29:00.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1136999999999997e-05 2024-09-17T07:29:00.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.143182+0000", 2024-09-17T07:29:00.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5390000000000007e-06 2024-09-17T07:29:00.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.143349+0000", 2024-09-17T07:29:00.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00016730200000000001 2024-09-17T07:29:00.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.143432+0000", 2024-09-17T07:29:00.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2614999999999994e-05 2024-09-17T07:29:00.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:29:00.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.143547+0000", 2024-09-17T07:29:00.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011579 2024-09-17T07:29:00.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.144464+0000", 2024-09-17T07:29:00.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00091682600000000003 2024-09-17T07:29:00.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.148374+0000", 2024-09-17T07:29:00.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0039098700000000002 2024-09-17T07:29:00.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.026 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.026 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.148406+0000", 2024-09-17T07:29:00.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1513000000000001e-05 2024-09-17T07:29:00.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.026 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.026 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.148434+0000", 2024-09-17T07:29:00.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8404000000000001e-05 2024-09-17T07:29:00.026 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.026 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.026 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.027 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:6761 3.12 3:49c1bd6d:::rbd_data.1183cff49e3a.0000000000000021:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:00.027 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.801470+0000", 2024-09-17T07:29:00.027 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.1738305850000001, 2024-09-17T07:29:00.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0052213930000000004, 2024-09-17T07:29:00.027 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.027 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.027 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:00.027 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.027 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.027 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.027 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 6761 2024-09-17T07:29:00.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.028 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.801470+0000", 2024-09-17T07:29:00.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.801470+0000", 2024-09-17T07:29:00.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.801471+0000", 2024-09-17T07:29:00.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6400000000000002e-07 2024-09-17T07:29:00.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.029 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.029 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.801474+0000", 2024-09-17T07:29:00.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0400000000000001e-06 2024-09-17T07:29:00.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.032 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.032 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.801484+0000", 2024-09-17T07:29:00.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4590000000000004e-06 2024-09-17T07:29:00.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.801490+0000", 2024-09-17T07:29:00.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7240000000000003e-06 2024-09-17T07:29:00.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.802689+0000", 2024-09-17T07:29:00.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011984809999999999 2024-09-17T07:29:00.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.802769+0000", 2024-09-17T07:29:00.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9993000000000005e-05 2024-09-17T07:29:00.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:00.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.802877+0000", 2024-09-17T07:29:00.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000108052 2024-09-17T07:29:00.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.803886+0000", 2024-09-17T07:29:00.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010088499999999999 2024-09-17T07:29:00.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.035 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.806610+0000", 2024-09-17T07:29:00.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027244639999999998 2024-09-17T07:29:00.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.035 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.035 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.806651+0000", 2024-09-17T07:29:00.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1282999999999998e-05 2024-09-17T07:29:00.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.035 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.035 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.806692+0000", 2024-09-17T07:29:00.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0383000000000003e-05 2024-09-17T07:29:00.035 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.035 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.036 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.036 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:6958 3.12 3:4934c8bb:::rbd_data.1183cff49e3a.0000000000000022:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:00.036 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.811159+0000", 2024-09-17T07:29:00.036 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.1641417780000003, 2024-09-17T07:29:00.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0066576079999999998, 2024-09-17T07:29:00.036 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.036 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.036 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:00.036 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.036 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.036 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.036 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 6958 2024-09-17T07:29:00.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.037 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.811159+0000", 2024-09-17T07:29:00.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.811159+0000", 2024-09-17T07:29:00.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.811160+0000", 2024-09-17T07:29:00.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.06e-07 2024-09-17T07:29:00.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.811163+0000", 2024-09-17T07:29:00.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1370000000000002e-06 2024-09-17T07:29:00.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.811169+0000", 2024-09-17T07:29:00.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2360000000000001e-06 2024-09-17T07:29:00.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.039 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.039 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.811174+0000", 2024-09-17T07:29:00.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9930000000000003e-06 2024-09-17T07:29:00.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.039 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.039 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.812074+0000", 2024-09-17T07:29:00.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00090006699999999995 2024-09-17T07:29:00.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.039 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.039 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.812124+0000", 2024-09-17T07:29:00.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9897e-05 2024-09-17T07:29:00.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.040 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:00.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.812235+0000", 2024-09-17T07:29:00.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111118 2024-09-17T07:29:00.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.040 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.813063+0000", 2024-09-17T07:29:00.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00082722099999999995 2024-09-17T07:29:00.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.040 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.041 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.817748+0000", 2024-09-17T07:29:00.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0046850370000000004 2024-09-17T07:29:00.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.041 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.041 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.817779+0000", 2024-09-17T07:29:00.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1000000000000001e-05 2024-09-17T07:29:00.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.041 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.041 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.817817+0000", 2024-09-17T07:29:00.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8296000000000001e-05 2024-09-17T07:29:00.042 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.042 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.042 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.042 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:8901 3.1f 3:ffa28dc9:::rbd_data.1183cff49e3a.000000000000005c:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T07:29:00.042 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.341545+0000", 2024-09-17T07:29:00.042 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.6337560189999998, 2024-09-17T07:29:00.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0078085919999999996, 2024-09-17T07:29:00.042 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.042 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.042 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:00.042 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.043 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.043 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.043 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 8901 2024-09-17T07:29:00.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.043 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.341545+0000", 2024-09-17T07:29:00.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.341545+0000", 2024-09-17T07:29:00.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.044 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.341546+0000", 2024-09-17T07:29:00.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3200000000000004e-07 2024-09-17T07:29:00.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.044 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.341548+0000", 2024-09-17T07:29:00.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5280000000000002e-06 2024-09-17T07:29:00.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.341556+0000", 2024-09-17T07:29:00.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6340000000000001e-06 2024-09-17T07:29:00.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.045 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.341560+0000", 2024-09-17T07:29:00.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.189e-06 2024-09-17T07:29:00.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.045 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.342496+0000", 2024-09-17T07:29:00.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00093552499999999999 2024-09-17T07:29:00.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.342550+0000", 2024-09-17T07:29:00.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4011000000000002e-05 2024-09-17T07:29:00.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:29:00.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.342665+0000", 2024-09-17T07:29:00.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000115297 2024-09-17T07:29:00.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.343388+0000", 2024-09-17T07:29:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072343099999999997 2024-09-17T07:29:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.349247+0000", 2024-09-17T07:29:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0058583589999999996 2024-09-17T07:29:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.349311+0000", 2024-09-17T07:29:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3948999999999994e-05 2024-09-17T07:29:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.047 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.349354+0000", 2024-09-17T07:29:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2936999999999999e-05 2024-09-17T07:29:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:8921 3.1f 3:fbdaace4:::rbd_data.1183cff49e3a.000000000000005e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T07:29:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.354952+0000", 2024-09-17T07:29:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.6203491110000003, 2024-09-17T07:29:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.005445152, 2024-09-17T07:29:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 8921 2024-09-17T07:29:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.354952+0000", 2024-09-17T07:29:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.049 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.354952+0000", 2024-09-17T07:29:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.354953+0000", 2024-09-17T07:29:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6599999999999994e-07 2024-09-17T07:29:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.355038+0000", 2024-09-17T07:29:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4888000000000004e-05 2024-09-17T07:29:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.355048+0000", 2024-09-17T07:29:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0472000000000001e-05 2024-09-17T07:29:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.355053+0000", 2024-09-17T07:29:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9570000000000001e-06 2024-09-17T07:29:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.355105+0000", 2024-09-17T07:29:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2163999999999997e-05 2024-09-17T07:29:00.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.355174+0000", 2024-09-17T07:29:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9076000000000001e-05 2024-09-17T07:29:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:29:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.355334+0000", 2024-09-17T07:29:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015978100000000001 2024-09-17T07:29:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.356563+0000", 2024-09-17T07:29:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012288819999999999 2024-09-17T07:29:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.360221+0000", 2024-09-17T07:29:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036581930000000001 2024-09-17T07:29:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.360356+0000", 2024-09-17T07:29:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000134633 2024-09-17T07:29:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.053 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.360397+0000", 2024-09-17T07:29:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1140000000000003e-05 2024-09-17T07:29:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:8922 3.1f 3:fbdaace4:::rbd_data.1183cff49e3a.000000000000005e:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T07:29:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.355079+0000", 2024-09-17T07:29:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.6202222690000001, 2024-09-17T07:29:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0076464619999999997, 2024-09-17T07:29:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.054 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 8922 2024-09-17T07:29:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.355079+0000", 2024-09-17T07:29:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.055 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.355079+0000", 2024-09-17T07:29:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.355079+0000", 2024-09-17T07:29:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1099999999999995e-07 2024-09-17T07:29:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.355083+0000", 2024-09-17T07:29:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.608e-06 2024-09-17T07:29:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.355091+0000", 2024-09-17T07:29:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8110000000000003e-06 2024-09-17T07:29:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.355097+0000", 2024-09-17T07:29:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0390000000000001e-06 2024-09-17T07:29:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.355940+0000", 2024-09-17T07:29:00.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00084265399999999997 2024-09-17T07:29:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.355997+0000", 2024-09-17T07:29:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7283e-05 2024-09-17T07:29:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:29:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.356115+0000", 2024-09-17T07:29:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000118032 2024-09-17T07:29:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.356840+0000", 2024-09-17T07:29:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072499000000000001 2024-09-17T07:29:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.362626+0000", 2024-09-17T07:29:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0057861079999999999 2024-09-17T07:29:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.362678+0000", 2024-09-17T07:29:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2237000000000001e-05 2024-09-17T07:29:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.362725+0000", 2024-09-17T07:29:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6989e-05 2024-09-17T07:29:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:8938 3.12 3:4fd4df9d:::rbd_data.1183cff49e3a.00000000000000b9:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T07:29:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.367281+0000", 2024-09-17T07:29:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.6080196400000002, 2024-09-17T07:29:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0065418910000000002, 2024-09-17T07:29:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.060 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 8938 2024-09-17T07:29:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.367281+0000", 2024-09-17T07:29:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.367281+0000", 2024-09-17T07:29:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.367282+0000", 2024-09-17T07:29:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4399999999999999e-07 2024-09-17T07:29:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.367286+0000", 2024-09-17T07:29:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9489999999999998e-06 2024-09-17T07:29:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.367295+0000", 2024-09-17T07:29:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2250000000000006e-06 2024-09-17T07:29:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.367301+0000", 2024-09-17T07:29:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9410000000000004e-06 2024-09-17T07:29:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.367357+0000", 2024-09-17T07:29:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5179000000000001e-05 2024-09-17T07:29:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.367462+0000", 2024-09-17T07:29:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105302 2024-09-17T07:29:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.367639+0000", 2024-09-17T07:29:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000176906 2024-09-17T07:29:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.369172+0000", 2024-09-17T07:29:00.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001533445 2024-09-17T07:29:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.373723+0000", 2024-09-17T07:29:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045504380000000004 2024-09-17T07:29:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.373782+0000", 2024-09-17T07:29:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9533000000000001e-05 2024-09-17T07:29:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.065 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.373823+0000", 2024-09-17T07:29:00.066 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1128999999999999e-05 2024-09-17T07:29:00.066 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.066 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.066 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.066 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:8946 3.1b 3:dec31326:::rbd_data.1183cff49e3a.0000000000000121:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T07:29:00.066 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.368632+0000", 2024-09-17T07:29:00.066 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.6066685779999998, 2024-09-17T07:29:00.066 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.008143806, 2024-09-17T07:29:00.066 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.067 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:00.067 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.067 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.067 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.067 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 8946 2024-09-17T07:29:00.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.067 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.067 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.067 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.368632+0000", 2024-09-17T07:29:00.067 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.067 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.068 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.368632+0000", 2024-09-17T07:29:00.068 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.068 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.068 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.368633+0000", 2024-09-17T07:29:00.068 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9899999999999999e-07 2024-09-17T07:29:00.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.068 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.068 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.368638+0000", 2024-09-17T07:29:00.068 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.177e-06 2024-09-17T07:29:00.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.069 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.069 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.368652+0000", 2024-09-17T07:29:00.069 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4132e-05 2024-09-17T07:29:00.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.069 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.069 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.368659+0000", 2024-09-17T07:29:00.069 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6729999999999992e-06 2024-09-17T07:29:00.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.069 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.069 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.368739+0000", 2024-09-17T07:29:00.069 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9414999999999998e-05 2024-09-17T07:29:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.368832+0000", 2024-09-17T07:29:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3238000000000004e-05 2024-09-17T07:29:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:29:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.369022+0000", 2024-09-17T07:29:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000190297 2024-09-17T07:29:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.370129+0000", 2024-09-17T07:29:00.070 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011063710000000001 2024-09-17T07:29:00.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.071 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.071 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.376674+0000", 2024-09-17T07:29:00.071 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.006545281 2024-09-17T07:29:00.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.071 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.071 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.376735+0000", 2024-09-17T07:29:00.071 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0961999999999997e-05 2024-09-17T07:29:00.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.071 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.072 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.376776+0000", 2024-09-17T07:29:00.072 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1360999999999999e-05 2024-09-17T07:29:00.072 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.072 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.072 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.072 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:8950 3.1f 3:ff4b6120:::rbd_data.1183cff49e3a.0000000000000125:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T07:29:00.072 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.370900+0000", 2024-09-17T07:29:00.072 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.6044005569999999, 2024-09-17T07:29:00.072 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0079747499999999992, 2024-09-17T07:29:00.072 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.072 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.073 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:00.073 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.073 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.073 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.073 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 8950 2024-09-17T07:29:00.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.073 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.073 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.073 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.370900+0000", 2024-09-17T07:29:00.073 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.074 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.074 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.370900+0000", 2024-09-17T07:29:00.074 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.074 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.074 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.370902+0000", 2024-09-17T07:29:00.074 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.105e-06 2024-09-17T07:29:00.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.074 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.074 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.370907+0000", 2024-09-17T07:29:00.074 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0000000000000004e-06 2024-09-17T07:29:00.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.075 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.075 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.370920+0000", 2024-09-17T07:29:00.075 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3878e-05 2024-09-17T07:29:00.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.075 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.075 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.370926+0000", 2024-09-17T07:29:00.075 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8200000000000002e-06 2024-09-17T07:29:00.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.075 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.075 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.370981+0000", 2024-09-17T07:29:00.076 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5064e-05 2024-09-17T07:29:00.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.076 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.076 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.371047+0000", 2024-09-17T07:29:00.076 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5531000000000007e-05 2024-09-17T07:29:00.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.076 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:29:00.076 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.371185+0000", 2024-09-17T07:29:00.076 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013822199999999999 2024-09-17T07:29:00.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.076 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.077 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.372002+0000", 2024-09-17T07:29:00.077 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00081690699999999996 2024-09-17T07:29:00.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.077 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.077 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.077 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.378808+0000", 2024-09-17T07:29:00.077 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0068062189999999996 2024-09-17T07:29:00.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.077 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.077 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.077 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.378838+0000", 2024-09-17T07:29:00.077 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9848e-05 2024-09-17T07:29:00.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.078 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.078 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.078 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.378875+0000", 2024-09-17T07:29:00.078 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7156000000000001e-05 2024-09-17T07:29:00.078 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.078 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.078 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.078 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.078 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:8955 3.1b 3:d9866841:::rbd_data.1183cff49e3a.0000000000000132:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-17T07:29:00.078 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.374436+0000", 2024-09-17T07:29:00.078 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.600864863, 2024-09-17T07:29:00.078 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0071762079999999999, 2024-09-17T07:29:00.079 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.079 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.079 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:00.079 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.079 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.079 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.079 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 8955 2024-09-17T07:29:00.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.079 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.079 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.079 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.079 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.374436+0000", 2024-09-17T07:29:00.079 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.080 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.080 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.080 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.374436+0000", 2024-09-17T07:29:00.080 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.080 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.080 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.080 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.374437+0000", 2024-09-17T07:29:00.081 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9299999999999997e-07 2024-09-17T07:29:00.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.081 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.081 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.081 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.374441+0000", 2024-09-17T07:29:00.081 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8990000000000003e-06 2024-09-17T07:29:00.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.081 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.081 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.081 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.374452+0000", 2024-09-17T07:29:00.081 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0716999999999999e-05 2024-09-17T07:29:00.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.081 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.082 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.082 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.374459+0000", 2024-09-17T07:29:00.082 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9330000000000004e-06 2024-09-17T07:29:00.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.082 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.082 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.374511+0000", 2024-09-17T07:29:00.082 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2126999999999997e-05 2024-09-17T07:29:00.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.082 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.082 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.374556+0000", 2024-09-17T07:29:00.082 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5631999999999999e-05 2024-09-17T07:29:00.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.083 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:29:00.083 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.374648+0000", 2024-09-17T07:29:00.083 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1955999999999998e-05 2024-09-17T07:29:00.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.083 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.083 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.375226+0000", 2024-09-17T07:29:00.083 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00057726700000000001 2024-09-17T07:29:00.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.083 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.084 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.381557+0000", 2024-09-17T07:29:00.084 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0063319680000000003 2024-09-17T07:29:00.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.084 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.084 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.381587+0000", 2024-09-17T07:29:00.084 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0001999999999998e-05 2024-09-17T07:29:00.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.084 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.084 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.381612+0000", 2024-09-17T07:29:00.084 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4913999999999999e-05 2024-09-17T07:29:00.084 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.085 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.085 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.085 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.085 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:00.085 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.84269353, 2024-09-17T07:29:00.085 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:29:00.085 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.085 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.085 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.085 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.085 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.086 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:29:00.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.086 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.086 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.086 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.086 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:00.086 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.086 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.086 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.086 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:00.086 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.087 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.087 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:29:00.087 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:29:00.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.087 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.087 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:29:00.087 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:29:00.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.087 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.087 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:29:00.087 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:29:00.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.088 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.088 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:29:00.088 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:29:00.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.088 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.088 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:29:00.088 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:29:00.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.088 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:29:00.088 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:29:00.089 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:29:00.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.089 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.089 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:29:00.089 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:29:00.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.089 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.089 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:29:00.089 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:29:00.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.089 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.090 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:29:00.090 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:29:00.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.090 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.090 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:29:00.090 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:29:00.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.090 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.090 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:29:00.090 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:29:00.090 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.090 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.091 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.091 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14607 3.1b 3:d9e58a41:::rbd_data.11837e68aeab.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5913594~52370] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-17T07:29:00.091 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:59.208099+0000", 2024-09-17T07:29:00.091 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.76720163100000005, 2024-09-17T07:29:00.091 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.021548115999999999, 2024-09-17T07:29:00.091 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.091 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.091 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:00.091 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.091 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.091 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.091 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14607 2024-09-17T07:29:00.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.092 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.092 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.092 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.208099+0000", 2024-09-17T07:29:00.092 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.092 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.092 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.208099+0000", 2024-09-17T07:29:00.092 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.092 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.093 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.208102+0000", 2024-09-17T07:29:00.093 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8040000000000002e-06 2024-09-17T07:29:00.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.093 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.093 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.211950+0000", 2024-09-17T07:29:00.093 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0038476909999999999 2024-09-17T07:29:00.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.093 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.093 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.211977+0000", 2024-09-17T07:29:00.093 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7413999999999999e-05 2024-09-17T07:29:00.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.094 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.094 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.211990+0000", 2024-09-17T07:29:00.094 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2925000000000001e-05 2024-09-17T07:29:00.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.094 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.094 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.212082+0000", 2024-09-17T07:29:00.094 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2211000000000002e-05 2024-09-17T07:29:00.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.094 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.094 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.212205+0000", 2024-09-17T07:29:00.094 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012216000000000001 2024-09-17T07:29:00.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.095 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:29:00.095 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.212747+0000", 2024-09-17T07:29:00.095 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000542823 2024-09-17T07:29:00.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.095 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.095 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.222819+0000", 2024-09-17T07:29:00.095 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010071208999999999 2024-09-17T07:29:00.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.095 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.095 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.229577+0000", 2024-09-17T07:29:00.096 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0067583809999999999 2024-09-17T07:29:00.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.096 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.096 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.229616+0000", 2024-09-17T07:29:00.096 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9360000000000003e-05 2024-09-17T07:29:00.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.096 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.096 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.229648+0000", 2024-09-17T07:29:00.096 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1137999999999999e-05 2024-09-17T07:29:00.096 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.096 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14866 3.17 3:ebf494f0:::rbd_data.11837e68aeab.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-17T07:29:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:59.442097+0000", 2024-09-17T07:29:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.53320397100000005, 2024-09-17T07:29:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011035921000000001, 2024-09-17T07:29:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.097 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14866 2024-09-17T07:29:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.442097+0000", 2024-09-17T07:29:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.442097+0000", 2024-09-17T07:29:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.098 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.442099+0000", 2024-09-17T07:29:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5540000000000001e-06 2024-09-17T07:29:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.442924+0000", 2024-09-17T07:29:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00082564599999999998 2024-09-17T07:29:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.442948+0000", 2024-09-17T07:29:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3427000000000001e-05 2024-09-17T07:29:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.099 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.442957+0000", 2024-09-17T07:29:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4439999999999997e-06 2024-09-17T07:29:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.443032+0000", 2024-09-17T07:29:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4506999999999997e-05 2024-09-17T07:29:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.443201+0000", 2024-09-17T07:29:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00016903299999999999 2024-09-17T07:29:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.100 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:29:00.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.444541+0000", 2024-09-17T07:29:00.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013398360000000001 2024-09-17T07:29:00.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:00.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.452026+0000", 2024-09-17T07:29:00.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074851839999999998 2024-09-17T07:29:00.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:00.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.453061+0000", 2024-09-17T07:29:00.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010350870000000001 2024-09-17T07:29:00.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:00.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.453102+0000", 2024-09-17T07:29:00.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0735999999999998e-05 2024-09-17T07:29:00.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.453133+0000", 2024-09-17T07:29:00.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1467000000000002e-05 2024-09-17T07:29:00.102 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.102 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.102 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.102 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.102 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.103 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:00.104 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:00.105 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:00.103+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:00.147 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T07:29:00.205 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:00.203+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:00.222 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:00.222 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:00.222 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:00.223 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:00.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:00.255 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone7 from 0xe4f7dbb (overlap 0x2ee712c) to 0x1773896 2024-09-17T07:29:00.305 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:00.304+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:00.379 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:00.380 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:00.380 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:00.380 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:00.380 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:00.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:00.406 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:00.406+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:00.426 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-17T07:29:00.507 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:00.506+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:00.538 INFO:teuthology.orchestra.run.smithi045.stdout:712 read 0xdf5a681 thru 0xdf699fc (0xf37c bytes) 2024-09-17T07:29:00.539 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:00.539 INFO:teuthology.orchestra.run.smithi045.stdout:714 writesame 0xc0255ad thru 0xc031026 data_size 0x1(0xba7a bytes) 2024-09-17T07:29:00.542 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:00.542 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:00.542 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:00.542 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:00.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.542 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.542 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:00.542 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.4072199369999998, 2024-09-17T07:29:00.542 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:29:00.542 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.543 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.543 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.543 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.543 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.543 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.543 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:29:00.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.543 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.543 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.543 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:00.543 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.544 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.544 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:00.544 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.544 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.544 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:29:00.544 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:29:00.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.544 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.544 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:29:00.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:29:00.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.545 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:29:00.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:29:00.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.545 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:29:00.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:29:00.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.546 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.546 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:29:00.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:29:00.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.546 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.546 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:29:00.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:29:00.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.546 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.546 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:29:00.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:29:00.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:29:00.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:29:00.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:29:00.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:29:00.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:29:00.548 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:29:00.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.548 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.548 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:29:00.548 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:29:00.548 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.548 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.548 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.548 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.548 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:00.548 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.4071830400000001, 2024-09-17T07:29:00.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:29:00.549 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.549 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.549 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.549 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.549 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.549 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.549 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:29:00.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.549 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.549 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.549 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:00.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.550 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.550 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:00.550 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.550 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.550 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:29:00.550 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:29:00.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.550 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.551 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:29:00.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:29:00.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.551 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.551 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:29:00.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:29:00.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.551 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.551 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:29:00.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:29:00.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:29:00.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:29:00.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:00.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:29:00.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:29:00.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:29:00.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:29:00.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:29:00.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:29:00.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:29:00.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:29:00.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:29:00.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:29:00.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:29:00.554 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.554 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.554 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.554 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.554 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:00.554 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.407157073, 2024-09-17T07:29:00.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:29:00.554 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.554 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.555 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.555 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.555 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.555 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.555 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:29:00.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.555 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.555 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.555 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:00.555 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.555 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.556 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:00.556 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.556 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.556 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:29:00.556 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:00.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:29:00.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:29:00.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:29:00.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:29:00.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:29:00.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:29:00.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.558 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:29:00.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:29:00.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.558 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:29:00.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:29:00.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.558 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:29:00.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:29:00.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:29:00.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:29:00.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:29:00.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:29:00.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:29:00.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:29:00.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.560 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.560 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:29:00.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:29:00.560 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.560 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.560 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.560 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.560 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:00.560 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.4071324779999999, 2024-09-17T07:29:00.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:29:00.561 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.561 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.561 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.561 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.561 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.561 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.561 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:29:00.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.561 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.561 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:00.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.562 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:00.562 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.562 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:00.562 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:29:00.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:29:00.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:29:00.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:29:00.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:29:00.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:29:00.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:29:00.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:29:00.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:29:00.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:29:00.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:29:00.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:29:00.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:29:00.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:29:00.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:29:00.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:29:00.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:29:00.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:29:00.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:29:00.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:29:00.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:29:00.566 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.566 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.566 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.567 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.567 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:00.567 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.4071089909999999, 2024-09-17T07:29:00.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:29:00.567 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.567 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.567 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.567 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.567 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.567 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.567 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:29:00.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.567 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:00.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:00.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:29:00.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:00.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:29:00.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:29:00.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:29:00.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:29:00.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:29:00.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:29:00.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:29:00.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:29:00.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:00.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:29:00.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.571 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:29:00.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:29:00.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.571 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:29:00.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:29:00.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:29:00.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:29:00.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:29:00.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:29:00.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:29:00.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:29:00.572 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.572 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.573 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.573 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.573 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:00.573 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.407095253, 2024-09-17T07:29:00.573 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:29:00.573 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.573 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.573 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.573 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.573 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:29:00.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.574 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.574 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.574 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:00.574 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.574 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.574 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:00.574 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.574 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:29:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:29:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:29:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:29:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:29:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.576 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:29:00.576 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:29:00.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.576 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.576 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:00.576 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:29:00.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.576 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.576 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:00.576 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:29:00.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.577 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:29:00.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:29:00.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.577 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:29:00.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:29:00.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.577 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:29:00.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:29:00.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:29:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:29:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:29:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:29:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.578 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.579 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:00.579 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.4070650640000002, 2024-09-17T07:29:00.579 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:29:00.579 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.579 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.579 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.579 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.579 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.579 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.579 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:29:00.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.579 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.579 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.580 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:00.580 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.580 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.580 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:00.580 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.580 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.580 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:29:00.580 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:29:00.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.581 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.581 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:29:00.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:29:00.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.581 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.581 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:00.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:29:00.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.581 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.581 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:29:00.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:29:00.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.582 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.582 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:29:00.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:29:00.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.582 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.582 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:29:00.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:29:00.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.582 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.582 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:29:00.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:29:00.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:29:00.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:00.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:29:00.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:29:00.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:29:00.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:29:00.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.584 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.584 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:29:00.584 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:29:00.584 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.584 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.584 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.584 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.584 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:00.585 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.407028559, 2024-09-17T07:29:00.585 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:29:00.585 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.585 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.585 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.585 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.585 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.585 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.585 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:29:00.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.585 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.585 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.585 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:00.586 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.586 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.586 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:00.586 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.586 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.586 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:00.586 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:29:00.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:29:00.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:29:00.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:29:00.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:29:00.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:29:00.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:29:00.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.588 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.588 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:29:00.588 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:29:00.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.588 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.588 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:00.588 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:29:00.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.588 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.588 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:29:00.588 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:29:00.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.589 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.589 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:29:00.589 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:29:00.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.589 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.589 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:29:00.589 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:29:00.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.589 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.589 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:29:00.589 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:29:00.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.590 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.590 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:29:00.590 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:29:00.590 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.590 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.590 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.590 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.590 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:00.590 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.4070008000000001, 2024-09-17T07:29:00.590 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:29:00.591 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.591 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.591 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.591 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.591 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.591 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:29:00.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.591 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.591 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.591 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:00.591 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.592 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.592 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:00.592 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.592 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.592 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:29:00.592 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:29:00.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.592 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.592 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:29:00.592 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:29:00.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.593 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.593 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:29:00.593 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:29:00.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.593 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.593 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:29:00.593 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:29:00.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.593 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.594 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:00.594 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:29:00.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.594 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.594 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:29:00.594 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:29:00.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.594 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.594 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:29:00.594 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:29:00.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.595 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.596 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:29:00.596 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:29:00.596 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.596 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.596 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.596 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.596 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.596 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:00.596 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.4069786400000002, 2024-09-17T07:29:00.596 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:29:00.596 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.596 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.596 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.596 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.597 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.597 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.597 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:29:00.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.597 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.597 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.597 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:00.597 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.597 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.597 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:00.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.598 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.598 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:00.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:29:00.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.598 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.598 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:29:00.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:29:00.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.599 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:29:00.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:00.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.599 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:29:00.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:29:00.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.599 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:29:00.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:29:00.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.600 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.600 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:29:00.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:29:00.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.600 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.600 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:29:00.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:29:00.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.600 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.600 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:29:00.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:29:00.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:29:00.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:29:00.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:29:00.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:29:00.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:29:00.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:29:00.602 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.602 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.602 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.602 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.602 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:00.602 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.4069654790000001, 2024-09-17T07:29:00.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:29:00.602 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.603 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.603 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.603 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.603 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.603 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.603 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:29:00.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.603 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.603 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:00.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.604 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.604 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:00.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.604 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.604 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:00.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:29:00.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.604 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.604 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:29:00.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:29:00.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:29:00.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:29:00.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:29:00.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:29:00.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:29:00.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:29:00.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:00.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:29:00.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:29:00.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:29:00.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:29:00.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:29:00.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:29:00.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:29:00.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:29:00.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:29:00.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.608 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:29:00.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:29:00.608 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.608 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.608 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.608 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.608 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:00.609 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.406946891, 2024-09-17T07:29:00.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:29:00.609 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.609 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.609 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.609 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.609 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.609 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:29:00.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.609 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.609 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:00.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:00.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:29:00.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:29:00.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:29:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:29:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:29:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:29:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:00.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:29:00.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:00.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:29:00.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:29:00.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:29:00.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:29:00.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:29:00.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:29:00.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:29:00.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:29:00.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:29:00.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:29:00.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:29:00.614 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.614 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.614 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.614 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.614 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:00.614 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.40691408, 2024-09-17T07:29:00.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:29:00.614 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:29:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:00.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:00.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:29:00.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:29:00.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:29:00.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:29:00.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:00.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:29:00.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:29:00.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:29:00.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:29:00.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:00.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:00.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:29:00.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:29:00.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:29:00.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:29:00.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:29:00.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:29:00.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:29:00.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:29:00.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:29:00.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:29:00.620 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.620 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.620 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.620 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.620 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:00.620 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.4068950230000001, 2024-09-17T07:29:00.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:29:00.620 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.620 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.620 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.620 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.620 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.621 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.621 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:29:00.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.621 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.621 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.621 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:00.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.621 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.621 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:00.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:29:00.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:29:00.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:29:00.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:29:00.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:29:00.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:29:00.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.623 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:29:00.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:29:00.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.623 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:29:00.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:29:00.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:29:00.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:29:00.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:29:00.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:29:00.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:29:00.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:29:00.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:29:00.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:29:00.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:29:00.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:29:00.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:29:00.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:29:00.626 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.626 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.626 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.626 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.626 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:00.626 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.406872914, 2024-09-17T07:29:00.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:29:00.626 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.626 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.626 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.626 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.626 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.627 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.627 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:29:00.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.627 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:00.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:00.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:29:00.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:29:00.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:00.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:29:00.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:29:00.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:29:00.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:29:00.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:29:00.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:29:00.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:29:00.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:29:00.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:29:00.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:29:00.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:29:00.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:29:00.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:29:00.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:29:00.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:29:00.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:29:00.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:29:00.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:29:00.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:29:00.631 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.632 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.632 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.632 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.632 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:00.632 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.406846807, 2024-09-17T07:29:00.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:29:00.632 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.632 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.632 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.632 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.633 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.633 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:29:00.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.633 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:00.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:00.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:29:00.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:29:00.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:00.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:29:00.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:29:00.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:29:00.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.635 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.635 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:29:00.635 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:29:00.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.635 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.635 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:29:00.635 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:29:00.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.635 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.635 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:29:00.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:29:00.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:29:00.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:29:00.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:29:00.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:29:00.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:29:00.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:29:00.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:29:00.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:29:00.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:29:00.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:29:00.637 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.638 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.638 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.638 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.638 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:00.638 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.4068176619999999, 2024-09-17T07:29:00.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:29:00.638 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.638 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.638 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.638 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.638 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.638 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.639 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:29:00.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.639 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:00.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:00.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:00.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:29:00.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:29:00.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:00.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:29:00.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:29:00.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:29:00.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:29:00.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:29:00.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:29:00.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:29:00.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:29:00.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:29:00.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:29:00.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:29:00.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:29:00.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:29:00.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:29:00.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:29:00.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:29:00.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:29:00.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:29:00.644 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.644 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.644 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.644 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.644 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:00.644 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.4067858590000002, 2024-09-17T07:29:00.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:29:00.644 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.644 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.644 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.645 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.645 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.645 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:29:00.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.645 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:00.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:00.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:29:00.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:29:00.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:00.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:29:00.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:29:00.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:29:00.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.647 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:29:00.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:29:00.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.647 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:29:00.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:29:00.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:29:00.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:29:00.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.648 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:29:00.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:29:00.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.648 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:29:00.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:29:00.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.648 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:29:00.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:29:00.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:29:00.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:29:00.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:29:00.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:29:00.649 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.649 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.650 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.650 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.650 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:00.650 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.4067589119999999, 2024-09-17T07:29:00.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:29:00.650 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.650 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.650 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.650 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.650 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.650 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.650 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:29:00.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.651 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:00.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:00.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:29:00.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:29:00.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:29:00.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:29:00.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:00.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:29:00.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:00.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:29:00.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:29:00.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:29:00.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:29:00.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:29:00.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:29:00.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:29:00.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:29:00.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:29:00.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:29:00.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:29:00.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:29:00.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:29:00.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:29:00.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:29:00.655 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.655 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.655 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.656 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:00.656 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:00.656 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.4067179630000002, 2024-09-17T07:29:00.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:29:00.656 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:00.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:00.656 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:00.656 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:00.656 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:00.657 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:00.657 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:29:00.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.657 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:00.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:00.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:00.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:00.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:00.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:00.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:00.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:29:00.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:29:00.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:00.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:29:00.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:29:00.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:00.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:29:00.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:29:00.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:00.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:29:00.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:29:00.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:29:00.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:29:00.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:00.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:29:00.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:29:00.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:29:00.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:29:00.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:00.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:29:00.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:29:00.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:00.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:29:00.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:29:00.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:00.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:29:00.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:29:00.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:00.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:00.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:00.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:29:00.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:29:00.662 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.662 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.662 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.662 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:00.662 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:00.662 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:00.664 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:00.665 INFO:teuthology.orchestra.run.smithi045.stdout:716 write 0xde97fb6 thru 0xde99c30 (0x1c7b bytes) 2024-09-17T07:29:00.665 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:00.665 INFO:teuthology.orchestra.run.smithi045.stdout:718 read 0x285569f thru 0x28568ba (0x121c bytes) 2024-09-17T07:29:00.665 INFO:teuthology.orchestra.run.smithi045.stdout:720 trunc from 0xe4f7dbb to 0xee4f461 2024-09-17T07:29:00.665 INFO:teuthology.orchestra.run.smithi045.stdout:722 punch from 0xafd9747 to 0xafdf8bc, (0x6175 bytes) 2024-09-17T07:29:00.665 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:00.665 INFO:teuthology.orchestra.run.smithi045.stdout:724 trunc from 0xee4f461 to 0x5d1394f 2024-09-17T07:29:00.665 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:00.663+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:00.678 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:00.678 INFO:teuthology.orchestra.run.smithi045.stdout:727 punch from 0x4d3b287 to 0x4d45883, (0xa5fc bytes) 2024-09-17T07:29:00.678 INFO:teuthology.orchestra.run.smithi045.stdout:728 read 0x9b8ee thru 0xa2247 (0x695a bytes) 2024-09-17T07:29:00.686 INFO:teuthology.orchestra.run.smithi045.stdout:729 writesame 0x7f9e6f3 thru 0x7fa0013 data_size 0x1(0x1921 bytes) 2024-09-17T07:29:00.687 INFO:teuthology.orchestra.run.smithi045.stdout:730 write 0x25ac696 thru 0x25b8b9f (0xc50a bytes) 2024-09-17T07:29:00.689 INFO:teuthology.orchestra.run.smithi045.stdout:731 read 0x496b3c2 thru 0x49795a1 (0xe1e0 bytes) 2024-09-17T07:29:00.689 INFO:teuthology.orchestra.run.smithi045.stdout:732 read 0x35c26fe thru 0x35cc61e (0x9f21 bytes) 2024-09-17T07:29:00.690 INFO:teuthology.orchestra.run.smithi045.stdout:733 read 0x7353fa6 thru 0x7363f5c (0xffb7 bytes) 2024-09-17T07:29:00.690 INFO:teuthology.orchestra.run.smithi045.stdout:734 read 0x27d0c45 thru 0x27dd0b8 (0xc474 bytes) 2024-09-17T07:29:00.690 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:00.692 INFO:teuthology.orchestra.run.smithi045.stdout:737 trunc from 0x7fa0014 to 0x6a78630 2024-09-17T07:29:00.711 INFO:teuthology.orchestra.run.smithi045.stdout:738 write 0xc35390c thru 0xc35fcd8 (0xc3cd bytes) 2024-09-17T07:29:00.712 INFO:teuthology.orchestra.run.smithi045.stdout:739 write 0xad16065 thru 0xad2242e (0xc3ca bytes) 2024-09-17T07:29:00.717 INFO:teuthology.orchestra.run.smithi045.stdout:740 writesame 0x4f89f6a thru 0x4f965a1 data_size 0x4(0xc638 bytes) 2024-09-17T07:29:00.729 INFO:teuthology.orchestra.run.smithi045.stdout:742 trunc from 0xc35fcd9 to 0x368098a 2024-09-17T07:29:00.744 INFO:teuthology.orchestra.run.smithi045.stdout:743 write 0x452e53c thru 0x4533766 (0x522b bytes) 2024-09-17T07:29:00.755 INFO:teuthology.orchestra.run.smithi045.stdout:744 write 0x7ab9a50 thru 0x7ac74bf (0xda70 bytes) 2024-09-17T07:29:00.757 INFO:teuthology.orchestra.run.smithi045.stdout:745 punch from 0x282e226 to 0x283c1da, (0xdfb4 bytes) 2024-09-17T07:29:00.757 INFO:teuthology.orchestra.run.smithi045.stdout:746 punch from 0x51d86f7 to 0x51d934c, (0xc55 bytes) 2024-09-17T07:29:00.764 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:00.764+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:00.765 INFO:teuthology.orchestra.run.smithi045.stdout:747 write 0xa435e2e thru 0xa4451f0 (0xf3c3 bytes) 2024-09-17T07:29:00.767 INFO:teuthology.orchestra.run.smithi045.stdout:749 punch from 0x8fc2d26 to 0x8fc59ae, (0x2c88 bytes) 2024-09-17T07:29:00.767 INFO:teuthology.orchestra.run.smithi045.stdout:752 writesame 0x93f5b04 thru 0x9401bb3 data_size 0x8(0xc0b0 bytes) 2024-09-17T07:29:00.774 INFO:teuthology.orchestra.run.smithi045.stdout:753 trunc from 0xa4451f1 to 0xd553c7e 2024-09-17T07:29:00.779 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:00.779 INFO:teuthology.orchestra.run.smithi045.stdout:755 write 0x9f0eb5c thru 0x9f1e7fe (0xfca3 bytes) 2024-09-17T07:29:00.781 INFO:teuthology.orchestra.run.smithi045.stdout:756 write 0xd51a972 thru 0xd51b423 (0xab2 bytes) 2024-09-17T07:29:00.783 INFO:teuthology.orchestra.run.smithi045.stdout:757 read 0xec472a thru 0xec5a57 (0x132e bytes) 2024-09-17T07:29:00.784 INFO:teuthology.orchestra.run.smithi045.stdout:758 writesame 0xd033adf thru 0xd0349f2 data_size 0xa(0xf14 bytes) 2024-09-17T07:29:00.787 INFO:teuthology.orchestra.run.smithi045.stdout:759 punch from 0xbcbc436 to 0xbcbe2cd, (0x1e97 bytes) 2024-09-17T07:29:00.787 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:00.787 INFO:teuthology.orchestra.run.smithi045.stdout:761 read 0xbdbe339 thru 0xbdc5f53 (0x7c1b bytes) 2024-09-17T07:29:00.787 INFO:teuthology.orchestra.run.smithi045.stdout:762 read 0x3be046c thru 0x3be6996 (0x652b bytes) 2024-09-17T07:29:00.788 INFO:teuthology.orchestra.run.smithi045.stdout:763 write 0x1fdd5fa thru 0x1fec7fc (0xf203 bytes) 2024-09-17T07:29:00.789 INFO:teuthology.orchestra.run.smithi045.stdout:764 write 0xa6bbd93 thru 0xa6bf414 (0x3682 bytes) 2024-09-17T07:29:00.790 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:00.790 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:00.790 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:00.790 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:00.799 INFO:teuthology.orchestra.run.smithi045.stdout:766 trunc from 0xd553c7e to 0x7494335 2024-09-17T07:29:00.804 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:00.814 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:00.814 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:00.814 INFO:teuthology.orchestra.run.smithi045.stdout:769 read 0x153aba4 thru 0x15437ce (0x8c2b bytes) 2024-09-17T07:29:00.814 INFO:teuthology.orchestra.run.smithi045.stdout:770 write 0x506af1a thru 0x507319f (0x8286 bytes) 2024-09-17T07:29:00.823 INFO:teuthology.orchestra.run.smithi045.stdout:771 trunc from 0x7494335 to 0xd798581 2024-09-17T07:29:00.829 INFO:teuthology.orchestra.run.smithi045.stdout:772 clone 9 order 25 su 8388608 sc 13 2024-09-17T07:29:00.865 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:00.865+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:00.884 INFO:teuthology.orchestra.run.smithi045.stdout: marking collection for removal 2024-09-17T07:29:00.884 INFO:teuthology.orchestra.run.smithi045.stdout:setting '_remove' omap key 2024-09-17T07:29:00.884 INFO:teuthology.orchestra.run.smithi045.stdout:finish_remove_pgs 3.7_head removing 3.7 2024-09-17T07:29:00.888 INFO:teuthology.orchestra.run.smithi045.stdout:Remove successful 2024-09-17T07:29:00.924 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:00.924 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:00.924 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:00.924 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:00.924 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:00.932 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:00.966 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:00.966+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:01.067 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:01.066+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:01.081 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:01.081 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:01.081 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:01.081 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:01.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.082 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:759 3.4 e28/23 3:20e84768:::rbd_data.1183350c2d82.0000000000000075:head v 28'6, mlcod=28'4)", 2024-09-17T07:29:01.082 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:29:01.082 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 12.155229127, 2024-09-17T07:29:01.082 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011641620000000001, 2024-09-17T07:29:01.082 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.082 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.082 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.082 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.082 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:29:01.082 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.083 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.083 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:29:01.083 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.083 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.083 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:29:01.083 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6100000000000002e-07 2024-09-17T07:29:01.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.083 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.083 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924866+0000", 2024-09-17T07:29:01.084 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:29:01.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.084 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.084 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924875+0000", 2024-09-17T07:29:01.084 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7909999999999993e-06 2024-09-17T07:29:01.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.084 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.084 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924882+0000", 2024-09-17T07:29:01.084 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.418e-06 2024-09-17T07:29:01.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.085 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.085 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924965+0000", 2024-09-17T07:29:01.085 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3029999999999996e-05 2024-09-17T07:29:01.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.085 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.085 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924987+0000", 2024-09-17T07:29:01.085 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1747e-05 2024-09-17T07:29:01.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.085 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.085 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.925980+0000", 2024-09-17T07:29:01.085 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00099299900000000005 2024-09-17T07:29:01.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.086 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.086 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.926025+0000", 2024-09-17T07:29:01.086 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4977e-05 2024-09-17T07:29:01.086 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.086 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.086 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.086 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.086 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1012 3.9 e29/23 3:93118134:::rbd_data.1183350c2d82.0000000000000067:head v 29'55, mlcod=29'53)", 2024-09-17T07:29:01.086 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.270691+0000", 2024-09-17T07:29:01.086 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 11.809399063000001, 2024-09-17T07:29:01.086 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011919299999999999, 2024-09-17T07:29:01.086 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.087 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.087 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.087 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.087 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.087 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270691+0000", 2024-09-17T07:29:01.087 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.087 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.087 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270691+0000", 2024-09-17T07:29:01.087 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.088 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.088 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270692+0000", 2024-09-17T07:29:01.088 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9800000000000002e-07 2024-09-17T07:29:01.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.088 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.088 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270699+0000", 2024-09-17T07:29:01.088 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1679999999999998e-06 2024-09-17T07:29:01.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.089 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.089 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270717+0000", 2024-09-17T07:29:01.089 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8193e-05 2024-09-17T07:29:01.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.089 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.089 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270726+0000", 2024-09-17T07:29:01.089 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4519999999999997e-06 2024-09-17T07:29:01.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.089 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.089 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270779+0000", 2024-09-17T07:29:01.089 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3202000000000003e-05 2024-09-17T07:29:01.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.090 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.090 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270800+0000", 2024-09-17T07:29:01.090 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0846e-05 2024-09-17T07:29:01.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.090 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.090 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271841+0000", 2024-09-17T07:29:01.090 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001040921 2024-09-17T07:29:01.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.090 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.090 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271883+0000", 2024-09-17T07:29:01.090 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2150000000000001e-05 2024-09-17T07:29:01.090 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.091 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.091 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.091 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1746 3.4 e30/23 3:21509c5f:::rbd_data.11832b928878.00000000000001bd:head v 30'14, mlcod=30'11)", 2024-09-17T07:29:01.091 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.791256+0000", 2024-09-17T07:29:01.091 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 10.288834327, 2024-09-17T07:29:01.091 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001583414, 2024-09-17T07:29:01.091 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.091 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.091 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.091 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.091 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.092 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791256+0000", 2024-09-17T07:29:01.092 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.092 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.092 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791256+0000", 2024-09-17T07:29:01.092 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.092 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.092 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791257+0000", 2024-09-17T07:29:01.092 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8599999999999997e-07 2024-09-17T07:29:01.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.093 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.093 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791290+0000", 2024-09-17T07:29:01.093 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3158999999999999e-05 2024-09-17T07:29:01.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.093 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.093 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791298+0000", 2024-09-17T07:29:01.093 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2700000000000004e-06 2024-09-17T07:29:01.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.093 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.093 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791305+0000", 2024-09-17T07:29:01.093 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8159999999999996e-06 2024-09-17T07:29:01.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.094 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.094 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791358+0000", 2024-09-17T07:29:01.094 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3300999999999997e-05 2024-09-17T07:29:01.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.094 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.094 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791377+0000", 2024-09-17T07:29:01.094 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9062000000000001e-05 2024-09-17T07:29:01.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.094 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.094 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.792791+0000", 2024-09-17T07:29:01.094 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014139160000000001 2024-09-17T07:29:01.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.095 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.095 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.792839+0000", 2024-09-17T07:29:01.095 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8103999999999999e-05 2024-09-17T07:29:01.095 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.095 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.095 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.095 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:2183 3.4 e30/23 3:221ed066:::rbd_data.11832b928878.0000000000000073:head v 30'19, mlcod=30'17)", 2024-09-17T07:29:01.095 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:51.139568+0000", 2024-09-17T07:29:01.095 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.9405222440000003, 2024-09-17T07:29:01.095 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00124438, 2024-09-17T07:29:01.096 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.096 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.096 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.096 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.096 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.096 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139568+0000", 2024-09-17T07:29:01.096 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.096 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.096 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139568+0000", 2024-09-17T07:29:01.096 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.097 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.097 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.097 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139569+0000", 2024-09-17T07:29:01.097 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3769999999999999e-06 2024-09-17T07:29:01.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.097 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.097 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.097 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139580+0000", 2024-09-17T07:29:01.097 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0229e-05 2024-09-17T07:29:01.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.097 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.097 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.097 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139601+0000", 2024-09-17T07:29:01.098 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1818999999999999e-05 2024-09-17T07:29:01.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.098 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139611+0000", 2024-09-17T07:29:01.098 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0121e-05 2024-09-17T07:29:01.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.098 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139676+0000", 2024-09-17T07:29:01.098 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4900999999999994e-05 2024-09-17T07:29:01.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.099 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139706+0000", 2024-09-17T07:29:01.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9762000000000001e-05 2024-09-17T07:29:01.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.099 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.099 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.140765+0000", 2024-09-17T07:29:01.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001059019 2024-09-17T07:29:01.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.099 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.099 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.140812+0000", 2024-09-17T07:29:01.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7151999999999998e-05 2024-09-17T07:29:01.100 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.100 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.100 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.100 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.100 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:2190 3.0 e30/23 3:02fd7b4f:::rbd_data.11832b928878.00000000000000b4:head v 30'16, mlcod=30'14)", 2024-09-17T07:29:01.100 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:29:01.100 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.9401774169999992, 2024-09-17T07:29:01.100 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001228395, 2024-09-17T07:29:01.100 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.100 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.100 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:29:01.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:29:01.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:29:01.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0399999999999996e-07 2024-09-17T07:29:01.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139918+0000", 2024-09-17T07:29:01.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8330000000000001e-06 2024-09-17T07:29:01.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139926+0000", 2024-09-17T07:29:01.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1720000000000007e-06 2024-09-17T07:29:01.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139932+0000", 2024-09-17T07:29:01.103 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1789999999999996e-06 2024-09-17T07:29:01.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.103 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.103 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139988+0000", 2024-09-17T07:29:01.103 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5265e-05 2024-09-17T07:29:01.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.103 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.103 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.140004+0000", 2024-09-17T07:29:01.103 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6171000000000002e-05 2024-09-17T07:29:01.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.104 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.141100+0000", 2024-09-17T07:29:01.104 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010964519999999999 2024-09-17T07:29:01.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.104 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.141141+0000", 2024-09-17T07:29:01.104 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0818999999999997e-05 2024-09-17T07:29:01.104 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.104 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.104 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.105 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.105 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:4848 3.9 e33/23 3:97363921:::rbd_data.1183cff49e3a.0000000000000004:head v 33'102, mlcod=32'100)", 2024-09-17T07:29:01.105 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:01.105 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 6.8405943499999999, 2024-09-17T07:29:01.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023715110000000002, 2024-09-17T07:29:01.105 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.105 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.105 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.105 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.105 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.105 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.105 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:01.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.106 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.106 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.106 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:01.106 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.106 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.106 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.106 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239497+0000", 2024-09-17T07:29:01.106 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.53e-06 2024-09-17T07:29:01.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.106 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.106 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.106 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240142+0000", 2024-09-17T07:29:01.107 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00064481300000000005 2024-09-17T07:29:01.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.107 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.107 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240160+0000", 2024-09-17T07:29:01.107 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8349999999999999e-05 2024-09-17T07:29:01.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.107 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.107 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240169+0000", 2024-09-17T07:29:01.107 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7690000000000002e-06 2024-09-17T07:29:01.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.107 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240246+0000", 2024-09-17T07:29:01.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6277000000000002e-05 2024-09-17T07:29:01.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.108 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240270+0000", 2024-09-17T07:29:01.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4575000000000001e-05 2024-09-17T07:29:01.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.108 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.241813+0000", 2024-09-17T07:29:01.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015432860000000001 2024-09-17T07:29:01.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.109 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.109 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.241867+0000", 2024-09-17T07:29:01.109 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3910999999999999e-05 2024-09-17T07:29:01.109 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.109 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.109 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.109 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:4849 3.9 e33/23 3:97363921:::rbd_data.1183cff49e3a.0000000000000004:head v 33'104, mlcod=32'100)", 2024-09-17T07:29:01.109 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.240572+0000", 2024-09-17T07:29:01.109 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 6.8395182329999997, 2024-09-17T07:29:01.109 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015942700000000001, 2024-09-17T07:29:01.110 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.110 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.110 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.110 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.110 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.110 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.110 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240572+0000", 2024-09-17T07:29:01.110 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.110 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.110 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.110 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240572+0000", 2024-09-17T07:29:01.110 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240572+0000", 2024-09-17T07:29:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3799999999999997e-07 2024-09-17T07:29:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240577+0000", 2024-09-17T07:29:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3000000000000003e-06 2024-09-17T07:29:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240584+0000", 2024-09-17T07:29:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2300000000000002e-06 2024-09-17T07:29:01.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240588+0000", 2024-09-17T07:29:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7340000000000002e-06 2024-09-17T07:29:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240936+0000", 2024-09-17T07:29:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00034820700000000001 2024-09-17T07:29:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.112 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240955+0000", 2024-09-17T07:29:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8808e-05 2024-09-17T07:29:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.242116+0000", 2024-09-17T07:29:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011614590000000001 2024-09-17T07:29:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.242166+0000", 2024-09-17T07:29:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9994000000000003e-05 2024-09-17T07:29:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.113 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:5460 3.9 e33/23 3:90773955:::rbd_data.1183cff49e3a.000000000000000a:head v 33'106, mlcod=33'104)", 2024-09-17T07:29:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 6.5534977640000003, 2024-09-17T07:29:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020825850000000001, 2024-09-17T07:29:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:01.114 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526593+0000", 2024-09-17T07:29:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.54e-07 2024-09-17T07:29:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.115 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527050+0000", 2024-09-17T07:29:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000456522 2024-09-17T07:29:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527061+0000", 2024-09-17T07:29:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1722000000000001e-05 2024-09-17T07:29:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527067+0000", 2024-09-17T07:29:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0599999999999998e-06 2024-09-17T07:29:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.116 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527112+0000", 2024-09-17T07:29:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999000000000001e-05 2024-09-17T07:29:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527128+0000", 2024-09-17T07:29:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6920999999999999e-05 2024-09-17T07:29:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528636+0000", 2024-09-17T07:29:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001507836 2024-09-17T07:29:01.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528675+0000", 2024-09-17T07:29:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8671000000000003e-05 2024-09-17T07:29:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:5461 3.9 e33/23 3:90773955:::rbd_data.1183cff49e3a.000000000000000a:head v 33'108, mlcod=33'104)", 2024-09-17T07:29:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.527712+0000", 2024-09-17T07:29:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 6.5523786179999997, 2024-09-17T07:29:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012502800000000001, 2024-09-17T07:29:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527712+0000", 2024-09-17T07:29:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527712+0000", 2024-09-17T07:29:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.119 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527712+0000", 2024-09-17T07:29:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.89e-07 2024-09-17T07:29:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527716+0000", 2024-09-17T07:29:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1520000000000002e-06 2024-09-17T07:29:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527724+0000", 2024-09-17T07:29:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9689999999999994e-06 2024-09-17T07:29:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527728+0000", 2024-09-17T07:29:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8909999999999997e-06 2024-09-17T07:29:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527776+0000", 2024-09-17T07:29:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7778999999999998e-05 2024-09-17T07:29:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527798+0000", 2024-09-17T07:29:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2558999999999999e-05 2024-09-17T07:29:01.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528931+0000", 2024-09-17T07:29:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001132814 2024-09-17T07:29:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528962+0000", 2024-09-17T07:29:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0626999999999997e-05 2024-09-17T07:29:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:6005 3.0 e33/23 3:037cd38f:::rbd_data.1183cff49e3a.0000000000000012:head v 33'25, mlcod=32'23)", 2024-09-17T07:29:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 6.4554646299999998, 2024-09-17T07:29:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019123899999999999, 2024-09-17T07:29:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.126 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624627+0000", 2024-09-17T07:29:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5600000000000001e-06 2024-09-17T07:29:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624777+0000", 2024-09-17T07:29:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015030800000000001 2024-09-17T07:29:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624792+0000", 2024-09-17T07:29:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5089000000000001e-05 2024-09-17T07:29:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624799+0000", 2024-09-17T07:29:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7939999999999997e-06 2024-09-17T07:29:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624855+0000", 2024-09-17T07:29:01.128 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5782999999999998e-05 2024-09-17T07:29:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624871+0000", 2024-09-17T07:29:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6192e-05 2024-09-17T07:29:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.626507+0000", 2024-09-17T07:29:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016354500000000001 2024-09-17T07:29:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.626538+0000", 2024-09-17T07:29:01.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1214000000000003e-05 2024-09-17T07:29:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:6683 3.9 e33/23 3:97ddc208:::rbd_data.1183cff49e3a.000000000000001e:head v 33'110, mlcod=33'108)", 2024-09-17T07:29:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.764442+0000", 2024-09-17T07:29:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 6.3156482279999997, 2024-09-17T07:29:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017329470000000001, 2024-09-17T07:29:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.131 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.131 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.764442+0000", 2024-09-17T07:29:01.131 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.131 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.131 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.764442+0000", 2024-09-17T07:29:01.131 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.131 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.131 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.764443+0000", 2024-09-17T07:29:01.131 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:29:01.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.132 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.132 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765126+0000", 2024-09-17T07:29:01.132 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00068291100000000002 2024-09-17T07:29:01.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.132 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.132 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.132 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765146+0000", 2024-09-17T07:29:01.132 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9406999999999999e-05 2024-09-17T07:29:01.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.132 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.132 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.132 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765156+0000", 2024-09-17T07:29:01.132 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0037000000000001e-05 2024-09-17T07:29:01.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.133 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.133 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765209+0000", 2024-09-17T07:29:01.133 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3732e-05 2024-09-17T07:29:01.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.133 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.133 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765226+0000", 2024-09-17T07:29:01.133 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6787e-05 2024-09-17T07:29:01.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.133 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.133 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.766139+0000", 2024-09-17T07:29:01.134 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00091277899999999996 2024-09-17T07:29:01.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.134 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.134 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.766175+0000", 2024-09-17T07:29:01.134 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5942999999999998e-05 2024-09-17T07:29:01.134 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.134 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.134 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.134 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:7670 3.4 e33/23 3:20e02957:::rbd_data.1183cff49e3a.000000000000002d:head v 33'30, mlcod=32'28)", 2024-09-17T07:29:01.134 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.955123+0000", 2024-09-17T07:29:01.134 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 6.1249667670000001, 2024-09-17T07:29:01.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0018080839999999999, 2024-09-17T07:29:01.135 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.135 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.135 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.135 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.135 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.135 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.135 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955123+0000", 2024-09-17T07:29:01.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.135 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.135 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.135 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955123+0000", 2024-09-17T07:29:01.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.136 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.136 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.136 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955125+0000", 2024-09-17T07:29:01.136 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3060000000000001e-06 2024-09-17T07:29:01.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.136 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.136 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.136 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955285+0000", 2024-09-17T07:29:01.136 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00016064700000000001 2024-09-17T07:29:01.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.136 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.136 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.136 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955303+0000", 2024-09-17T07:29:01.136 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7465999999999998e-05 2024-09-17T07:29:01.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.137 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955311+0000", 2024-09-17T07:29:01.137 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4780000000000008e-06 2024-09-17T07:29:01.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.137 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955373+0000", 2024-09-17T07:29:01.137 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1977000000000006e-05 2024-09-17T07:29:01.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.137 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955388+0000", 2024-09-17T07:29:01.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4260000000000001e-05 2024-09-17T07:29:01.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.138 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.138 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.956893+0000", 2024-09-17T07:29:01.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001505309 2024-09-17T07:29:01.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.138 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.138 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.956931+0000", 2024-09-17T07:29:01.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8640999999999998e-05 2024-09-17T07:29:01.138 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.138 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.138 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.139 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8595 3.4 e33/23 3:237b2a73:::rbd_data.1183cff49e3a.0000000000000049:head v 33'37, mlcod=33'35)", 2024-09-17T07:29:01.139 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:01.139 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 6.0054476709999998, 2024-09-17T07:29:01.139 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020228939999999999, 2024-09-17T07:29:01.139 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.139 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.139 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.139 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.139 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.139 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:01.139 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.140 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.140 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:01.140 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.140 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.140 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074643+0000", 2024-09-17T07:29:01.140 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3099999999999997e-07 2024-09-17T07:29:01.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.140 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.140 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074834+0000", 2024-09-17T07:29:01.141 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000191137 2024-09-17T07:29:01.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.141 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.141 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074850+0000", 2024-09-17T07:29:01.141 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6101999999999999e-05 2024-09-17T07:29:01.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.141 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.141 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074859+0000", 2024-09-17T07:29:01.141 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2169999999999994e-06 2024-09-17T07:29:01.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.142 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.142 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074919+0000", 2024-09-17T07:29:01.142 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.003e-05 2024-09-17T07:29:01.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.142 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.142 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.142 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074944+0000", 2024-09-17T07:29:01.142 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5761e-05 2024-09-17T07:29:01.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.142 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.142 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.142 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076622+0000", 2024-09-17T07:29:01.142 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001677977 2024-09-17T07:29:01.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.143 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.143 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076665+0000", 2024-09-17T07:29:01.143 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2938999999999997e-05 2024-09-17T07:29:01.143 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.143 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.143 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.143 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8620 3.9 e33/23 3:95565201:::rbd_data.1183cff49e3a.0000000000000057:head v 33'116, mlcod=33'114)", 2024-09-17T07:29:01.143 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.075453+0000", 2024-09-17T07:29:01.143 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 6.0046370810000003, 2024-09-17T07:29:01.143 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011371619999999999, 2024-09-17T07:29:01.144 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.144 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.144 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.144 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.144 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.144 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075453+0000", 2024-09-17T07:29:01.144 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.144 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.144 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.144 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075453+0000", 2024-09-17T07:29:01.144 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.145 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.145 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.145 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075454+0000", 2024-09-17T07:29:01.145 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6799999999999999e-07 2024-09-17T07:29:01.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.145 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.145 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.145 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075461+0000", 2024-09-17T07:29:01.145 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8469999999999998e-06 2024-09-17T07:29:01.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.145 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.145 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.145 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075479+0000", 2024-09-17T07:29:01.145 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8014999999999999e-05 2024-09-17T07:29:01.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.146 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.146 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075486+0000", 2024-09-17T07:29:01.146 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8700000000000003e-06 2024-09-17T07:29:01.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.146 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.146 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075527+0000", 2024-09-17T07:29:01.146 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1556e-05 2024-09-17T07:29:01.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.146 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.146 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075541+0000", 2024-09-17T07:29:01.146 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3752e-05 2024-09-17T07:29:01.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.147 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.147 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.147 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076527+0000", 2024-09-17T07:29:01.147 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098600500000000008 2024-09-17T07:29:01.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.147 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.147 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.147 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076590+0000", 2024-09-17T07:29:01.147 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3349000000000006e-05 2024-09-17T07:29:01.147 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.147 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.147 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.147 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.148 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8596 3.4 e33/23 3:237b2a73:::rbd_data.1183cff49e3a.0000000000000049:head v 33'39, mlcod=33'35)", 2024-09-17T07:29:01.148 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.075842+0000", 2024-09-17T07:29:01.148 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 6.0042476819999999, 2024-09-17T07:29:01.148 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012443630000000001, 2024-09-17T07:29:01.148 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.148 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.148 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.148 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.148 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.148 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.148 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075842+0000", 2024-09-17T07:29:01.148 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.148 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.149 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.149 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075842+0000", 2024-09-17T07:29:01.149 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.149 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.149 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075843+0000", 2024-09-17T07:29:01.149 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1099999999999995e-07 2024-09-17T07:29:01.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.149 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.149 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075848+0000", 2024-09-17T07:29:01.149 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1139999999999997e-06 2024-09-17T07:29:01.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.150 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.150 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075858+0000", 2024-09-17T07:29:01.150 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5920000000000007e-06 2024-09-17T07:29:01.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.150 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.150 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075862+0000", 2024-09-17T07:29:01.150 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5580000000000002e-06 2024-09-17T07:29:01.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.150 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.150 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075917+0000", 2024-09-17T07:29:01.150 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4274000000000002e-05 2024-09-17T07:29:01.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.151 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.151 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075938+0000", 2024-09-17T07:29:01.151 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1693000000000001e-05 2024-09-17T07:29:01.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.151 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.151 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.077049+0000", 2024-09-17T07:29:01.151 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011106079999999999 2024-09-17T07:29:01.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.151 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.151 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.077087+0000", 2024-09-17T07:29:01.151 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7913000000000001e-05 2024-09-17T07:29:01.151 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.152 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.152 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.152 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8721 3.4 e33/23 3:207ac2b1:::rbd_data.1183cff49e3a.000000000000004a:head v 33'40, mlcod=33'37)", 2024-09-17T07:29:01.152 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.079669+0000", 2024-09-17T07:29:01.152 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 6.0004215900000002, 2024-09-17T07:29:01.152 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017112749999999999, 2024-09-17T07:29:01.152 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.152 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.152 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.152 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.152 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.153 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079669+0000", 2024-09-17T07:29:01.153 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.153 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.153 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079669+0000", 2024-09-17T07:29:01.153 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.153 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.153 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079670+0000", 2024-09-17T07:29:01.153 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1540000000000001e-06 2024-09-17T07:29:01.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.154 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.154 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079849+0000", 2024-09-17T07:29:01.154 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017954900000000001 2024-09-17T07:29:01.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.154 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.154 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079864+0000", 2024-09-17T07:29:01.154 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4973e-05 2024-09-17T07:29:01.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.154 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.154 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079872+0000", 2024-09-17T07:29:01.154 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5170000000000002e-06 2024-09-17T07:29:01.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.155 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.155 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079929+0000", 2024-09-17T07:29:01.155 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7083000000000002e-05 2024-09-17T07:29:01.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.155 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.155 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079959+0000", 2024-09-17T07:29:01.155 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0474e-05 2024-09-17T07:29:01.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.155 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.155 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.081348+0000", 2024-09-17T07:29:01.155 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001388481 2024-09-17T07:29:01.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.156 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.156 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.081380+0000", 2024-09-17T07:29:01.156 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2044e-05 2024-09-17T07:29:01.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.156 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.156 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8872 3.0 e34/23 3:042b6c8e:::rbd_data.1183cff49e3a.0000000000000058:head v 34'31, mlcod=33'29)", 2024-09-17T07:29:01.156 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.322367+0000", 2024-09-17T07:29:01.156 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.7577227149999999, 2024-09-17T07:29:01.156 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016244619999999999, 2024-09-17T07:29:01.156 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.157 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.157 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.157 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.157 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322367+0000", 2024-09-17T07:29:01.157 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.157 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.157 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322367+0000", 2024-09-17T07:29:01.157 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322368+0000", 2024-09-17T07:29:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.16e-07 2024-09-17T07:29:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322539+0000", 2024-09-17T07:29:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017174200000000001 2024-09-17T07:29:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322554+0000", 2024-09-17T07:29:01.158 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4637000000000001e-05 2024-09-17T07:29:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322563+0000", 2024-09-17T07:29:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2369999999999998e-06 2024-09-17T07:29:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322622+0000", 2024-09-17T07:29:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8928000000000002e-05 2024-09-17T07:29:01.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322641+0000", 2024-09-17T07:29:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8601e-05 2024-09-17T07:29:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.323955+0000", 2024-09-17T07:29:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013146410000000001 2024-09-17T07:29:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.160 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.323992+0000", 2024-09-17T07:29:01.161 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.646e-05 2024-09-17T07:29:01.161 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.161 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.161 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.161 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8873 3.0 e34/23 3:042b6c8e:::rbd_data.1183cff49e3a.0000000000000058:head v 34'33, mlcod=33'29)", 2024-09-17T07:29:01.161 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.323189+0000", 2024-09-17T07:29:01.161 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.7569015170000002, 2024-09-17T07:29:01.161 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001142459, 2024-09-17T07:29:01.161 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.161 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.161 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.162 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.162 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.323189+0000", 2024-09-17T07:29:01.162 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.162 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.162 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.323189+0000", 2024-09-17T07:29:01.162 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.162 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.162 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.323189+0000", 2024-09-17T07:29:01.163 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9999999999999997e-07 2024-09-17T07:29:01.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.163 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.163 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.323194+0000", 2024-09-17T07:29:01.163 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0390000000000004e-06 2024-09-17T07:29:01.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.163 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.163 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.323206+0000", 2024-09-17T07:29:01.163 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1450999999999999e-05 2024-09-17T07:29:01.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.163 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.163 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.323212+0000", 2024-09-17T07:29:01.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:29:01.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.323290+0000", 2024-09-17T07:29:01.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8410000000000003e-05 2024-09-17T07:29:01.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.323308+0000", 2024-09-17T07:29:01.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7339000000000001e-05 2024-09-17T07:29:01.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.165 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.324300+0000", 2024-09-17T07:29:01.165 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000992197 2024-09-17T07:29:01.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.165 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.165 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.324331+0000", 2024-09-17T07:29:01.165 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1316000000000003e-05 2024-09-17T07:29:01.165 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.165 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.165 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.165 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:15673 3.0 e39/23 3:0518bbb2:::rbd_object_map.1183515b0721:head v 39'84, mlcod=39'81)", 2024-09-17T07:29:01.165 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.622698+0000", 2024-09-17T07:29:01.166 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.45739200400000002, 2024-09-17T07:29:01.166 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001189617, 2024-09-17T07:29:01.166 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.166 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.166 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.166 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.166 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622698+0000", 2024-09-17T07:29:01.166 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.166 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.166 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622698+0000", 2024-09-17T07:29:01.167 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.167 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.167 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622699+0000", 2024-09-17T07:29:01.167 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7400000000000002e-07 2024-09-17T07:29:01.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.167 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.167 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622712+0000", 2024-09-17T07:29:01.167 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3105000000000001e-05 2024-09-17T07:29:01.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.167 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.168 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622726+0000", 2024-09-17T07:29:01.168 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3995000000000001e-05 2024-09-17T07:29:01.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.168 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.168 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622734+0000", 2024-09-17T07:29:01.168 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6009999999999997e-06 2024-09-17T07:29:01.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.168 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.168 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622784+0000", 2024-09-17T07:29:01.168 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.058e-05 2024-09-17T07:29:01.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.169 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.169 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622805+0000", 2024-09-17T07:29:01.169 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1012000000000001e-05 2024-09-17T07:29:01.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.169 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.169 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.623823+0000", 2024-09-17T07:29:01.169 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010180670000000001 2024-09-17T07:29:01.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.169 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.169 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.623888+0000", 2024-09-17T07:29:01.169 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4382999999999997e-05 2024-09-17T07:29:01.170 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.170 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.170 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.170 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:15683 3.0 e39/23 3:0518bbb2:::rbd_object_map.1183515b0721:head v 39'90, mlcod=39'84)", 2024-09-17T07:29:01.170 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.635895+0000", 2024-09-17T07:29:01.170 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.44419482799999999, 2024-09-17T07:29:01.170 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001403357, 2024-09-17T07:29:01.170 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:01.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:01.170 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:01.170 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:01.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.171 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:01.171 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635895+0000", 2024-09-17T07:29:01.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.171 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:01.171 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635895+0000", 2024-09-17T07:29:01.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:01.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.171 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:01.171 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635896+0000", 2024-09-17T07:29:01.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9999999999999995e-07 2024-09-17T07:29:01.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.172 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:01.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635905+0000", 2024-09-17T07:29:01.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6510000000000007e-06 2024-09-17T07:29:01.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.172 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:01.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635918+0000", 2024-09-17T07:29:01.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3168e-05 2024-09-17T07:29:01.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.172 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:01.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635931+0000", 2024-09-17T07:29:01.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2723e-05 2024-09-17T07:29:01.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.173 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:01.173 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.636183+0000", 2024-09-17T07:29:01.173 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00025214999999999998 2024-09-17T07:29:01.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.173 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:01.173 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.636210+0000", 2024-09-17T07:29:01.173 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6667000000000001e-05 2024-09-17T07:29:01.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.173 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:01.173 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.637240+0000", 2024-09-17T07:29:01.174 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010305889999999999 2024-09-17T07:29:01.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:01.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:01.174 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:01.174 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.637299+0000", 2024-09-17T07:29:01.174 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8409000000000003e-05 2024-09-17T07:29:01.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.174 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:01.174 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:01.174 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:01.176 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:01.177 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-17T07:29:01.177 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:01.175+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:01.265 INFO:teuthology.orchestra.run.smithi045.stdout:flattening image image_client.0-clone8 2024-09-17T07:29:01.276 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:01.275+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:01.284 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:01.284 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:01.293 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:01.377 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:01.377+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:01.397 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.7.3 2024-09-17T07:29:01.443 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:01.443 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:01.453 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:01.461 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-17T07:29:01.478 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:01.477+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:01.509 INFO:teuthology.orchestra.run.smithi045.stdout:773 write 0x1952086 thru 0x195b686 (0x9601 bytes) 2024-09-17T07:29:01.541 INFO:teuthology.orchestra.run.smithi045.stdout:774 punch from 0x13dadd0 to 0x13dd890, (0x2ac0 bytes) 2024-09-17T07:29:01.541 INFO:teuthology.orchestra.run.smithi045.stdout:775 punch from 0x2bdac9d to 0x2bdd7a5, (0x2b08 bytes) 2024-09-17T07:29:01.541 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:01.541 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:01.541 INFO:teuthology.orchestra.run.smithi045.stdout:779 write 0x19c67d6 thru 0x19cac03 (0x442e bytes) 2024-09-17T07:29:01.543 INFO:teuthology.orchestra.run.smithi045.stdout:780 trunc from 0xd798581 to 0xed9be8d 2024-09-17T07:29:01.549 INFO:teuthology.orchestra.run.smithi045.stdout:781 writesame 0x3f0a523 thru 0x3f0dfea data_size 0x26(0x3ac8 bytes) 2024-09-17T07:29:01.553 INFO:teuthology.orchestra.run.smithi045.stdout:782 writesame 0x1627a7 thru 0x17115f data_size 0x1(0xe9b9 bytes) 2024-09-17T07:29:01.579 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:01.579+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:01.611 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:01.612 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:01.613 INFO:teuthology.orchestra.run.smithi045.stdout:783 read 0xe72b2f1 thru 0xe733b1a (0x882a bytes) 2024-09-17T07:29:01.613 INFO:teuthology.orchestra.run.smithi045.stdout:784 writesame 0x46e4081 thru 0x46f1bc2 data_size 0xf(0xdb42 bytes) 2024-09-17T07:29:01.618 INFO:teuthology.orchestra.run.smithi045.stdout:788 write 0xb0da4c thru 0xb11da0 (0x4355 bytes) 2024-09-17T07:29:01.623 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:01.645 INFO:teuthology.orchestra.run.smithi045.stdout:789 trunc from 0xed9be8d to 0x9b5fc89 2024-09-17T07:29:01.680 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:01.678+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:01.685 INFO:teuthology.orchestra.run.smithi045.stdout:790 read 0x545e930 thru 0x5469aa6 (0xb177 bytes) 2024-09-17T07:29:01.695 INFO:teuthology.orchestra.run.smithi045.stdout:791 write 0xbd3835b thru 0xbd399cd (0x1673 bytes) 2024-09-17T07:29:01.696 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:01.696 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:01.697 INFO:teuthology.orchestra.run.smithi045.stdout:795 write 0x52940b1 thru 0x529d86a (0x97ba bytes) 2024-09-17T07:29:01.697 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:01.699 INFO:teuthology.orchestra.run.smithi045.stdout:797 trunc from 0xbd399ce to 0xa1a2c57 2024-09-17T07:29:01.706 INFO:teuthology.orchestra.run.smithi045.stdout:798 read 0xf9cf51 thru 0xfab966 (0xea16 bytes) 2024-09-17T07:29:01.706 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:01.717 INFO:teuthology.orchestra.run.smithi045.stdout:801 write 0xea85e14 thru 0xea888fe (0x2aeb bytes) 2024-09-17T07:29:01.719 INFO:teuthology.orchestra.run.smithi045.stdout:805 punch from 0x26be606 to 0x26ca768, (0xc162 bytes) 2024-09-17T07:29:01.719 INFO:teuthology.orchestra.run.smithi045.stdout:807 punch from 0x51ac0ad to 0x51ad49c, (0x13ef bytes) 2024-09-17T07:29:01.719 INFO:teuthology.orchestra.run.smithi045.stdout:808 read 0x764c5f1 thru 0x765be77 (0xf887 bytes) 2024-09-17T07:29:01.720 INFO:teuthology.orchestra.run.smithi045.stdout:810 punch from 0x534bf0d to 0x534ca63, (0xb56 bytes) 2024-09-17T07:29:01.724 INFO:teuthology.orchestra.run.smithi045.stdout:811 trunc from 0xea888ff to 0xa8d7aab 2024-09-17T07:29:01.732 INFO:teuthology.orchestra.run.smithi045.stdout:816 trunc from 0xa8d7aab to 0xb1f0b50 2024-09-17T07:29:01.739 INFO:teuthology.orchestra.run.smithi045.stdout:817 read 0x4824f08 thru 0x482db1f (0x8c18 bytes) 2024-09-17T07:29:01.739 INFO:teuthology.orchestra.run.smithi045.stdout:819 read 0xa67ec85 thru 0xa68b678 (0xc9f4 bytes) 2024-09-17T07:29:01.739 INFO:teuthology.orchestra.run.smithi045.stdout:820 writesame 0x5df8c41 thru 0x5e04db8 data_size 0x52(0xc178 bytes) 2024-09-17T07:29:01.741 INFO:teuthology.orchestra.run.smithi045.stdout:821 read 0x5aa5033 thru 0x5aa758a (0x2558 bytes) 2024-09-17T07:29:01.742 INFO:teuthology.orchestra.run.smithi045.stdout:822 flatten 2024-09-17T07:29:01.773 INFO:teuthology.orchestra.run.smithi045.stdout:823 write 0x394998d thru 0x394e86e (0x4ee2 bytes) 2024-09-17T07:29:01.777 INFO:teuthology.orchestra.run.smithi045.stdout:824 trunc from 0xb1f0b50 to 0xe0e3d5c 2024-09-17T07:29:01.780 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:01.780 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:01.780 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:01.779+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:01.783 INFO:teuthology.orchestra.run.smithi045.stdout:828 write 0x1233c8 thru 0x128ede (0x5b17 bytes) 2024-09-17T07:29:01.786 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:01.786 INFO:teuthology.orchestra.run.smithi045.stdout:830 write 0x2666fc8 thru 0x266d8cf (0x6908 bytes) 2024-09-17T07:29:01.788 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:01.788 INFO:teuthology.orchestra.run.smithi045.stdout:832 read 0x77822c4 thru 0x778580e (0x354b bytes) 2024-09-17T07:29:01.790 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:01.792 INFO:teuthology.orchestra.run.smithi045.stdout:833 trunc from 0xe0e3d5c to 0xb312887 2024-09-17T07:29:01.799 INFO:teuthology.orchestra.run.smithi045.stdout:834 write 0x180b0b9 thru 0x181a44c (0xf394 bytes) 2024-09-17T07:29:01.800 INFO:teuthology.orchestra.run.smithi045.stdout:836 writesame 0x9326d0 thru 0x93d3f6 data_size 0x1(0xad27 bytes) 2024-09-17T07:29:01.825 INFO:teuthology.orchestra.run.smithi045.stdout:839 write 0xdaf8268 thru 0xdb04923 (0xc6bc bytes) 2024-09-17T07:29:01.825 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:01.826 INFO:teuthology.orchestra.run.smithi045.stdout:841 read 0x1b2cc75 thru 0x1b3a66b (0xd9f7 bytes) 2024-09-17T07:29:01.826 INFO:teuthology.orchestra.run.smithi045.stdout:842 writesame 0x1538cd0 thru 0x154881f data_size 0x8(0xfb50 bytes) 2024-09-17T07:29:01.839 INFO:teuthology.orchestra.run.smithi045.stdout:844 trunc from 0xdb04924 to 0x5a19061 2024-09-17T07:29:01.854 INFO:teuthology.orchestra.run.smithi045.stdout:845 read 0x3ec0d0a thru 0x3ec8c9b (0x7f92 bytes) 2024-09-17T07:29:01.854 INFO:teuthology.orchestra.run.smithi045.stdout:846 punch from 0x53e7a0d to 0x53f03dd, (0x89d0 bytes) 2024-09-17T07:29:01.854 INFO:teuthology.orchestra.run.smithi045.stdout:847 read 0x3741b2d thru 0x374641e (0x48f2 bytes) 2024-09-17T07:29:01.857 INFO:teuthology.orchestra.run.smithi045.stdout:848 trunc from 0x5a19061 to 0x237e7f0 2024-09-17T07:29:01.881 INFO:teuthology.orchestra.run.smithi045.stdout:849 punch from 0x1f8221c to 0x1f85fd9, (0x3dbd bytes) 2024-09-17T07:29:01.881 INFO:teuthology.orchestra.run.smithi045.stdout:850 read 0x1cef8ba thru 0x1cf5e86 (0x65cd bytes) 2024-09-17T07:29:01.882 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:01.881+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:01.888 INFO:teuthology.orchestra.run.smithi045.stdout:851 write 0x332f9f4 thru 0x333e905 (0xef12 bytes) 2024-09-17T07:29:01.891 INFO:teuthology.orchestra.run.smithi045.stdout:854 read 0x27e491a thru 0x27f2cef (0xe3d6 bytes) 2024-09-17T07:29:01.891 INFO:teuthology.orchestra.run.smithi045.stdout:856 read 0xe9c3c0 thru 0xe9e205 (0x1e46 bytes) 2024-09-17T07:29:01.891 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:01.892 INFO:teuthology.orchestra.run.smithi045.stdout:859 punch from 0x14c325c to 0x14cf2d8, (0xc07c bytes) 2024-09-17T07:29:01.905 INFO:teuthology.orchestra.run.smithi045.stdout:861 writesame 0xa78f723 thru 0xa79405d data_size 0x3(0x493b bytes) 2024-09-17T07:29:01.905 INFO:teuthology.orchestra.run.smithi045.stdout:862 write 0x4916613 thru 0x4926092 (0xfa80 bytes) 2024-09-17T07:29:01.907 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:01.907 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:01.908 INFO:teuthology.orchestra.run.smithi045.stdout:864 read 0x44d41cf thru 0x44d440a (0x23c bytes) 2024-09-17T07:29:01.908 INFO:teuthology.orchestra.run.smithi045.stdout:865 write 0x4194a79 thru 0x419b776 (0x6cfe bytes) 2024-09-17T07:29:01.911 INFO:teuthology.orchestra.run.smithi045.stdout:867 write 0x53f0c9f thru 0x53f289e (0x1c00 bytes) 2024-09-17T07:29:01.917 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:01.920 INFO:teuthology.orchestra.run.smithi045.stdout:868 writesame 0xb3fd8c0 thru 0xb3fd9e4 data_size 0x1(0x125 bytes) 2024-09-17T07:29:01.921 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:01.921 INFO:teuthology.orchestra.run.smithi045.stdout:871 writesame 0x25986e2 thru 0x25a72aa data_size 0x1(0xebc9 bytes) 2024-09-17T07:29:01.948 INFO:teuthology.orchestra.run.smithi045.stdout:872 read 0x814fedc thru 0x8159ea6 (0x9fcb bytes) 2024-09-17T07:29:01.948 INFO:teuthology.orchestra.run.smithi045.stdout:873 punch from 0x814465d to 0x814e3bf, (0x9d62 bytes) 2024-09-17T07:29:01.948 INFO:teuthology.orchestra.run.smithi045.stdout:874 punch from 0x4e78f30 to 0x4e7e8fc, (0x59cc bytes) 2024-09-17T07:29:01.948 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:01.948 INFO:teuthology.orchestra.run.smithi045.stdout:877 writesame 0x41d0cab thru 0x41e0b38 data_size 0x3(0xfe8e bytes) 2024-09-17T07:29:01.958 INFO:teuthology.orchestra.run.smithi045.stdout:878 punch from 0xa75ce1b to 0xa766b83, (0x9d68 bytes) 2024-09-17T07:29:01.959 INFO:teuthology.orchestra.run.smithi045.stdout:881 punch from 0xde9f84 to 0xdf8745, (0xe7c1 bytes) 2024-09-17T07:29:01.959 INFO:teuthology.orchestra.run.smithi045.stdout:882 read 0x9762850 thru 0x976c85f (0xa010 bytes) 2024-09-17T07:29:01.959 INFO:teuthology.orchestra.run.smithi045.stdout:883 write 0x23703f9 thru 0x23747ca (0x43d2 bytes) 2024-09-17T07:29:01.959 INFO:teuthology.orchestra.run.smithi045.stdout:884 writesame 0x4bd4e1b thru 0x4be1d1a data_size 0x48(0xcf00 bytes) 2024-09-17T07:29:01.961 INFO:teuthology.orchestra.run.smithi045.stdout:885 write 0x68ee3ae thru 0x68f9c22 (0xb875 bytes) 2024-09-17T07:29:01.963 INFO:teuthology.orchestra.run.smithi045.stdout:886 read 0x68c3f29 thru 0x68c4972 (0xa4a bytes) 2024-09-17T07:29:01.963 INFO:teuthology.orchestra.run.smithi045.stdout:887 write 0x51283c6 thru 0x51309e4 (0x861f bytes) 2024-09-17T07:29:01.964 INFO:teuthology.orchestra.run.smithi045.stdout:889 read 0x1f0879d thru 0x1f11a5e (0x92c2 bytes) 2024-09-17T07:29:01.965 INFO:teuthology.orchestra.run.smithi045.stdout:890 punch from 0xa8913c4 to 0xa894de5, (0x3a21 bytes) 2024-09-17T07:29:01.965 INFO:teuthology.orchestra.run.smithi045.stdout:891 writesame 0x1a95269 thru 0x1a9cafc data_size 0x2(0x7894 bytes) 2024-09-17T07:29:01.972 INFO:teuthology.orchestra.run.smithi045.stdout:892 read 0x84bb927 thru 0x84c1a75 (0x614f bytes) 2024-09-17T07:29:01.973 INFO:teuthology.orchestra.run.smithi045.stdout:893 read 0x766a505 thru 0x7679fb5 (0xfab1 bytes) 2024-09-17T07:29:01.973 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:01.973 INFO:teuthology.orchestra.run.smithi045.stdout:896 write 0xa94403f thru 0xa9471fd (0x31bf bytes) 2024-09-17T07:29:01.974 INFO:teuthology.orchestra.run.smithi045.stdout:899 writesame 0x2651d3c thru 0x2655ebb data_size 0x40(0x4180 bytes) 2024-09-17T07:29:01.976 INFO:teuthology.orchestra.run.smithi045.stdout:902 read 0x5f4733 thru 0x5fff07 (0xb7d5 bytes) 2024-09-17T07:29:01.976 INFO:teuthology.orchestra.run.smithi045.stdout:903 writesame 0x1c2fe6d thru 0x1c3a45a data_size 0x43(0xa5ee bytes) 2024-09-17T07:29:01.978 INFO:teuthology.orchestra.run.smithi045.stdout:904 punch from 0x2f6ac23 to 0x2f71140, (0x651d bytes) 2024-09-17T07:29:01.978 INFO:teuthology.orchestra.run.smithi045.stdout:905 writesame 0x9fdef27 thru 0x9fe4cc7 data_size 0x1(0x5da1 bytes) 2024-09-17T07:29:01.997 INFO:teuthology.orchestra.run.smithi045.stdout:906 write 0xcf8e0fc thru 0xcf90bee (0x2af3 bytes) 2024-09-17T07:29:01.999 INFO:teuthology.orchestra.run.smithi045.stdout:910 write 0x1b09318 thru 0x1b131fc (0x9ee5 bytes) 2024-09-17T07:29:02.001 INFO:teuthology.orchestra.run.smithi045.stdout:911 write 0x63fffcd thru 0x64006af (0x6e3 bytes) 2024-09-17T07:29:02.002 INFO:teuthology.orchestra.run.smithi045.stdout:912 punch from 0x2659c6 to 0x270b47, (0xb181 bytes) 2024-09-17T07:29:02.002 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:02.002+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:02.004 INFO:teuthology.orchestra.run.smithi045.stdout:913 trunc from 0xcf90bef to 0xb7a2c9b 2024-09-17T07:29:02.010 INFO:teuthology.orchestra.run.smithi045.stdout:914 write 0x1588c7f thru 0x1593c6a (0xafec bytes) 2024-09-17T07:29:02.012 INFO:teuthology.orchestra.run.smithi045.stdout:918 read 0x39b3d74 thru 0x39bab19 (0x6da6 bytes) 2024-09-17T07:29:02.013 INFO:teuthology.orchestra.run.smithi045.stdout:920 trunc from 0xb7a2c9b to 0xa6691a1 2024-09-17T07:29:02.020 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:02.020 INFO:teuthology.orchestra.run.smithi045.stdout:922 punch from 0x8e321f0 to 0x8e41ebb, (0xfccb bytes) 2024-09-17T07:29:02.021 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:02.021 INFO:teuthology.orchestra.run.smithi045.stdout:924 write 0x2675d22 thru 0x26855e5 (0xf8c4 bytes) 2024-09-17T07:29:02.022 INFO:teuthology.orchestra.run.smithi045.stdout:925 write 0x9f2ad41 thru 0x9f3164d (0x690d bytes) 2024-09-17T07:29:02.034 INFO:teuthology.orchestra.run.smithi045.stdout:926 write 0xeaa4ef9 thru 0xeaad1a1 (0x82a9 bytes) 2024-09-17T07:29:02.035 INFO:teuthology.orchestra.run.smithi045.stdout:927 punch from 0xfdcc67 to 0xfe5acd, (0x8e66 bytes) 2024-09-17T07:29:02.035 INFO:teuthology.orchestra.run.smithi045.stdout:930 punch from 0x81fa9b2 to 0x82084a8, (0xdaf6 bytes) 2024-09-17T07:29:02.035 INFO:teuthology.orchestra.run.smithi045.stdout:934 writesame 0x610c thru 0x123b1 data_size 0x97(0xc2a6 bytes) 2024-09-17T07:29:02.037 INFO:teuthology.orchestra.run.smithi045.stdout:935 read 0x3ce6405 thru 0x3ced238 (0x6e34 bytes) 2024-09-17T07:29:02.037 INFO:teuthology.orchestra.run.smithi045.stdout:936 read 0x2497b4b thru 0x249fa4f (0x7f05 bytes) 2024-09-17T07:29:02.037 INFO:teuthology.orchestra.run.smithi045.stdout:937 writesame 0xa78c6dc thru 0xa798da5 data_size 0x23(0xc6ca bytes) 2024-09-17T07:29:02.040 INFO:teuthology.orchestra.run.smithi045.stdout:938 read 0xe4e197b thru 0xe4f17ee (0xfe74 bytes) 2024-09-17T07:29:02.041 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:02.041 INFO:teuthology.orchestra.run.smithi045.stdout:941 write 0x3844bf2 thru 0x384cb28 (0x7f37 bytes) 2024-09-17T07:29:02.042 INFO:teuthology.orchestra.run.smithi045.stdout:942 write 0x3c7d4c7 thru 0x3c813e9 (0x3f23 bytes) 2024-09-17T07:29:02.043 INFO:teuthology.orchestra.run.smithi045.stdout:943 writesame 0xc28b8c9 thru 0xc294a09 data_size 0x43(0x9141 bytes) 2024-09-17T07:29:02.045 INFO:teuthology.orchestra.run.smithi045.stdout:944 write 0x93be3a thru 0x945e0f (0x9fd6 bytes) 2024-09-17T07:29:02.047 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:02.047 INFO:teuthology.orchestra.run.smithi045.stdout:948 write 0x7c84ab7 thru 0x7c853bf (0x909 bytes) 2024-09-17T07:29:02.048 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:02.048 INFO:teuthology.orchestra.run.smithi045.stdout:950 writesame 0x5a6c77e thru 0x5a723b3 data_size 0x1d(0x5c36 bytes) 2024-09-17T07:29:02.049 INFO:teuthology.orchestra.run.smithi045.stdout:951 write 0xea0270c thru 0xea0713c (0x4a31 bytes) 2024-09-17T07:29:02.050 INFO:teuthology.orchestra.run.smithi045.stdout:952 write 0x5a6f412 thru 0x5a76473 (0x7062 bytes) 2024-09-17T07:29:02.051 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:02.051 INFO:teuthology.orchestra.run.smithi045.stdout:955 read 0xd020c66 thru 0xd02a895 (0x9c30 bytes) 2024-09-17T07:29:02.052 INFO:teuthology.orchestra.run.smithi045.stdout:957 clone 10 order 24 su 8388608 sc 6 2024-09-17T07:29:02.071 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:02.071 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:02.080 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:02.083 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:02.083+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:02.184 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:02.183+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:02.231 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:02.231 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:02.239 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:02.286 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:02.286+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:02.344 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:02.345 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:02.354 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:02.357 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-17T07:29:02.387 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:02.386+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:02.488 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:02.488+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:02.501 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:02.501 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:02.509 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:02.593 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:02.593+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:02.612 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:02.612 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:02.624 INFO:teuthology.orchestra.run.smithi045.stdout:960 trunc from 0xeaad1a2 to 0x46462c7 2024-09-17T07:29:02.664 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:02.664 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:02.678 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:02.680 INFO:teuthology.orchestra.run.smithi045.stdout:961 read 0x45245b5 thru 0x4529385 (0x4dd1 bytes) 2024-09-17T07:29:02.681 INFO:teuthology.orchestra.run.smithi045.stdout:962 write 0x2aba957 thru 0x2abb6f5 (0xd9f bytes) 2024-09-17T07:29:02.682 INFO:teuthology.orchestra.run.smithi045.stdout:963 punch from 0x531c38 to 0x537e8c, (0x6254 bytes) 2024-09-17T07:29:02.682 INFO:teuthology.orchestra.run.smithi045.stdout:964 write 0x3beb2a6 thru 0x3bee7b7 (0x3512 bytes) 2024-09-17T07:29:02.715 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:02.715 INFO:teuthology.orchestra.run.smithi045.stdout:966 punch from 0x3948617 to 0x395457a, (0xbf63 bytes) 2024-09-17T07:29:02.715 INFO:teuthology.orchestra.run.smithi045.stdout:967 read 0x41e4f29 thru 0x41ed9bc (0x8a94 bytes) 2024-09-17T07:29:02.715 INFO:teuthology.orchestra.run.smithi045.stdout:968 write 0x36bc19c thru 0x36c4b13 (0x8978 bytes) 2024-09-17T07:29:02.720 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:02.720+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:02.751 INFO:teuthology.orchestra.run.smithi045.stdout:970 trunc from 0x46462c7 to 0x4a71d71 2024-09-17T07:29:02.757 INFO:teuthology.orchestra.run.smithi045.stdout:971 punch from 0xd25e50 to 0xd30ced, (0xae9d bytes) 2024-09-17T07:29:02.757 INFO:teuthology.orchestra.run.smithi045.stdout:973 write 0x30627f0 thru 0x306876e (0x5f7f bytes) 2024-09-17T07:29:02.760 INFO:teuthology.orchestra.run.smithi045.stdout:974 punch from 0x19a5ac0 to 0x19ad456, (0x7996 bytes) 2024-09-17T07:29:02.765 INFO:teuthology.orchestra.run.smithi045.stdout:975 write 0x5d6d11b thru 0x5d7c07c (0xef62 bytes) 2024-09-17T07:29:02.767 INFO:teuthology.orchestra.run.smithi045.stdout:976 read 0x2195160 thru 0x219aece (0x5d6f bytes) 2024-09-17T07:29:02.773 INFO:teuthology.orchestra.run.smithi045.stdout:978 write 0x646ff7c thru 0x647089b (0x920 bytes) 2024-09-17T07:29:02.790 INFO:teuthology.orchestra.run.smithi045.stdout:979 write 0xc95b1a7 thru 0xc95b432 (0x28c bytes) 2024-09-17T07:29:02.791 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:02.790+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:02.792 INFO:teuthology.orchestra.run.smithi045.stdout:982 writesame 0x8808d58 thru 0x8815206 data_size 0x1(0xc4af bytes) 2024-09-17T07:29:02.818 INFO:teuthology.orchestra.run.smithi045.stdout:983 punch from 0x31233c to 0x3153c8, (0x308c bytes) 2024-09-17T07:29:02.818 INFO:teuthology.orchestra.run.smithi045.stdout:984 read 0x428667b thru 0x42948ff (0xe285 bytes) 2024-09-17T07:29:02.819 INFO:teuthology.orchestra.run.smithi045.stdout:986 read 0x74f9e20 thru 0x74fdee7 (0x40c8 bytes) 2024-09-17T07:29:02.819 INFO:teuthology.orchestra.run.smithi045.stdout:987 writesame 0x56c79cf thru 0x56d7321 data_size 0x1(0xf953 bytes) 2024-09-17T07:29:02.830 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:02.830 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:02.840 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:02.847 INFO:teuthology.orchestra.run.smithi045.stdout:988 write 0x69ded86 thru 0x69e227d (0x34f8 bytes) 2024-09-17T07:29:02.850 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:02.851 INFO:teuthology.orchestra.run.smithi045.stdout:990 writesame 0x94a279a thru 0x94a4015 data_size 0x2(0x187c bytes) 2024-09-17T07:29:02.853 INFO:teuthology.orchestra.run.smithi045.stdout:991 punch from 0x4c05274 to 0x4c060bb, (0xe47 bytes) 2024-09-17T07:29:02.853 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:02.853 INFO:teuthology.orchestra.run.smithi045.stdout:994 writesame 0x5d5e68a thru 0x5d604a6 data_size 0x1(0x1e1d bytes) 2024-09-17T07:29:02.857 INFO:teuthology.orchestra.run.smithi045.stdout:995 punch from 0x7e7d96 to 0x7f375a, (0xb9c4 bytes) 2024-09-17T07:29:02.857 INFO:teuthology.orchestra.run.smithi045.stdout:996 write 0xb88eca7 thru 0xb8997fb (0xab55 bytes) 2024-09-17T07:29:02.859 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:02.859 INFO:teuthology.orchestra.run.smithi045.stdout:998 read 0x4c9a212 thru 0x4c9a35a (0x149 bytes) 2024-09-17T07:29:02.859 INFO:teuthology.orchestra.run.smithi045.stdout:1000 write 0x74902ef thru 0x7497583 (0x7295 bytes) 2024-09-17T07:29:02.862 INFO:teuthology.orchestra.run.smithi045.stdout:1001 punch from 0x9fceb3f to 0x9fdb63d, (0xcafe bytes) 2024-09-17T07:29:02.862 INFO:teuthology.orchestra.run.smithi045.stdout:1002 punch from 0x2fa7309 to 0x2faa5ab, (0x32a2 bytes) 2024-09-17T07:29:02.862 INFO:teuthology.orchestra.run.smithi045.stdout:1003 write 0x5ffea37 thru 0x6000904 (0x1ece bytes) 2024-09-17T07:29:02.865 INFO:teuthology.orchestra.run.smithi045.stdout:1005 trunc from 0xc95b433 to 0xe04ba3e 2024-09-17T07:29:02.871 INFO:teuthology.orchestra.run.smithi045.stdout:1006 punch from 0x7be9369 to 0x7bf01a5, (0x6e3c bytes) 2024-09-17T07:29:02.871 INFO:teuthology.orchestra.run.smithi045.stdout:1007 write 0x636201f thru 0x63638f3 (0x18d5 bytes) 2024-09-17T07:29:02.872 INFO:teuthology.orchestra.run.smithi045.stdout:1008 read 0x1d57fcb thru 0x1d62438 (0xa46e bytes) 2024-09-17T07:29:02.872 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:02.873 INFO:teuthology.orchestra.run.smithi045.stdout:1010 trunc from 0xe04ba3e to 0xd72e6aa 2024-09-17T07:29:02.878 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:02.878 INFO:teuthology.orchestra.run.smithi045.stdout:1012 write 0x93ed75d thru 0x93f3a5a (0x62fe bytes) 2024-09-17T07:29:02.880 INFO:teuthology.orchestra.run.smithi045.stdout:1013 read 0x55f9d36 thru 0x55fece8 (0x4fb3 bytes) 2024-09-17T07:29:02.880 INFO:teuthology.orchestra.run.smithi045.stdout:1017 punch from 0x519d5ec to 0x51a95ed, (0xc001 bytes) 2024-09-17T07:29:02.880 INFO:teuthology.orchestra.run.smithi045.stdout:1018 write 0x955a361 thru 0x956355d (0x91fd bytes) 2024-09-17T07:29:02.881 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:02.881 INFO:teuthology.orchestra.run.smithi045.stdout:1021 writesame 0x8cc5b7a thru 0x8cc702d data_size 0x32(0x14b4 bytes) 2024-09-17T07:29:02.883 INFO:teuthology.orchestra.run.smithi045.stdout:1022 write 0x570b126 thru 0x5714a2c (0x9907 bytes) 2024-09-17T07:29:02.885 INFO:teuthology.orchestra.run.smithi045.stdout:1023 write 0x26ee1d9 thru 0x26f2f5c (0x4d84 bytes) 2024-09-17T07:29:02.887 INFO:teuthology.orchestra.run.smithi045.stdout:1024 punch from 0xc78e372 to 0xc794cc2, (0x6950 bytes) 2024-09-17T07:29:02.887 INFO:teuthology.orchestra.run.smithi045.stdout:1025 clone 11 order 19 su 16384 sc 14 2024-09-17T07:29:02.924 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:02.924+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:02.983 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:02.983 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:02.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:02.992 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:02.992+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:03.057 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:03.074 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:03.075 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:03.092 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:03.092 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:03.093 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:03.092+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:03.100 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:03.100 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:03.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:03.109 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:03.109 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:03.126 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:03.126 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:03.129 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:03.130 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:03.178 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T07:29:03.194 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:03.193+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:03.222 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:03.222 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:03.222 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:03.222 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:03.223 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:03.223 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone10 from 0xd72e6aa (overlap 0x46462c7) to 0x0 2024-09-17T07:29:03.235 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:03.277 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-17T07:29:03.295 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:03.295+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:03.314 INFO:teuthology.orchestra.run.smithi045.stdout:get_pg_num_history pg_num_history pg_num_history(e36 pg_nums {1={16=1},2={19=8},3={23=32}} deleted_pools ) 2024-09-17T07:29:03.314 INFO:teuthology.orchestra.run.smithi045.stdout:Importing pgid 3.7 2024-09-17T07:29:03.314 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e0c12f65:::rbd_data.1183cff49e3a.0000000000000077:7# 2024-09-17T07:29:03.315 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e0c12f65:::rbd_data.1183cff49e3a.0000000000000077:head# 2024-09-17T07:29:03.315 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 7=[]:{7=[7]} 2024-09-17T07:29:03.316 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e1042ee7:::rbd_data.1183350c2d82.0000000000000062:head# 2024-09-17T07:29:03.316 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 0=[]:{} 2024-09-17T07:29:03.319 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e16017d4:::rbd_data.11833dd73a5d.000000000000006a:head# 2024-09-17T07:29:03.319 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 8=[]:{} 2024-09-17T07:29:03.319 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e1790928:::rbd_data.11833dd73a5d.0000000000000003:8# 2024-09-17T07:29:03.322 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e1790928:::rbd_data.11833dd73a5d.0000000000000003:head# 2024-09-17T07:29:03.322 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 8=[]:{8=[8]} 2024-09-17T07:29:03.323 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e1b78bc5:::rbd_data.1183cff49e3a.0000000000000130:7# 2024-09-17T07:29:03.324 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e1b78bc5:::rbd_data.1183cff49e3a.0000000000000130:head# 2024-09-17T07:29:03.325 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 7=[]:{7=[7]} 2024-09-17T07:29:03.325 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e36c193a:::rbd_data.118312a5715.0000000000000017:head# 2024-09-17T07:29:03.325 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 6=[]:{} 2024-09-17T07:29:03.325 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e3c941af:::rbd_data.11833dd73a5d.0000000000000075:head# 2024-09-17T07:29:03.326 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 8=[]:{} 2024-09-17T07:29:03.326 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e3e93c9d:::rbd_data.1183cff49e3a.0000000000000038:head# 2024-09-17T07:29:03.326 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 7=[]:{} 2024-09-17T07:29:03.326 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e4c7175c:::rbd_data.11833dd73a5d.0000000000000008:8# 2024-09-17T07:29:03.328 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e4c7175c:::rbd_data.11833dd73a5d.0000000000000008:head# 2024-09-17T07:29:03.328 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 8=[]:{8=[8]} 2024-09-17T07:29:03.328 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e4ea5cb5:::rbd_data.1183350c2d82.0000000000000002:head# 2024-09-17T07:29:03.328 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 0=[]:{} 2024-09-17T07:29:03.330 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e51adc2a:::rbd_object_map.11832b928878.0000000000000005:head# 2024-09-17T07:29:03.330 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 0=[]:{} 2024-09-17T07:29:03.330 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e550acae:::rbd_data.11833dd73a5d.000000000000007e:head# 2024-09-17T07:29:03.330 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 8=[]:{} 2024-09-17T07:29:03.331 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e5a4793c:::rbd_data.11833dd73a5d.000000000000002e:head# 2024-09-17T07:29:03.331 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 8=[]:{} 2024-09-17T07:29:03.331 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e5d57cd4:::rbd_data.1183cff49e3a.00000000000000ba:7# 2024-09-17T07:29:03.332 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e5d57cd4:::rbd_data.1183cff49e3a.00000000000000ba:head# 2024-09-17T07:29:03.333 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 7=[]:{7=[7]} 2024-09-17T07:29:03.335 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e6553d3f:::rbd_data.11833dd73a5d.0000000000000046:head# 2024-09-17T07:29:03.335 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 8=[]:{} 2024-09-17T07:29:03.335 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e723d14d:::rbd_data.1183cff49e3a.0000000000000072:7# 2024-09-17T07:29:03.336 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e723d14d:::rbd_data.1183cff49e3a.0000000000000072:head# 2024-09-17T07:29:03.337 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 7=[]:{7=[7]} 2024-09-17T07:29:03.337 INFO:teuthology.orchestra.run.smithi045.stdout:Write #3:e7e9ecdf:::rbd_data.11833dd73a5d.0000000000000038:head# 2024-09-17T07:29:03.337 INFO:teuthology.orchestra.run.smithi045.stdout:snapset 8=[]:{} 2024-09-17T07:29:03.338 INFO:teuthology.orchestra.run.smithi045.stdout:write_pg epoch 35 info 3.7( v 35'55 (0'0,35'55] local-lis/les=23/24 n=23 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) 2024-09-17T07:29:03.338 INFO:teuthology.orchestra.run.smithi045.stdout:Import successful 2024-09-17T07:29:03.396 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:03.395+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:03.397 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:03.397 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:03.397 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:03.397 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:03.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.397 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:03.398 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:03.398 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.2627020380000005, 2024-09-17T07:29:03.398 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:29:03.398 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:03.398 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.398 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.398 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.398 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:29:03.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.398 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.398 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:03.399 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:03.399 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:29:03.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:29:03.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:29:03.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:29:03.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:29:03.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:29:03.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:29:03.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:29:03.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:29:03.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:29:03.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:29:03.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:29:03.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:29:03.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.402 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.402 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:29:03.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:29:03.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.402 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:03.402 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:29:03.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:29:03.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:29:03.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:29:03.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:29:03.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:29:03.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:29:03.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:29:03.403 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.404 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.404 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.404 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14607 3.1b 3:d9e58a41:::rbd_data.11837e68aeab.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 5913594~52370] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-17T07:29:03.404 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:59.208099+0000", 2024-09-17T07:29:03.404 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.1872101390000003, 2024-09-17T07:29:03.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.021548115999999999, 2024-09-17T07:29:03.404 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.404 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:03.404 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.404 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.405 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.405 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14607 2024-09-17T07:29:03.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.405 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.208099+0000", 2024-09-17T07:29:03.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.208099+0000", 2024-09-17T07:29:03.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.406 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.208102+0000", 2024-09-17T07:29:03.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8040000000000002e-06 2024-09-17T07:29:03.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.406 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.211950+0000", 2024-09-17T07:29:03.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0038476909999999999 2024-09-17T07:29:03.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.406 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.211977+0000", 2024-09-17T07:29:03.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7413999999999999e-05 2024-09-17T07:29:03.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.407 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.407 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.211990+0000", 2024-09-17T07:29:03.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2925000000000001e-05 2024-09-17T07:29:03.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.407 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.407 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.212082+0000", 2024-09-17T07:29:03.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2211000000000002e-05 2024-09-17T07:29:03.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.408 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.408 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.212205+0000", 2024-09-17T07:29:03.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012216000000000001 2024-09-17T07:29:03.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.408 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:29:03.408 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.212747+0000", 2024-09-17T07:29:03.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000542823 2024-09-17T07:29:03.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.408 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:03.408 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.222819+0000", 2024-09-17T07:29:03.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010071208999999999 2024-09-17T07:29:03.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:03.409 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.229577+0000", 2024-09-17T07:29:03.409 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0067583809999999999 2024-09-17T07:29:03.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:03.409 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.229616+0000", 2024-09-17T07:29:03.409 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9360000000000003e-05 2024-09-17T07:29:03.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.229648+0000", 2024-09-17T07:29:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1137999999999999e-05 2024-09-17T07:29:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15412 3.1b 3:dc936ee0:::rbd_data.1183515b0721.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.549072+0000", 2024-09-17T07:29:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.846237146, 2024-09-17T07:29:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031263618, 2024-09-17T07:29:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15412 2024-09-17T07:29:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.549072+0000", 2024-09-17T07:29:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.549072+0000", 2024-09-17T07:29:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.549074+0000", 2024-09-17T07:29:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6139999999999999e-06 2024-09-17T07:29:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.550472+0000", 2024-09-17T07:29:03.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013981390000000001 2024-09-17T07:29:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.550497+0000", 2024-09-17T07:29:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4904000000000001e-05 2024-09-17T07:29:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.550507+0000", 2024-09-17T07:29:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0339e-05 2024-09-17T07:29:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.413 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.550585+0000", 2024-09-17T07:29:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7643000000000002e-05 2024-09-17T07:29:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.550755+0000", 2024-09-17T07:29:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000170302 2024-09-17T07:29:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:29:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.556770+0000", 2024-09-17T07:29:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060148299999999997 2024-09-17T07:29:03.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.580210+0000", 2024-09-17T07:29:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.023439712000000001 2024-09-17T07:29:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.580252+0000", 2024-09-17T07:29:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2463000000000003e-05 2024-09-17T07:29:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.580298+0000", 2024-09-17T07:29:03.415 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5547999999999998e-05 2024-09-17T07:29:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.580336+0000", 2024-09-17T07:29:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8124000000000003e-05 2024-09-17T07:29:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15636 3.12 3:480d6a20:::rbd_data.11837e68aeab.0000000000000003:a [sparse-read 3932160~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.613580+0000", 2024-09-17T07:29:03.416 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.7817300180000002, 2024-09-17T07:29:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019203499999999998, 2024-09-17T07:29:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15636 2024-09-17T07:29:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613580+0000", 2024-09-17T07:29:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613580+0000", 2024-09-17T07:29:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613580+0000", 2024-09-17T07:29:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.01e-07 2024-09-17T07:29:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613583+0000", 2024-09-17T07:29:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6699999999999998e-06 2024-09-17T07:29:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613589+0000", 2024-09-17T07:29:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4690000000000002e-06 2024-09-17T07:29:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613594+0000", 2024-09-17T07:29:03.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5360000000000003e-06 2024-09-17T07:29:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.632297+0000", 2024-09-17T07:29:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.018703814999999999 2024-09-17T07:29:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.632376+0000", 2024-09-17T07:29:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8994999999999998e-05 2024-09-17T07:29:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.632783+0000", 2024-09-17T07:29:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00040661400000000002 2024-09-17T07:29:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15637 3.12 3:480d6a20:::rbd_data.11837e68aeab.0000000000000003:a [sparse-read 7340032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.613602+0000", 2024-09-17T07:29:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.781707323, 2024-09-17T07:29:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019383694, 2024-09-17T07:29:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15637 2024-09-17T07:29:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613602+0000", 2024-09-17T07:29:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613602+0000", 2024-09-17T07:29:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613603+0000", 2024-09-17T07:29:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0999999999999999e-07 2024-09-17T07:29:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613605+0000", 2024-09-17T07:29:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1459999999999999e-06 2024-09-17T07:29:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613611+0000", 2024-09-17T07:29:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2720000000000003e-06 2024-09-17T07:29:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613616+0000", 2024-09-17T07:29:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2850000000000002e-06 2024-09-17T07:29:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.632815+0000", 2024-09-17T07:29:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019198685 2024-09-17T07:29:03.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.632897+0000", 2024-09-17T07:29:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1866000000000005e-05 2024-09-17T07:29:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.632986+0000", 2024-09-17T07:29:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9030000000000006e-05 2024-09-17T07:29:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15647 3.12 3:4a12945c:::rbd_data.11837e68aeab.0000000000000000:a [sparse-read 1048576~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.613719+0000", 2024-09-17T07:29:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.7815905069999998, 2024-09-17T07:29:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019824970000000001, 2024-09-17T07:29:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15647 2024-09-17T07:29:03.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613719+0000", 2024-09-17T07:29:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613719+0000", 2024-09-17T07:29:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.427 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613720+0000", 2024-09-17T07:29:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9699999999999996e-07 2024-09-17T07:29:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613725+0000", 2024-09-17T07:29:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9240000000000004e-06 2024-09-17T07:29:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613732+0000", 2024-09-17T07:29:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:29:03.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613738+0000", 2024-09-17T07:29:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2889999999999997e-06 2024-09-17T07:29:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.633029+0000", 2024-09-17T07:29:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019291841000000001 2024-09-17T07:29:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.633118+0000", 2024-09-17T07:29:03.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8778999999999998e-05 2024-09-17T07:29:03.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.633544+0000", 2024-09-17T07:29:03.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00042574300000000002 2024-09-17T07:29:03.430 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.430 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.430 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.430 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15648 3.12 3:4a12945c:::rbd_data.11837e68aeab.0000000000000000:a [sparse-read 4456448~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.430 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.613746+0000", 2024-09-17T07:29:03.431 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.7815640310000003, 2024-09-17T07:29:03.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.020304853000000001, 2024-09-17T07:29:03.431 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.431 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:03.431 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.431 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.431 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.431 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15648 2024-09-17T07:29:03.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.431 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613746+0000", 2024-09-17T07:29:03.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613746+0000", 2024-09-17T07:29:03.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613746+0000", 2024-09-17T07:29:03.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:29:03.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613748+0000", 2024-09-17T07:29:03.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2649999999999999e-06 2024-09-17T07:29:03.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613755+0000", 2024-09-17T07:29:03.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:29:03.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613759+0000", 2024-09-17T07:29:03.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2769999999999999e-06 2024-09-17T07:29:03.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.633578+0000", 2024-09-17T07:29:03.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019818721000000001 2024-09-17T07:29:03.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.633651+0000", 2024-09-17T07:29:03.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3588000000000001e-05 2024-09-17T07:29:03.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.634050+0000", 2024-09-17T07:29:03.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00039928099999999997 2024-09-17T07:29:03.435 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.435 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.435 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.435 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15649 3.12 3:4a12945c:::rbd_data.11837e68aeab.0000000000000000:a [sparse-read 7864320~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.435 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.613765+0000", 2024-09-17T07:29:03.435 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.7815445199999997, 2024-09-17T07:29:03.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.020775378000000001, 2024-09-17T07:29:03.436 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.436 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:03.436 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.436 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.436 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.436 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15649 2024-09-17T07:29:03.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.436 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613765+0000", 2024-09-17T07:29:03.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613765+0000", 2024-09-17T07:29:03.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613765+0000", 2024-09-17T07:29:03.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9400000000000001e-07 2024-09-17T07:29:03.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613769+0000", 2024-09-17T07:29:03.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0879999999999999e-06 2024-09-17T07:29:03.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613775+0000", 2024-09-17T07:29:03.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5799999999999997e-06 2024-09-17T07:29:03.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613780+0000", 2024-09-17T07:29:03.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0270000000000003e-06 2024-09-17T07:29:03.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.634085+0000", 2024-09-17T07:29:03.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.020305042999999998 2024-09-17T07:29:03.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.634169+0000", 2024-09-17T07:29:03.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4028000000000003e-05 2024-09-17T07:29:03.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.634540+0000", 2024-09-17T07:29:03.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00037121799999999998 2024-09-17T07:29:03.440 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.440 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.440 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.440 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15652 3.1b 3:d9e58a41:::rbd_data.11837e68aeab.0000000000000002:a [sparse-read 3145728~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.440 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.613786+0000", 2024-09-17T07:29:03.440 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.7815230720000002, 2024-09-17T07:29:03.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019036137000000002, 2024-09-17T07:29:03.440 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.440 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.440 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:03.440 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.441 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.441 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.441 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15652 2024-09-17T07:29:03.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.441 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613786+0000", 2024-09-17T07:29:03.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613786+0000", 2024-09-17T07:29:03.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613787+0000", 2024-09-17T07:29:03.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:29:03.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613789+0000", 2024-09-17T07:29:03.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.945e-06 2024-09-17T07:29:03.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613795+0000", 2024-09-17T07:29:03.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4919999999999998e-06 2024-09-17T07:29:03.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613800+0000", 2024-09-17T07:29:03.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8539999999999996e-06 2024-09-17T07:29:03.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.632096+0000", 2024-09-17T07:29:03.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.018295948999999999 2024-09-17T07:29:03.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.632177+0000", 2024-09-17T07:29:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0698999999999994e-05 2024-09-17T07:29:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.632823+0000", 2024-09-17T07:29:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00064582300000000001 2024-09-17T07:29:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15653 3.1b 3:d9e58a41:::rbd_data.11837e68aeab.0000000000000002:a [sparse-read 6553600~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.613807+0000", 2024-09-17T07:29:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.781502218, 2024-09-17T07:29:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.01950499, 2024-09-17T07:29:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15653 2024-09-17T07:29:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613807+0000", 2024-09-17T07:29:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613807+0000", 2024-09-17T07:29:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613808+0000", 2024-09-17T07:29:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:29:03.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613810+0000", 2024-09-17T07:29:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7130000000000001e-06 2024-09-17T07:29:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613817+0000", 2024-09-17T07:29:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4320000000000004e-06 2024-09-17T07:29:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613823+0000", 2024-09-17T07:29:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8390000000000003e-06 2024-09-17T07:29:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.632855+0000", 2024-09-17T07:29:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019032250000000001 2024-09-17T07:29:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.632935+0000", 2024-09-17T07:29:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9832000000000004e-05 2024-09-17T07:29:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.633312+0000", 2024-09-17T07:29:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00037747799999999999 2024-09-17T07:29:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15654 3.12 3:480d6a20:::rbd_data.11837e68aeab.0000000000000003:a [sparse-read 2883584~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.613830+0000", 2024-09-17T07:29:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.7814799409999997, 2024-09-17T07:29:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.021282134000000001, 2024-09-17T07:29:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15654 2024-09-17T07:29:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613830+0000", 2024-09-17T07:29:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613830+0000", 2024-09-17T07:29:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613830+0000", 2024-09-17T07:29:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4099999999999999e-07 2024-09-17T07:29:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613832+0000", 2024-09-17T07:29:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9429999999999999e-06 2024-09-17T07:29:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613838+0000", 2024-09-17T07:29:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9880000000000001e-06 2024-09-17T07:29:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613843+0000", 2024-09-17T07:29:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6789999999999996e-06 2024-09-17T07:29:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.634573+0000", 2024-09-17T07:29:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.020730299000000001 2024-09-17T07:29:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.634660+0000", 2024-09-17T07:29:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7404000000000005e-05 2024-09-17T07:29:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635112+0000", 2024-09-17T07:29:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00045127999999999998 2024-09-17T07:29:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15655 3.12 3:480d6a20:::rbd_data.11837e68aeab.0000000000000003:a [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.613855+0000", 2024-09-17T07:29:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.7814540910000001, 2024-09-17T07:29:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.021503781, 2024-09-17T07:29:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15655 2024-09-17T07:29:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.454 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613855+0000", 2024-09-17T07:29:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613855+0000", 2024-09-17T07:29:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613856+0000", 2024-09-17T07:29:03.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7100000000000002e-07 2024-09-17T07:29:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613858+0000", 2024-09-17T07:29:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0279999999999999e-06 2024-09-17T07:29:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613864+0000", 2024-09-17T07:29:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6860000000000003e-06 2024-09-17T07:29:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613870+0000", 2024-09-17T07:29:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.93e-06 2024-09-17T07:29:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635169+0000", 2024-09-17T07:29:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.021299196999999999 2024-09-17T07:29:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635263+0000", 2024-09-17T07:29:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3783000000000007e-05 2024-09-17T07:29:03.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635359+0000", 2024-09-17T07:29:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6686000000000002e-05 2024-09-17T07:29:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15661 3.1b 3:da5d75ff:::rbd_data.11837e68aeab.0000000000000006:a [sparse-read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.613936+0000", 2024-09-17T07:29:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.7813732369999999, 2024-09-17T07:29:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019601668999999999, 2024-09-17T07:29:03.458 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15661 2024-09-17T07:29:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613936+0000", 2024-09-17T07:29:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613936+0000", 2024-09-17T07:29:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613937+0000", 2024-09-17T07:29:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4799999999999999e-07 2024-09-17T07:29:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613939+0000", 2024-09-17T07:29:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9659999999999999e-06 2024-09-17T07:29:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613945+0000", 2024-09-17T07:29:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2389999999999999e-06 2024-09-17T07:29:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613950+0000", 2024-09-17T07:29:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7119999999999999e-06 2024-09-17T07:29:03.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.633350+0000", 2024-09-17T07:29:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019399907000000001 2024-09-17T07:29:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.633439+0000", 2024-09-17T07:29:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9854999999999999e-05 2024-09-17T07:29:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.633538+0000", 2024-09-17T07:29:03.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8542000000000006e-05 2024-09-17T07:29:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15662 3.1b 3:da5d75ff:::rbd_data.11837e68aeab.0000000000000006:a [sparse-read 4718592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.613975+0000", 2024-09-17T07:29:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.7813348490000003, 2024-09-17T07:29:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019769788, 2024-09-17T07:29:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.463 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.464 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.464 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15662 2024-09-17T07:29:03.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.464 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613975+0000", 2024-09-17T07:29:03.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613975+0000", 2024-09-17T07:29:03.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613975+0000", 2024-09-17T07:29:03.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4000000000000002e-07 2024-09-17T07:29:03.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613978+0000", 2024-09-17T07:29:03.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4789999999999999e-06 2024-09-17T07:29:03.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613985+0000", 2024-09-17T07:29:03.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8959999999999997e-06 2024-09-17T07:29:03.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613990+0000", 2024-09-17T07:29:03.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4600000000000002e-06 2024-09-17T07:29:03.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.633569+0000", 2024-09-17T07:29:03.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019578958 2024-09-17T07:29:03.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.633644+0000", 2024-09-17T07:29:03.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5193999999999999e-05 2024-09-17T07:29:03.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.633744+0000", 2024-09-17T07:29:03.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010026099999999999 2024-09-17T07:29:03.467 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.467 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.467 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.467 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15663 3.1b 3:da5d75ff:::rbd_data.11837e68aeab.0000000000000006:a [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.467 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.613999+0000", 2024-09-17T07:29:03.468 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.7813110390000002, 2024-09-17T07:29:03.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019952095, 2024-09-17T07:29:03.468 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.468 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:03.468 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.468 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.468 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.468 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15663 2024-09-17T07:29:03.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.468 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.468 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613999+0000", 2024-09-17T07:29:03.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613999+0000", 2024-09-17T07:29:03.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.613999+0000", 2024-09-17T07:29:03.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1699999999999999e-07 2024-09-17T07:29:03.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.614006+0000", 2024-09-17T07:29:03.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.199e-06 2024-09-17T07:29:03.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.614013+0000", 2024-09-17T07:29:03.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4999999999999996e-06 2024-09-17T07:29:03.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.614020+0000", 2024-09-17T07:29:03.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9840000000000004e-06 2024-09-17T07:29:03.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.633771+0000", 2024-09-17T07:29:03.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019751563999999999 2024-09-17T07:29:03.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.633851+0000", 2024-09-17T07:29:03.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9488999999999997e-05 2024-09-17T07:29:03.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.633951+0000", 2024-09-17T07:29:03.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9942e-05 2024-09-17T07:29:03.472 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.472 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.472 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.472 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15674 3.12 3:492ab736:::rbd_data.1183515b0721.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7602176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.472 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.623947+0000", 2024-09-17T07:29:03.472 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.7713627600000001, 2024-09-17T07:29:03.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.029298226, 2024-09-17T07:29:03.472 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.472 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.472 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:03.473 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.473 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.473 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.473 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15674 2024-09-17T07:29:03.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.473 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.623947+0000", 2024-09-17T07:29:03.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.623947+0000", 2024-09-17T07:29:03.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.474 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.474 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.623949+0000", 2024-09-17T07:29:03.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.804e-06 2024-09-17T07:29:03.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.474 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.474 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.626589+0000", 2024-09-17T07:29:03.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0026404169999999999 2024-09-17T07:29:03.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.626607+0000", 2024-09-17T07:29:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.819e-05 2024-09-17T07:29:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.626628+0000", 2024-09-17T07:29:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.109e-05 2024-09-17T07:29:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635404+0000", 2024-09-17T07:29:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0087760310000000001 2024-09-17T07:29:03.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635563+0000", 2024-09-17T07:29:03.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000158968 2024-09-17T07:29:03.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:03.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.636433+0000", 2024-09-17T07:29:03.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00086989399999999996 2024-09-17T07:29:03.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:03.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.641971+0000", 2024-09-17T07:29:03.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0055374079999999997 2024-09-17T07:29:03.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:03.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.653031+0000", 2024-09-17T07:29:03.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011060604 2024-09-17T07:29:03.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:03.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.653137+0000", 2024-09-17T07:29:03.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010536599999999999 2024-09-17T07:29:03.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.653245+0000", 2024-09-17T07:29:03.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000108454 2024-09-17T07:29:03.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.478 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.478 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15677 3.17 3:ee3d9a6e:::rbd_data.1183515b0721.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7602176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.478 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.635302+0000", 2024-09-17T07:29:03.478 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.7600076360000001, 2024-09-17T07:29:03.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.026553516999999999, 2024-09-17T07:29:03.479 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.479 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:03.479 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.479 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.479 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.479 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15677 2024-09-17T07:29:03.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.479 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635302+0000", 2024-09-17T07:29:03.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635302+0000", 2024-09-17T07:29:03.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635303+0000", 2024-09-17T07:29:03.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.384e-06 2024-09-17T07:29:03.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.638080+0000", 2024-09-17T07:29:03.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027763520000000002 2024-09-17T07:29:03.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.638116+0000", 2024-09-17T07:29:03.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6134000000000003e-05 2024-09-17T07:29:03.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.638132+0000", 2024-09-17T07:29:03.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6433999999999999e-05 2024-09-17T07:29:03.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.638223+0000", 2024-09-17T07:29:03.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1036e-05 2024-09-17T07:29:03.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.638361+0000", 2024-09-17T07:29:03.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000138116 2024-09-17T07:29:03.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:29:03.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.638714+0000", 2024-09-17T07:29:03.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00035305900000000002 2024-09-17T07:29:03.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:03.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.643650+0000", 2024-09-17T07:29:03.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0049356909999999999 2024-09-17T07:29:03.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:03.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.661774+0000", 2024-09-17T07:29:03.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.018123599000000001 2024-09-17T07:29:03.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:03.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.661821+0000", 2024-09-17T07:29:03.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6928999999999997e-05 2024-09-17T07:29:03.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.661855+0000", 2024-09-17T07:29:03.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4783000000000002e-05 2024-09-17T07:29:03.484 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.484 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.484 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.484 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15678 3.1 3:8658ca32:::rbd_data.1183515b0721.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7602176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.485 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.638179+0000", 2024-09-17T07:29:03.485 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.757130654, 2024-09-17T07:29:03.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.029311198, 2024-09-17T07:29:03.485 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.485 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.485 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:03.485 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.485 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.485 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.485 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15678 2024-09-17T07:29:03.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.485 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.638179+0000", 2024-09-17T07:29:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.638179+0000", 2024-09-17T07:29:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.638180+0000", 2024-09-17T07:29:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:29:03.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.640765+0000", 2024-09-17T07:29:03.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025846739999999999 2024-09-17T07:29:03.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.640784+0000", 2024-09-17T07:29:03.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8961999999999999e-05 2024-09-17T07:29:03.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.488 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.640793+0000", 2024-09-17T07:29:03.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0149999999999992e-06 2024-09-17T07:29:03.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.488 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.488 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.640906+0000", 2024-09-17T07:29:03.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112812 2024-09-17T07:29:03.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.488 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.488 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.641136+0000", 2024-09-17T07:29:03.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00023037900000000001 2024-09-17T07:29:03.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:29:03.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.641781+0000", 2024-09-17T07:29:03.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00064479700000000001 2024-09-17T07:29:03.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:03.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.648222+0000", 2024-09-17T07:29:03.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064415109999999996 2024-09-17T07:29:03.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:03.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.667366+0000", 2024-09-17T07:29:03.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019143509 2024-09-17T07:29:03.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:03.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.667440+0000", 2024-09-17T07:29:03.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4006999999999998e-05 2024-09-17T07:29:03.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.667490+0000", 2024-09-17T07:29:03.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0093000000000003e-05 2024-09-17T07:29:03.490 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.490 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.490 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.491 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:15686 3.1b 3:deed2ce3:::rbd_data.1183515b0721.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7340032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T07:29:03.491 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.640833+0000", 2024-09-17T07:29:03.491 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.7544767329999997, 2024-09-17T07:29:03.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.026652561000000002, 2024-09-17T07:29:03.491 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.491 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.491 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:03.492 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.492 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.492 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.492 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 15686 2024-09-17T07:29:03.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.492 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.492 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.492 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.640833+0000", 2024-09-17T07:29:03.492 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.640833+0000", 2024-09-17T07:29:03.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.640834+0000", 2024-09-17T07:29:03.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6800000000000009e-07 2024-09-17T07:29:03.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.642455+0000", 2024-09-17T07:29:03.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016211579999999999 2024-09-17T07:29:03.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.494 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.642468+0000", 2024-09-17T07:29:03.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2863e-05 2024-09-17T07:29:03.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.494 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.642475+0000", 2024-09-17T07:29:03.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1679999999999998e-06 2024-09-17T07:29:03.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.495 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.642531+0000", 2024-09-17T07:29:03.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6382999999999999e-05 2024-09-17T07:29:03.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.495 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.495 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.642691+0000", 2024-09-17T07:29:03.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000159438 2024-09-17T07:29:03.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.495 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:29:03.495 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.643038+0000", 2024-09-17T07:29:03.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00034699200000000001 2024-09-17T07:29:03.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.496 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:03.496 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.647779+0000", 2024-09-17T07:29:03.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0047410270000000001 2024-09-17T07:29:03.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.496 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:03.496 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.667380+0000", 2024-09-17T07:29:03.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019601567 2024-09-17T07:29:03.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.496 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:03.496 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.667436+0000", 2024-09-17T07:29:03.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5769000000000001e-05 2024-09-17T07:29:03.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.667485+0000", 2024-09-17T07:29:03.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9227999999999997e-05 2024-09-17T07:29:03.497 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.497 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.497 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.497 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:17064 3.1 3:82de6a1a:::rbd_data.11838dd88fc3.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 12497574~13586] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-17T07:29:03.497 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.695201+0000", 2024-09-17T07:29:03.497 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.70010858600000003, 2024-09-17T07:29:03.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.019885645, 2024-09-17T07:29:03.498 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:03.498 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:03.498 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:03.498 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:03.498 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:03.498 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:03.498 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 17064 2024-09-17T07:29:03.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.498 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:03.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:03.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.695201+0000", 2024-09-17T07:29:03.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:03.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.695201+0000", 2024-09-17T07:29:03.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:03.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:03.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.695202+0000", 2024-09-17T07:29:03.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3790000000000001e-06 2024-09-17T07:29:03.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:03.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.699227+0000", 2024-09-17T07:29:03.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00402455 2024-09-17T07:29:03.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:03.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.699254+0000", 2024-09-17T07:29:03.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7234999999999999e-05 2024-09-17T07:29:03.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:03.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.699263+0000", 2024-09-17T07:29:03.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9800000000000004e-06 2024-09-17T07:29:03.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:03.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.699328+0000", 2024-09-17T07:29:03.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5055999999999995e-05 2024-09-17T07:29:03.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:03.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.699451+0000", 2024-09-17T07:29:03.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000122352 2024-09-17T07:29:03.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:29:03.502 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.700070+0000", 2024-09-17T07:29:03.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061912000000000002 2024-09-17T07:29:03.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.502 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:29:03.502 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.710580+0000", 2024-09-17T07:29:03.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010510392 2024-09-17T07:29:03.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.502 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:03.502 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.714971+0000", 2024-09-17T07:29:03.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043910119999999997 2024-09-17T07:29:03.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:03.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.715034+0000", 2024-09-17T07:29:03.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3425999999999999e-05 2024-09-17T07:29:03.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:03.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:03.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:03.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.715087+0000", 2024-09-17T07:29:03.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2142999999999999e-05 2024-09-17T07:29:03.503 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.503 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.503 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.503 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:03.503 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:03.503 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:03.505 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:03.505 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:03.504+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:03.567 INFO:teuthology.orchestra.run.smithi045.stdout:1026 trunc from 0xd72e6aa to 0x4117685 2024-09-17T07:29:03.605 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:03.605+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:03.636 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:03.636 INFO:teuthology.orchestra.run.smithi045.stdout:1029 read 0x947db4 thru 0x952c06 (0xae53 bytes) 2024-09-17T07:29:03.643 INFO:teuthology.orchestra.run.smithi045.stdout:1030 writesame 0x4edb595 thru 0x4ee8787 data_size 0x1(0xd1f3 bytes) 2024-09-17T07:29:03.643 INFO:teuthology.orchestra.run.smithi045.stdout:1031 read 0x1935cec thru 0x19389ed (0x2d02 bytes) 2024-09-17T07:29:03.647 INFO:teuthology.orchestra.run.smithi045.stdout:1032 trunc from 0x4ee8788 to 0x24e35b1 2024-09-17T07:29:03.673 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:03.674 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:03.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:03.726 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:03.725+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:03.731 INFO:teuthology.orchestra.run.smithi045.stdout:1033 read 0x75fdef thru 0x76916f (0x9381 bytes) 2024-09-17T07:29:03.732 INFO:teuthology.orchestra.run.smithi045.stdout:1036 writesame 0x142cad thru 0x14b79a data_size 0x1(0x8aee bytes) 2024-09-17T07:29:03.770 INFO:teuthology.orchestra.run.smithi045.stdout:1037 trunc from 0x24e35b1 to 0x2c6c726 2024-09-17T07:29:03.784 INFO:teuthology.orchestra.run.smithi045.stdout:1038 write 0x6bb0fca thru 0x6bb6e78 (0x5eaf bytes) 2024-09-17T07:29:03.788 INFO:teuthology.orchestra.run.smithi045.stdout:1039 read 0x4f5da60 thru 0x4f627e4 (0x4d85 bytes) 2024-09-17T07:29:03.799 INFO:teuthology.orchestra.run.smithi045.stdout:1041 writesame 0xc454cf4 thru 0xc461645 data_size 0x49(0xc952 bytes) 2024-09-17T07:29:03.807 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:03.807+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:03.809 INFO:teuthology.orchestra.run.smithi045.stdout:1042 trunc from 0xc461646 to 0x3429507 2024-09-17T07:29:03.818 INFO:teuthology.orchestra.run.smithi045.stdout:1043 read 0x526a11 thru 0x52c8e8 (0x5ed8 bytes) 2024-09-17T07:29:03.832 INFO:teuthology.orchestra.run.smithi045.stdout:1045 write 0xab16a5d thru 0xab1a6be (0x3c62 bytes) 2024-09-17T07:29:03.835 INFO:teuthology.orchestra.run.smithi045.stdout:1047 write 0x8df6a7 thru 0x8e4a12 (0x536c bytes) 2024-09-17T07:29:03.858 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:03.858 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:03.858 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:03.858 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:03.858 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:03.861 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /var/log/ceph/exp.3.7.3 2024-09-17T07:29:03.862 INFO:teuthology.orchestra.run.smithi045.stdout:1048 read 0x94a5fa4 thru 0x94af226 (0x9283 bytes) 2024-09-17T07:29:03.862 INFO:teuthology.orchestra.run.smithi045.stdout:1049 read 0x7106358 thru 0x71091ac (0x2e55 bytes) 2024-09-17T07:29:03.862 INFO:teuthology.orchestra.run.smithi045.stdout:1050 read 0x5e8274f thru 0x5e8a9a4 (0x8256 bytes) 2024-09-17T07:29:03.862 INFO:teuthology.orchestra.run.smithi045.stdout:1051 write 0x17db1d6 thru 0x17dd93c (0x2767 bytes) 2024-09-17T07:29:03.870 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:03.877 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:03.877 INFO:teuthology.orchestra.run.smithi045.stdout:1053 writesame 0x5c7c3f6 thru 0x5c88e13 data_size 0x29(0xca1e bytes) 2024-09-17T07:29:03.894 INFO:teuthology.orchestra.run.smithi045.stdout:1054 trunc from 0xab1a6bf to 0x20b2aad 2024-09-17T07:29:03.935 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:03.934+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:04.004 INFO:teuthology.orchestra.run.smithi045.stdout:1056 write 0x5e89400 thru 0x5e8da78 (0x4679 bytes) 2024-09-17T07:29:04.010 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:04.009+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:04.014 INFO:teuthology.orchestra.run.smithi045.stdout:1057 writesame 0x721a86e thru 0x7227dfb data_size 0x2(0xd58e bytes) 2024-09-17T07:29:04.014 INFO:teuthology.orchestra.run.smithi045.stdout:1061 read 0x276fcf thru 0x283a01 (0xca33 bytes) 2024-09-17T07:29:04.018 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:04.018 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:04.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:04.018 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:04.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.018 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.018 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:04.018 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8828654250000003, 2024-09-17T07:29:04.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:29:04.018 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.019 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.019 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.019 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.019 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.019 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.019 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:29:04.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.019 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:04.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:04.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:29:04.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:29:04.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:29:04.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:29:04.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:29:04.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:29:04.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:29:04.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:29:04.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:29:04.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:29:04.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:29:04.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:29:04.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:29:04.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:29:04.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:29:04.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:29:04.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:29:04.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:29:04.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:29:04.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:29:04.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:29:04.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:29:04.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.024 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.024 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.024 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:04.025 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8828285280000001, 2024-09-17T07:29:04.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:29:04.025 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.025 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.025 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.025 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.025 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.025 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.025 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:29:04.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.025 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.026 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:04.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.026 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.026 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:04.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.026 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.026 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:29:04.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:29:04.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:29:04.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:29:04.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:29:04.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:29:04.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:29:04.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:29:04.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:29:04.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:29:04.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:04.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:29:04.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.029 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.029 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:29:04.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:29:04.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.029 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.029 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:29:04.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:29:04.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.029 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.029 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:29:04.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:29:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:29:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:29:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:29:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:29:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.031 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.031 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:04.031 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8828025610000001, 2024-09-17T07:29:04.031 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:29:04.031 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.031 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.031 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.031 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.031 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.031 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.031 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:29:04.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.032 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.032 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.032 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:04.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.032 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.032 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:04.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:29:04.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:04.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:29:04.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:29:04.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:29:04.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:29:04.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:29:04.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:29:04.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:29:04.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:29:04.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.035 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:29:04.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:29:04.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.035 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.035 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:29:04.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:29:04.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.035 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.035 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:29:04.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:29:04.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:29:04.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:29:04.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:29:04.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:29:04.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:29:04.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:29:04.037 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.037 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.037 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.037 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.037 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:04.037 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8827779659999999, 2024-09-17T07:29:04.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:29:04.037 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.037 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.037 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.037 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.037 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.038 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.038 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:29:04.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.038 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:04.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:04.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.039 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.039 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:04.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:29:04.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.039 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.039 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:29:04.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:29:04.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.039 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:29:04.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:29:04.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.040 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:29:04.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:29:04.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.040 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:29:04.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:29:04.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.041 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.041 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:29:04.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:29:04.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.041 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.041 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:29:04.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:29:04.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.041 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.041 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:29:04.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:29:04.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.042 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:29:04.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:29:04.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.042 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:29:04.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:29:04.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.042 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:29:04.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:29:04.043 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.043 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.043 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.043 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.043 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:04.043 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8827544789999999, 2024-09-17T07:29:04.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:29:04.043 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.043 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.043 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.044 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.044 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.044 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.044 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:29:04.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.044 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.044 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:04.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.045 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:04.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.045 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:29:04.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:04.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.045 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:29:04.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:29:04.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:29:04.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:29:04.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:29:04.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:29:04.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:29:04.047 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:29:04.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.047 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.047 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.047 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:04.047 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:29:04.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.047 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.047 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.047 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:29:04.047 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:29:04.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.048 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.048 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:29:04.048 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:29:04.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.048 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.048 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:29:04.048 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:29:04.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.048 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.048 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:29:04.049 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:29:04.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.049 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.049 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:29:04.049 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:29:04.049 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.049 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.049 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.049 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.049 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8827407410000001, 2024-09-17T07:29:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:29:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:29:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:04.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:04.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:04.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:29:04.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:29:04.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:29:04.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:29:04.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:29:04.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:29:04.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:29:04.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.053 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.053 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:04.053 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:29:04.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.053 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.053 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:04.053 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:29:04.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.053 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.053 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:29:04.053 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:29:04.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.054 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.054 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:29:04.054 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:29:04.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.054 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.054 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:29:04.054 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:29:04.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.054 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.055 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:29:04.055 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:29:04.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.055 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.055 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:29:04.055 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:29:04.055 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.055 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.055 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.055 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.055 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:04.056 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8827105519999998, 2024-09-17T07:29:04.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:29:04.056 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.056 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.056 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.056 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.056 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.056 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.056 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:29:04.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.056 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:04.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:04.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:29:04.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:29:04.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:29:04.058 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:29:04.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:04.058 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:29:04.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:29:04.058 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:29:04.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.059 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:29:04.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:29:04.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.059 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:29:04.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:29:04.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.060 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:29:04.060 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:29:04.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.060 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.060 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:29:04.060 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:04.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.060 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.060 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:29:04.060 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:29:04.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.061 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:29:04.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:29:04.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.061 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:29:04.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:29:04.061 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.061 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.061 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.062 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.062 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:04.062 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8826740470000001, 2024-09-17T07:29:04.062 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:29:04.062 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.062 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.062 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.062 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.062 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.062 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:29:04.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.063 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.063 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:04.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.063 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:04.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.063 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:04.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:29:04.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:29:04.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:29:04.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:29:04.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:29:04.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.065 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.065 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:29:04.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:29:04.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.065 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.065 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:29:04.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:29:04.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.065 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.065 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:04.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:29:04.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.066 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.066 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:29:04.066 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:29:04.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.066 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.066 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:29:04.066 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:29:04.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.066 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.067 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:29:04.067 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:29:04.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.067 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.067 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:29:04.067 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:29:04.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.067 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.067 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:29:04.067 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:29:04.067 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.067 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.068 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.068 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.068 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:04.068 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8826462880000001, 2024-09-17T07:29:04.068 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:29:04.068 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.068 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.068 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.068 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.068 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.068 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.069 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:29:04.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.069 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.069 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.069 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:04.069 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.069 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.069 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:04.069 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.070 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.070 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:29:04.070 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:29:04.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.070 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.070 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:29:04.070 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:29:04.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.070 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.070 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:29:04.070 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:29:04.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.071 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.071 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:29:04.071 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:29:04.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.071 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.071 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:04.071 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:29:04.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.071 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.072 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:29:04.072 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:29:04.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.072 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.072 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:29:04.072 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:29:04.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.072 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.072 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:29:04.072 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:04.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.073 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.073 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:29:04.073 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:29:04.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.073 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.073 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:29:04.073 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:29:04.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.073 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.073 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:29:04.073 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:29:04.074 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.074 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.074 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.074 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.074 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:04.074 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8826241279999998, 2024-09-17T07:29:04.074 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:29:04.074 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.074 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.074 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.074 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.075 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.075 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.075 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:29:04.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.075 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.075 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.075 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:04.075 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.075 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.075 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:04.075 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.076 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.076 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:04.076 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:29:04.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.076 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.076 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:29:04.076 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:29:04.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.077 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.077 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:29:04.077 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:04.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.077 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.077 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.077 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:29:04.077 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:29:04.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.077 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.077 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.077 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:29:04.077 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:29:04.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.078 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.078 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.078 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:29:04.078 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:29:04.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.078 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.078 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.078 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:29:04.078 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:29:04.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.078 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.078 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.078 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:29:04.079 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:29:04.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.079 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.079 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.079 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:29:04.079 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:29:04.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.079 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.079 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.079 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:29:04.079 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:29:04.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.079 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.079 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.080 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:29:04.080 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:29:04.080 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.080 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.080 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.080 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.080 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.080 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:04.080 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8826109669999997, 2024-09-17T07:29:04.080 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:29:04.080 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.081 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.081 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.081 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.081 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.081 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:29:04.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.081 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.081 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.081 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.081 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:04.081 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.081 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.082 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.082 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:04.082 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.082 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.082 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:04.082 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:29:04.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.082 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.082 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:29:04.082 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:29:04.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.083 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.083 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:29:04.083 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:29:04.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.083 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.083 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:29:04.083 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:29:04.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.083 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.083 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:29:04.083 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:29:04.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.084 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.084 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:04.084 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:29:04.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.084 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.084 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:29:04.084 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:29:04.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.084 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.085 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:29:04.085 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:29:04.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.085 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.085 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:29:04.085 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:29:04.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.085 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.085 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:29:04.085 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:29:04.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.086 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.086 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:29:04.086 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:29:04.086 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.086 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.086 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.086 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.086 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.086 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:04.086 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8825923790000001, 2024-09-17T07:29:04.086 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:29:04.086 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.086 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:29:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:29:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:29:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:29:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:29:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:29:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:29:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:29:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:29:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:29:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:29:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:29:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:29:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:29:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:29:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:29:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:29:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:29:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:29:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8825595679999996, 2024-09-17T07:29:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:29:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:29:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:29:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:29:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:29:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:29:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:29:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:29:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:29:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:29:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:29:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:29:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:29:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:29:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:29:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:29:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:29:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:29:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:29:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:29:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8825405110000002, 2024-09-17T07:29:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:29:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:29:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:29:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:29:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:29:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:29:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:29:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:29:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:29:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:29:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:29:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:29:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:29:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:29:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:29:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:29:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:29:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:29:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:29:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:29:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:29:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:29:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:29:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:29:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8825184019999996, 2024-09-17T07:29:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:29:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:29:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:29:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:29:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:29:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:29:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:29:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:29:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:29:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:29:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:29:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:29:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:29:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:29:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:29:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:29:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:29:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:29:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:29:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.110 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:29:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:29:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:29:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:29:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8824922950000005, 2024-09-17T07:29:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:29:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:29:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:29:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:29:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:29:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:29:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:29:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:29:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:29:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:29:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:29:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:29:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:29:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:29:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:29:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:29:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:29:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:29:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:29:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:29:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:29:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:29:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:29:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8824631499999995, 2024-09-17T07:29:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:29:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:29:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:29:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:29:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:29:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:29:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:29:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:29:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:29:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:29:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:29:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:29:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:29:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:29:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:29:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:29:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:29:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:29:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:29:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:29:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:29:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:29:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8824313469999998, 2024-09-17T07:29:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:29:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:29:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:29:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:29:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:29:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:29:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:29:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:29:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:29:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:29:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:29:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:29:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:29:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:29:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:29:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:29:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:29:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:29:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:29:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:29:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:29:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:29:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:29:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8824044000000004, 2024-09-17T07:29:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:29:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:29:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:29:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:29:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:29:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:29:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:29:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:29:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:29:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:29:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:29:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:29:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:29:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:29:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:29:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:29:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:29:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:29:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:29:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:29:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:29:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:29:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.8823634509999998, 2024-09-17T07:29:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:29:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:29:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:29:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:29:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:29:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:29:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:29:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:29:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:29:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:29:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:29:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:29:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:29:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:29:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:29:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:29:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:29:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:29:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:29:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:29:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:29:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:29:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:29:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:29:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.142 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:04.143 INFO:teuthology.orchestra.run.smithi045.stdout:1063 trunc from 0x7227dfc to 0x391cad7 2024-09-17T07:29:04.144 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:04.145 INFO:teuthology.orchestra.run.smithi045.stdout:1064 trunc from 0x391cad7 to 0xb25a7c7 2024-09-17T07:29:04.145 INFO:teuthology.orchestra.run.smithi045.stdout:1067 write 0x3d13f7 thru 0x3d8539 (0x7143 bytes) 2024-09-17T07:29:04.145 INFO:teuthology.orchestra.run.smithi045.stdout:1068 trunc from 0xb25a7c7 to 0x94a6ea3 2024-09-17T07:29:04.145 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:04.146 INFO:teuthology.orchestra.run.smithi045.stdout:1070 trunc from 0x94a6ea3 to 0x5b5c0e8 2024-09-17T07:29:04.146 INFO:teuthology.orchestra.run.smithi045.stdout:1071 trunc from 0x5b5c0e8 to 0x658dbe7 2024-09-17T07:29:04.146 INFO:teuthology.orchestra.run.smithi045.stdout:1072 read 0xbac5ba thru 0xbb1cb2 (0x56f9 bytes) 2024-09-17T07:29:04.146 INFO:teuthology.orchestra.run.smithi045.stdout:1075 trunc from 0x658dbe7 to 0xaef306d 2024-09-17T07:29:04.146 INFO:teuthology.orchestra.run.smithi045.stdout:1076 write 0x27c0080 thru 0x27c7444 (0x73c5 bytes) 2024-09-17T07:29:04.146 INFO:teuthology.orchestra.run.smithi045.stdout:1078 write 0xeaa2fb thru 0xeb96ba (0xf3c0 bytes) 2024-09-17T07:29:04.146 INFO:teuthology.orchestra.run.smithi045.stdout:1080 read 0x87b790c thru 0x87babfa (0x32ef bytes) 2024-09-17T07:29:04.146 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:04.143+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:04.148 INFO:teuthology.orchestra.run.smithi045.stdout:1081 write 0xcbebad5 thru 0xcbeea8e (0x2fba bytes) 2024-09-17T07:29:04.151 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:04.153 INFO:teuthology.orchestra.run.smithi045.stdout:1085 trunc from 0xcbeea8f to 0xe273bc4 2024-09-17T07:29:04.159 INFO:teuthology.orchestra.run.smithi045.stdout:1086 write 0x8740b3c thru 0x8742211 (0x16d6 bytes) 2024-09-17T07:29:04.161 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-17T07:29:04.164 INFO:teuthology.orchestra.run.smithi045.stdout:1087 clone 12 order 19 su 8192 sc 7 2024-09-17T07:29:04.244 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:04.243+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:04.259 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:04.260 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:04.260 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:04.260 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:04.268 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:04.352 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:04.352+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:04.416 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:04.416 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:04.416 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:04.416 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:04.416 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:04.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:04.446 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:04.446+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:04.548 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:04.547+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:04.573 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:759 3.4 e28/23 3:20e84768:::rbd_data.1183350c2d82.0000000000000075:head v 28'6, mlcod=28'4)", 2024-09-17T07:29:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:29:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.647218459999999, 2024-09-17T07:29:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011641620000000001, 2024-09-17T07:29:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.574 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:29:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:29:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.575 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924861+0000", 2024-09-17T07:29:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6100000000000002e-07 2024-09-17T07:29:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924866+0000", 2024-09-17T07:29:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:29:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924875+0000", 2024-09-17T07:29:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7909999999999993e-06 2024-09-17T07:29:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.576 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924882+0000", 2024-09-17T07:29:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.418e-06 2024-09-17T07:29:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924965+0000", 2024-09-17T07:29:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3029999999999996e-05 2024-09-17T07:29:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.924987+0000", 2024-09-17T07:29:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1747e-05 2024-09-17T07:29:04.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.578 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.578 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.925980+0000", 2024-09-17T07:29:04.578 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00099299900000000005 2024-09-17T07:29:04.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.578 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.578 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:48.926025+0000", 2024-09-17T07:29:04.578 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4977e-05 2024-09-17T07:29:04.578 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.578 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.578 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.579 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1012 3.9 e29/23 3:93118134:::rbd_data.1183350c2d82.0000000000000067:head v 29'55, mlcod=29'53)", 2024-09-17T07:29:04.579 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:49.270691+0000", 2024-09-17T07:29:04.579 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.301388396, 2024-09-17T07:29:04.579 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011919299999999999, 2024-09-17T07:29:04.579 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.579 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.579 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.579 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.579 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.579 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270691+0000", 2024-09-17T07:29:04.579 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.580 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.580 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270691+0000", 2024-09-17T07:29:04.580 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.580 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.580 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270692+0000", 2024-09-17T07:29:04.580 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9800000000000002e-07 2024-09-17T07:29:04.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.580 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.580 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270699+0000", 2024-09-17T07:29:04.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1679999999999998e-06 2024-09-17T07:29:04.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.581 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.581 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270717+0000", 2024-09-17T07:29:04.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8193e-05 2024-09-17T07:29:04.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.581 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.581 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270726+0000", 2024-09-17T07:29:04.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4519999999999997e-06 2024-09-17T07:29:04.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.582 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.582 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270779+0000", 2024-09-17T07:29:04.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3202000000000003e-05 2024-09-17T07:29:04.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.582 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.582 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.270800+0000", 2024-09-17T07:29:04.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0846e-05 2024-09-17T07:29:04.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271841+0000", 2024-09-17T07:29:04.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001040921 2024-09-17T07:29:04.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:49.271883+0000", 2024-09-17T07:29:04.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2150000000000001e-05 2024-09-17T07:29:04.583 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.583 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.583 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.584 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1746 3.4 e30/23 3:21509c5f:::rbd_data.11832b928878.00000000000001bd:head v 30'14, mlcod=30'11)", 2024-09-17T07:29:04.584 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.791256+0000", 2024-09-17T07:29:04.584 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.780823659999999, 2024-09-17T07:29:04.584 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001583414, 2024-09-17T07:29:04.584 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.584 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.584 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.584 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.584 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791256+0000", 2024-09-17T07:29:04.584 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.585 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.585 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791256+0000", 2024-09-17T07:29:04.585 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.585 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.585 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791257+0000", 2024-09-17T07:29:04.585 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8599999999999997e-07 2024-09-17T07:29:04.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.585 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.585 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791290+0000", 2024-09-17T07:29:04.586 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3158999999999999e-05 2024-09-17T07:29:04.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.586 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.586 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791298+0000", 2024-09-17T07:29:04.586 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2700000000000004e-06 2024-09-17T07:29:04.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.586 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.586 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791305+0000", 2024-09-17T07:29:04.586 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8159999999999996e-06 2024-09-17T07:29:04.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791358+0000", 2024-09-17T07:29:04.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3300999999999997e-05 2024-09-17T07:29:04.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791377+0000", 2024-09-17T07:29:04.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9062000000000001e-05 2024-09-17T07:29:04.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.792791+0000", 2024-09-17T07:29:04.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014139160000000001 2024-09-17T07:29:04.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.588 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.588 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.792839+0000", 2024-09-17T07:29:04.588 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8103999999999999e-05 2024-09-17T07:29:04.588 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.588 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.588 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.588 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:2183 3.4 e30/23 3:221ed066:::rbd_data.11832b928878.0000000000000073:head v 30'19, mlcod=30'17)", 2024-09-17T07:29:04.588 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:51.139568+0000", 2024-09-17T07:29:04.588 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.432511577, 2024-09-17T07:29:04.588 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00124438, 2024-09-17T07:29:04.588 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.589 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.589 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.589 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.589 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.589 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139568+0000", 2024-09-17T07:29:04.589 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.589 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.589 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139568+0000", 2024-09-17T07:29:04.589 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.590 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.590 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139569+0000", 2024-09-17T07:29:04.590 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3769999999999999e-06 2024-09-17T07:29:04.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.590 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.590 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139580+0000", 2024-09-17T07:29:04.590 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0229e-05 2024-09-17T07:29:04.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.590 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.590 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139601+0000", 2024-09-17T07:29:04.591 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1818999999999999e-05 2024-09-17T07:29:04.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.591 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.591 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139611+0000", 2024-09-17T07:29:04.591 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0121e-05 2024-09-17T07:29:04.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.591 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.591 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139676+0000", 2024-09-17T07:29:04.591 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4900999999999994e-05 2024-09-17T07:29:04.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.591 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.592 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139706+0000", 2024-09-17T07:29:04.592 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9762000000000001e-05 2024-09-17T07:29:04.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.592 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.592 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.140765+0000", 2024-09-17T07:29:04.592 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001059019 2024-09-17T07:29:04.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.592 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.592 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.140812+0000", 2024-09-17T07:29:04.592 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7151999999999998e-05 2024-09-17T07:29:04.592 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.593 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.593 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.593 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:2190 3.0 e30/23 3:02fd7b4f:::rbd_data.11832b928878.00000000000000b4:head v 30'16, mlcod=30'14)", 2024-09-17T07:29:04.593 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:29:04.593 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.43216675, 2024-09-17T07:29:04.593 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001228395, 2024-09-17T07:29:04.593 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.593 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.593 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.593 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.594 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.594 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:29:04.594 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.597 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.597 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:29:04.597 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.597 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.597 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:29:04.597 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0399999999999996e-07 2024-09-17T07:29:04.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.597 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139918+0000", 2024-09-17T07:29:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8330000000000001e-06 2024-09-17T07:29:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139926+0000", 2024-09-17T07:29:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1720000000000007e-06 2024-09-17T07:29:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139932+0000", 2024-09-17T07:29:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1789999999999996e-06 2024-09-17T07:29:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139988+0000", 2024-09-17T07:29:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5265e-05 2024-09-17T07:29:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.140004+0000", 2024-09-17T07:29:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6171000000000002e-05 2024-09-17T07:29:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.141100+0000", 2024-09-17T07:29:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010964519999999999 2024-09-17T07:29:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.141141+0000", 2024-09-17T07:29:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0818999999999997e-05 2024-09-17T07:29:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:4848 3.9 e33/23 3:97363921:::rbd_data.1183cff49e3a.0000000000000004:head v 33'102, mlcod=32'100)", 2024-09-17T07:29:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 10.332583682999999, 2024-09-17T07:29:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023715110000000002, 2024-09-17T07:29:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239497+0000", 2024-09-17T07:29:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.53e-06 2024-09-17T07:29:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240142+0000", 2024-09-17T07:29:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00064481300000000005 2024-09-17T07:29:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240160+0000", 2024-09-17T07:29:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8349999999999999e-05 2024-09-17T07:29:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240169+0000", 2024-09-17T07:29:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7690000000000002e-06 2024-09-17T07:29:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240246+0000", 2024-09-17T07:29:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6277000000000002e-05 2024-09-17T07:29:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240270+0000", 2024-09-17T07:29:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4575000000000001e-05 2024-09-17T07:29:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.241813+0000", 2024-09-17T07:29:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015432860000000001 2024-09-17T07:29:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.241867+0000", 2024-09-17T07:29:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3910999999999999e-05 2024-09-17T07:29:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:4849 3.9 e33/23 3:97363921:::rbd_data.1183cff49e3a.0000000000000004:head v 33'104, mlcod=32'100)", 2024-09-17T07:29:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.240572+0000", 2024-09-17T07:29:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 10.331507565999999, 2024-09-17T07:29:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015942700000000001, 2024-09-17T07:29:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240572+0000", 2024-09-17T07:29:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240572+0000", 2024-09-17T07:29:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240572+0000", 2024-09-17T07:29:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3799999999999997e-07 2024-09-17T07:29:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240577+0000", 2024-09-17T07:29:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3000000000000003e-06 2024-09-17T07:29:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240584+0000", 2024-09-17T07:29:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2300000000000002e-06 2024-09-17T07:29:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240588+0000", 2024-09-17T07:29:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7340000000000002e-06 2024-09-17T07:29:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240936+0000", 2024-09-17T07:29:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00034820700000000001 2024-09-17T07:29:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240955+0000", 2024-09-17T07:29:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8808e-05 2024-09-17T07:29:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.242116+0000", 2024-09-17T07:29:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011614590000000001 2024-09-17T07:29:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.242166+0000", 2024-09-17T07:29:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9994000000000003e-05 2024-09-17T07:29:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:5460 3.9 e33/23 3:90773955:::rbd_data.1183cff49e3a.000000000000000a:head v 33'106, mlcod=33'104)", 2024-09-17T07:29:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 10.045487097000001, 2024-09-17T07:29:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020825850000000001, 2024-09-17T07:29:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526593+0000", 2024-09-17T07:29:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.54e-07 2024-09-17T07:29:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527050+0000", 2024-09-17T07:29:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000456522 2024-09-17T07:29:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527061+0000", 2024-09-17T07:29:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1722000000000001e-05 2024-09-17T07:29:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527067+0000", 2024-09-17T07:29:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0599999999999998e-06 2024-09-17T07:29:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527112+0000", 2024-09-17T07:29:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999000000000001e-05 2024-09-17T07:29:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527128+0000", 2024-09-17T07:29:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6920999999999999e-05 2024-09-17T07:29:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528636+0000", 2024-09-17T07:29:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001507836 2024-09-17T07:29:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528675+0000", 2024-09-17T07:29:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8671000000000003e-05 2024-09-17T07:29:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:5461 3.9 e33/23 3:90773955:::rbd_data.1183cff49e3a.000000000000000a:head v 33'108, mlcod=33'104)", 2024-09-17T07:29:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.527712+0000", 2024-09-17T07:29:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 10.044367951, 2024-09-17T07:29:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012502800000000001, 2024-09-17T07:29:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527712+0000", 2024-09-17T07:29:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527712+0000", 2024-09-17T07:29:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527712+0000", 2024-09-17T07:29:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.89e-07 2024-09-17T07:29:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527716+0000", 2024-09-17T07:29:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1520000000000002e-06 2024-09-17T07:29:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527724+0000", 2024-09-17T07:29:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9689999999999994e-06 2024-09-17T07:29:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527728+0000", 2024-09-17T07:29:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8909999999999997e-06 2024-09-17T07:29:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527776+0000", 2024-09-17T07:29:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7778999999999998e-05 2024-09-17T07:29:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527798+0000", 2024-09-17T07:29:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2558999999999999e-05 2024-09-17T07:29:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528931+0000", 2024-09-17T07:29:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001132814 2024-09-17T07:29:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528962+0000", 2024-09-17T07:29:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0626999999999997e-05 2024-09-17T07:29:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:6005 3.0 e33/23 3:037cd38f:::rbd_data.1183cff49e3a.0000000000000012:head v 33'25, mlcod=32'23)", 2024-09-17T07:29:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.9474539629999992, 2024-09-17T07:29:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019123899999999999, 2024-09-17T07:29:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624627+0000", 2024-09-17T07:29:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5600000000000001e-06 2024-09-17T07:29:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624777+0000", 2024-09-17T07:29:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015030800000000001 2024-09-17T07:29:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624792+0000", 2024-09-17T07:29:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5089000000000001e-05 2024-09-17T07:29:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624799+0000", 2024-09-17T07:29:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7939999999999997e-06 2024-09-17T07:29:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624855+0000", 2024-09-17T07:29:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5782999999999998e-05 2024-09-17T07:29:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624871+0000", 2024-09-17T07:29:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6192e-05 2024-09-17T07:29:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.626507+0000", 2024-09-17T07:29:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016354500000000001 2024-09-17T07:29:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.626538+0000", 2024-09-17T07:29:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1214000000000003e-05 2024-09-17T07:29:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:6683 3.9 e33/23 3:97ddc208:::rbd_data.1183cff49e3a.000000000000001e:head v 33'110, mlcod=33'108)", 2024-09-17T07:29:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.764442+0000", 2024-09-17T07:29:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.8076375609999999, 2024-09-17T07:29:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017329470000000001, 2024-09-17T07:29:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.764442+0000", 2024-09-17T07:29:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.764442+0000", 2024-09-17T07:29:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.764443+0000", 2024-09-17T07:29:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:29:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765126+0000", 2024-09-17T07:29:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00068291100000000002 2024-09-17T07:29:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765146+0000", 2024-09-17T07:29:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9406999999999999e-05 2024-09-17T07:29:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765156+0000", 2024-09-17T07:29:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0037000000000001e-05 2024-09-17T07:29:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765209+0000", 2024-09-17T07:29:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3732e-05 2024-09-17T07:29:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765226+0000", 2024-09-17T07:29:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6787e-05 2024-09-17T07:29:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.766139+0000", 2024-09-17T07:29:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00091277899999999996 2024-09-17T07:29:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.766175+0000", 2024-09-17T07:29:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5942999999999998e-05 2024-09-17T07:29:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:7670 3.4 e33/23 3:20e02957:::rbd_data.1183cff49e3a.000000000000002d:head v 33'30, mlcod=32'28)", 2024-09-17T07:29:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.955123+0000", 2024-09-17T07:29:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.6169560999999995, 2024-09-17T07:29:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0018080839999999999, 2024-09-17T07:29:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955123+0000", 2024-09-17T07:29:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955123+0000", 2024-09-17T07:29:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955125+0000", 2024-09-17T07:29:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3060000000000001e-06 2024-09-17T07:29:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955285+0000", 2024-09-17T07:29:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00016064700000000001 2024-09-17T07:29:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955303+0000", 2024-09-17T07:29:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7465999999999998e-05 2024-09-17T07:29:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955311+0000", 2024-09-17T07:29:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4780000000000008e-06 2024-09-17T07:29:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955373+0000", 2024-09-17T07:29:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1977000000000006e-05 2024-09-17T07:29:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955388+0000", 2024-09-17T07:29:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4260000000000001e-05 2024-09-17T07:29:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.956893+0000", 2024-09-17T07:29:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001505309 2024-09-17T07:29:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.956931+0000", 2024-09-17T07:29:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8640999999999998e-05 2024-09-17T07:29:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8595 3.4 e33/23 3:237b2a73:::rbd_data.1183cff49e3a.0000000000000049:head v 33'37, mlcod=33'35)", 2024-09-17T07:29:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.497437004, 2024-09-17T07:29:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020228939999999999, 2024-09-17T07:29:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074643+0000", 2024-09-17T07:29:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3099999999999997e-07 2024-09-17T07:29:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074834+0000", 2024-09-17T07:29:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000191137 2024-09-17T07:29:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074850+0000", 2024-09-17T07:29:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6101999999999999e-05 2024-09-17T07:29:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074859+0000", 2024-09-17T07:29:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2169999999999994e-06 2024-09-17T07:29:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074919+0000", 2024-09-17T07:29:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.003e-05 2024-09-17T07:29:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074944+0000", 2024-09-17T07:29:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5761e-05 2024-09-17T07:29:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076622+0000", 2024-09-17T07:29:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001677977 2024-09-17T07:29:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076665+0000", 2024-09-17T07:29:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2938999999999997e-05 2024-09-17T07:29:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8596 3.4 e33/23 3:237b2a73:::rbd_data.1183cff49e3a.0000000000000049:head v 33'39, mlcod=33'35)", 2024-09-17T07:29:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.075842+0000", 2024-09-17T07:29:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.4962370150000002, 2024-09-17T07:29:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012443630000000001, 2024-09-17T07:29:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075842+0000", 2024-09-17T07:29:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075842+0000", 2024-09-17T07:29:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075843+0000", 2024-09-17T07:29:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1099999999999995e-07 2024-09-17T07:29:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075848+0000", 2024-09-17T07:29:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1139999999999997e-06 2024-09-17T07:29:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075858+0000", 2024-09-17T07:29:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5920000000000007e-06 2024-09-17T07:29:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075862+0000", 2024-09-17T07:29:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5580000000000002e-06 2024-09-17T07:29:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075917+0000", 2024-09-17T07:29:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4274000000000002e-05 2024-09-17T07:29:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075938+0000", 2024-09-17T07:29:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1693000000000001e-05 2024-09-17T07:29:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.077049+0000", 2024-09-17T07:29:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011106079999999999 2024-09-17T07:29:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.077087+0000", 2024-09-17T07:29:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7913000000000001e-05 2024-09-17T07:29:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8721 3.4 e33/23 3:207ac2b1:::rbd_data.1183cff49e3a.000000000000004a:head v 33'40, mlcod=33'37)", 2024-09-17T07:29:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.079669+0000", 2024-09-17T07:29:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.4924109229999996, 2024-09-17T07:29:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017112749999999999, 2024-09-17T07:29:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079669+0000", 2024-09-17T07:29:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079669+0000", 2024-09-17T07:29:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079670+0000", 2024-09-17T07:29:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1540000000000001e-06 2024-09-17T07:29:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079849+0000", 2024-09-17T07:29:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017954900000000001 2024-09-17T07:29:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079864+0000", 2024-09-17T07:29:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4973e-05 2024-09-17T07:29:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079872+0000", 2024-09-17T07:29:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5170000000000002e-06 2024-09-17T07:29:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079929+0000", 2024-09-17T07:29:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7083000000000002e-05 2024-09-17T07:29:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079959+0000", 2024-09-17T07:29:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0474e-05 2024-09-17T07:29:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.081348+0000", 2024-09-17T07:29:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001388481 2024-09-17T07:29:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.081380+0000", 2024-09-17T07:29:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2044e-05 2024-09-17T07:29:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8872 3.0 e34/23 3:042b6c8e:::rbd_data.1183cff49e3a.0000000000000058:head v 34'31, mlcod=33'29)", 2024-09-17T07:29:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.322367+0000", 2024-09-17T07:29:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.2497120479999992, 2024-09-17T07:29:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016244619999999999, 2024-09-17T07:29:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322367+0000", 2024-09-17T07:29:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322367+0000", 2024-09-17T07:29:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322368+0000", 2024-09-17T07:29:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.16e-07 2024-09-17T07:29:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322539+0000", 2024-09-17T07:29:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017174200000000001 2024-09-17T07:29:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322554+0000", 2024-09-17T07:29:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4637000000000001e-05 2024-09-17T07:29:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322563+0000", 2024-09-17T07:29:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2369999999999998e-06 2024-09-17T07:29:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322622+0000", 2024-09-17T07:29:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8928000000000002e-05 2024-09-17T07:29:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322641+0000", 2024-09-17T07:29:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8601e-05 2024-09-17T07:29:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.323955+0000", 2024-09-17T07:29:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013146410000000001 2024-09-17T07:29:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.323992+0000", 2024-09-17T07:29:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.646e-05 2024-09-17T07:29:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:15673 3.0 e39/23 3:0518bbb2:::rbd_object_map.1183515b0721:head v 39'84, mlcod=39'81)", 2024-09-17T07:29:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.622698+0000", 2024-09-17T07:29:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.9493813370000002, 2024-09-17T07:29:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001189617, 2024-09-17T07:29:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622698+0000", 2024-09-17T07:29:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622698+0000", 2024-09-17T07:29:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622699+0000", 2024-09-17T07:29:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7400000000000002e-07 2024-09-17T07:29:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622712+0000", 2024-09-17T07:29:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3105000000000001e-05 2024-09-17T07:29:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622726+0000", 2024-09-17T07:29:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3995000000000001e-05 2024-09-17T07:29:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622734+0000", 2024-09-17T07:29:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6009999999999997e-06 2024-09-17T07:29:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622784+0000", 2024-09-17T07:29:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.058e-05 2024-09-17T07:29:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.622805+0000", 2024-09-17T07:29:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1012000000000001e-05 2024-09-17T07:29:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.623823+0000", 2024-09-17T07:29:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010180670000000001 2024-09-17T07:29:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.623888+0000", 2024-09-17T07:29:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4382999999999997e-05 2024-09-17T07:29:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:15683 3.0 e39/23 3:0518bbb2:::rbd_object_map.1183515b0721:head v 39'90, mlcod=39'84)", 2024-09-17T07:29:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.635895+0000", 2024-09-17T07:29:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.9361841609999999, 2024-09-17T07:29:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001403357, 2024-09-17T07:29:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635895+0000", 2024-09-17T07:29:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635895+0000", 2024-09-17T07:29:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635896+0000", 2024-09-17T07:29:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9999999999999995e-07 2024-09-17T07:29:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635905+0000", 2024-09-17T07:29:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6510000000000007e-06 2024-09-17T07:29:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635918+0000", 2024-09-17T07:29:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3168e-05 2024-09-17T07:29:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635931+0000", 2024-09-17T07:29:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2723e-05 2024-09-17T07:29:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.636183+0000", 2024-09-17T07:29:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00025214999999999998 2024-09-17T07:29:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.636210+0000", 2024-09-17T07:29:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6667000000000001e-05 2024-09-17T07:29:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.637240+0000", 2024-09-17T07:29:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010305889999999999 2024-09-17T07:29:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.637299+0000", 2024-09-17T07:29:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8409000000000003e-05 2024-09-17T07:29:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17069 3.0 e41/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 41'110, mlcod=40'108)", 2024-09-17T07:29:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.8323272799999999, 2024-09-17T07:29:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011408908000000001, 2024-09-17T07:29:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739753+0000", 2024-09-17T07:29:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:29:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743532+0000", 2024-09-17T07:29:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037794510000000001 2024-09-17T07:29:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743557+0000", 2024-09-17T07:29:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5253e-05 2024-09-17T07:29:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743569+0000", 2024-09-17T07:29:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1161999999999999e-05 2024-09-17T07:29:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743650+0000", 2024-09-17T07:29:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1087999999999993e-05 2024-09-17T07:29:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743674+0000", 2024-09-17T07:29:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4562e-05 2024-09-17T07:29:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751095+0000", 2024-09-17T07:29:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074211140000000004 2024-09-17T07:29:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751161+0000", 2024-09-17T07:29:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5707e-05 2024-09-17T07:29:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17240 3.0 e42/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 42'113, mlcod=41'110)", 2024-09-17T07:29:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 1.3466687959999999, 2024-09-17T07:29:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064392440000000002, 2024-09-17T07:29:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:29:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225418+0000", 2024-09-17T07:29:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8360000000000003e-06 2024-09-17T07:29:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225426+0000", 2024-09-17T07:29:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2670000000000006e-06 2024-09-17T07:29:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225434+0000", 2024-09-17T07:29:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0560000000000005e-06 2024-09-17T07:29:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225514+0000", 2024-09-17T07:29:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9979000000000001e-05 2024-09-17T07:29:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225535+0000", 2024-09-17T07:29:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0965e-05 2024-09-17T07:29:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231797+0000", 2024-09-17T07:29:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062621129999999997 2024-09-17T07:29:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231850+0000", 2024-09-17T07:29:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2611999999999997e-05 2024-09-17T07:29:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:04.670 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:04.671 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:04.672 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:04.671+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:04.772 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:04.771+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:04.776 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:04.777 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:04.777 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:04.777 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:04.785 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:04.873 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:04.873+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:04.887 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:04.888 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:04.888 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:04.896 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:04.982 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:04.981+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:05.041 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:05.041 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:05.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:05.041 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:05.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.041 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:6683 3.9 3:97ddc208:::rbd_data.1183cff49e3a.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:05.042 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.763526+0000", 2024-09-17T07:29:05.042 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.276050916999999, 2024-09-17T07:29:05.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032433677000000001, 2024-09-17T07:29:05.042 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.042 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.042 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:05.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.042 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6683 2024-09-17T07:29:05.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.042 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.763526+0000", 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.763526+0000", 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.763527+0000", 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.78e-07 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764048+0000", 2024-09-17T07:29:05.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00052137799999999997 2024-09-17T07:29:05.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764060+0000", 2024-09-17T07:29:05.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2014000000000001e-05 2024-09-17T07:29:05.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764072+0000", 2024-09-17T07:29:05.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1762e-05 2024-09-17T07:29:05.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764130+0000", 2024-09-17T07:29:05.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7759e-05 2024-09-17T07:29:05.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:05.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764194+0000", 2024-09-17T07:29:05.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4189999999999994e-05 2024-09-17T07:29:05.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-17T07:29:05.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764332+0000", 2024-09-17T07:29:05.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013821300000000001 2024-09-17T07:29:05.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:05.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795822+0000", 2024-09-17T07:29:05.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031489847000000001 2024-09-17T07:29:05.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:05.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795858+0000", 2024-09-17T07:29:05.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6452999999999998e-05 2024-09-17T07:29:05.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:05.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795917+0000", 2024-09-17T07:29:05.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8851999999999998e-05 2024-09-17T07:29:05.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795959+0000", 2024-09-17T07:29:05.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2231000000000003e-05 2024-09-17T07:29:05.047 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.047 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.047 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.047 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:6684 3.9 3:97ddc208:::rbd_data.1183cff49e3a.000000000000001e:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:05.047 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.764091+0000", 2024-09-17T07:29:05.047 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.275485734, 2024-09-17T07:29:05.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032300207999999997, 2024-09-17T07:29:05.048 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.048 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.048 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:05.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.048 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6684 2024-09-17T07:29:05.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.048 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764091+0000", 2024-09-17T07:29:05.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764091+0000", 2024-09-17T07:29:05.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764092+0000", 2024-09-17T07:29:05.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8700000000000005e-07 2024-09-17T07:29:05.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764094+0000", 2024-09-17T07:29:05.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9189999999999999e-06 2024-09-17T07:29:05.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764102+0000", 2024-09-17T07:29:05.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2710000000000003e-06 2024-09-17T07:29:05.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764118+0000", 2024-09-17T07:29:05.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6231000000000001e-05 2024-09-17T07:29:05.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764995+0000", 2024-09-17T07:29:05.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00087753699999999996 2024-09-17T07:29:05.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:05.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.765094+0000", 2024-09-17T07:29:05.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8027000000000002e-05 2024-09-17T07:29:05.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-17T07:29:05.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794998+0000", 2024-09-17T07:29:05.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029904612000000001 2024-09-17T07:29:05.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:05.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796267+0000", 2024-09-17T07:29:05.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001268911 2024-09-17T07:29:05.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:05.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796328+0000", 2024-09-17T07:29:05.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0594000000000003e-05 2024-09-17T07:29:05.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:05.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796356+0000", 2024-09-17T07:29:05.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8787999999999999e-05 2024-09-17T07:29:05.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796391+0000", 2024-09-17T07:29:05.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4631000000000001e-05 2024-09-17T07:29:05.053 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.053 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.053 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.053 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4483.0:6683 3.9 e33/23 ondisk, result = 0)", 2024-09-17T07:29:05.053 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:29:05.054 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.273337937999999, 2024-09-17T07:29:05.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029728505999999998, 2024-09-17T07:29:05.054 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.054 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.054 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:05.054 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:29:05.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:29:05.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:29:05.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9400000000000004e-07 2024-09-17T07:29:05.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766243+0000", 2024-09-17T07:29:05.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9419999999999997e-06 2024-09-17T07:29:05.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794812+0000", 2024-09-17T07:29:05.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028568324999999999 2024-09-17T07:29:05.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794822+0000", 2024-09-17T07:29:05.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0786e-05 2024-09-17T07:29:05.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795847+0000", 2024-09-17T07:29:05.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010248919999999999 2024-09-17T07:29:05.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795854+0000", 2024-09-17T07:29:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4919999999999998e-06 2024-09-17T07:29:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795967+0000", 2024-09-17T07:29:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113275 2024-09-17T07:29:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:6685 3.14 3:2a16c27c:::rbd_object_map.1183cff49e3a.0000000000000007:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.770642+0000", 2024-09-17T07:29:05.057 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.268935037, 2024-09-17T07:29:05.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027608490999999999, 2024-09-17T07:29:05.058 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.058 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.058 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:05.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.058 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6685 2024-09-17T07:29:05.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.058 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770642+0000", 2024-09-17T07:29:05.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770642+0000", 2024-09-17T07:29:05.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770643+0000", 2024-09-17T07:29:05.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.824e-06 2024-09-17T07:29:05.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770651+0000", 2024-09-17T07:29:05.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9680000000000006e-06 2024-09-17T07:29:05.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794795+0000", 2024-09-17T07:29:05.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024143286 2024-09-17T07:29:05.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794811+0000", 2024-09-17T07:29:05.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6112000000000001e-05 2024-09-17T07:29:05.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794961+0000", 2024-09-17T07:29:05.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014987 2024-09-17T07:29:05.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:05.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795048+0000", 2024-09-17T07:29:05.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7740999999999999e-05 2024-09-17T07:29:05.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:05.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795317+0000", 2024-09-17T07:29:05.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026868500000000001 2024-09-17T07:29:05.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:05.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.797274+0000", 2024-09-17T07:29:05.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019571660000000002 2024-09-17T07:29:05.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:05.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.798166+0000", 2024-09-17T07:29:05.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00089202400000000005 2024-09-17T07:29:05.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:05.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.798212+0000", 2024-09-17T07:29:05.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.596e-05 2024-09-17T07:29:05.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.798250+0000", 2024-09-17T07:29:05.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7855000000000003e-05 2024-09-17T07:29:05.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.065 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.065 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18266 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.065 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657062+0000", 2024-09-17T07:29:05.065 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.3825147600000001, 2024-09-17T07:29:05.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011987574000000001, 2024-09-17T07:29:05.065 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.066 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.066 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.066 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18266 2024-09-17T07:29:05.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.066 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657062+0000", 2024-09-17T07:29:05.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657062+0000", 2024-09-17T07:29:05.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657062+0000", 2024-09-17T07:29:05.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:05.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657063+0000", 2024-09-17T07:29:05.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2799999999999995e-07 2024-09-17T07:29:05.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657066+0000", 2024-09-17T07:29:05.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8609999999999998e-06 2024-09-17T07:29:05.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657068+0000", 2024-09-17T07:29:05.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2069999999999998e-06 2024-09-17T07:29:05.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669046+0000", 2024-09-17T07:29:05.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011977685 2024-09-17T07:29:05.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669049+0000", 2024-09-17T07:29:05.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7809999999999999e-06 2024-09-17T07:29:05.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.069 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.069 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18267 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.069 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657071+0000", 2024-09-17T07:29:05.070 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.382505622, 2024-09-17T07:29:05.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011999589, 2024-09-17T07:29:05.070 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.070 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.070 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.070 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18267 2024-09-17T07:29:05.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.071 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657071+0000", 2024-09-17T07:29:05.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657071+0000", 2024-09-17T07:29:05.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657071+0000", 2024-09-17T07:29:05.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:29:05.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657073+0000", 2024-09-17T07:29:05.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6309999999999999e-06 2024-09-17T07:29:05.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657076+0000", 2024-09-17T07:29:05.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9840000000000001e-06 2024-09-17T07:29:05.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657078+0000", 2024-09-17T07:29:05.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.221e-06 2024-09-17T07:29:05.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669066+0000", 2024-09-17T07:29:05.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011987799 2024-09-17T07:29:05.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669071+0000", 2024-09-17T07:29:05.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7439999999999998e-06 2024-09-17T07:29:05.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.075 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.075 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18268 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.075 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657081+0000", 2024-09-17T07:29:05.075 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.3824955299999999, 2024-09-17T07:29:05.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012009094999999999, 2024-09-17T07:29:05.075 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.075 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.075 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.076 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18268 2024-09-17T07:29:05.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.076 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657081+0000", 2024-09-17T07:29:05.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657081+0000", 2024-09-17T07:29:05.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657081+0000", 2024-09-17T07:29:05.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1500000000000001e-07 2024-09-17T07:29:05.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657082+0000", 2024-09-17T07:29:05.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0299999999999997e-07 2024-09-17T07:29:05.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657085+0000", 2024-09-17T07:29:05.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.915e-06 2024-09-17T07:29:05.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657087+0000", 2024-09-17T07:29:05.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.249e-06 2024-09-17T07:29:05.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669086+0000", 2024-09-17T07:29:05.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011998879 2024-09-17T07:29:05.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669090+0000", 2024-09-17T07:29:05.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9339999999999999e-06 2024-09-17T07:29:05.079 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.080 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.080 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18269 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.080 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657090+0000", 2024-09-17T07:29:05.080 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.382486337, 2024-09-17T07:29:05.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0120194, 2024-09-17T07:29:05.081 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.081 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.081 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.081 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18269 2024-09-17T07:29:05.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.081 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657090+0000", 2024-09-17T07:29:05.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657090+0000", 2024-09-17T07:29:05.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657090+0000", 2024-09-17T07:29:05.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:05.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657091+0000", 2024-09-17T07:29:05.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0500000000000002e-07 2024-09-17T07:29:05.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657094+0000", 2024-09-17T07:29:05.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.067e-06 2024-09-17T07:29:05.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657100+0000", 2024-09-17T07:29:05.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.507e-06 2024-09-17T07:29:05.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669106+0000", 2024-09-17T07:29:05.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012005767000000001 2024-09-17T07:29:05.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669110+0000", 2024-09-17T07:29:05.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9380000000000002e-06 2024-09-17T07:29:05.086 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.086 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.086 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.087 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18270 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.087 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657103+0000", 2024-09-17T07:29:05.087 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.3824733410000001, 2024-09-17T07:29:05.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012026421000000001, 2024-09-17T07:29:05.087 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.087 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.087 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.088 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18270 2024-09-17T07:29:05.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.088 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657103+0000", 2024-09-17T07:29:05.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657103+0000", 2024-09-17T07:29:05.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657103+0000", 2024-09-17T07:29:05.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2000000000000001e-07 2024-09-17T07:29:05.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657104+0000", 2024-09-17T07:29:05.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1399999999999995e-07 2024-09-17T07:29:05.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657107+0000", 2024-09-17T07:29:05.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0759999999999999e-06 2024-09-17T07:29:05.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657110+0000", 2024-09-17T07:29:05.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2989999999999999e-06 2024-09-17T07:29:05.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669126+0000", 2024-09-17T07:29:05.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012015915 2024-09-17T07:29:05.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669130+0000", 2024-09-17T07:29:05.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.997e-06 2024-09-17T07:29:05.092 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.092 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.092 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.092 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18271 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.092 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657113+0000", 2024-09-17T07:29:05.092 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.38246376, 2024-09-17T07:29:05.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012038475, 2024-09-17T07:29:05.093 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.093 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.093 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.093 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18271 2024-09-17T07:29:05.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.093 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657113+0000", 2024-09-17T07:29:05.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657113+0000", 2024-09-17T07:29:05.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657113+0000", 2024-09-17T07:29:05.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0900000000000001e-07 2024-09-17T07:29:05.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657114+0000", 2024-09-17T07:29:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1200000000000001e-07 2024-09-17T07:29:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657117+0000", 2024-09-17T07:29:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0000000000000001e-06 2024-09-17T07:29:05.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657119+0000", 2024-09-17T07:29:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999999e-06 2024-09-17T07:29:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669147+0000", 2024-09-17T07:29:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012028133999999999 2024-09-17T07:29:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669151+0000", 2024-09-17T07:29:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8199999999999998e-06 2024-09-17T07:29:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18272 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657122+0000", 2024-09-17T07:29:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.3824542229999999, 2024-09-17T07:29:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012048522000000001, 2024-09-17T07:29:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.097 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18272 2024-09-17T07:29:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657122+0000", 2024-09-17T07:29:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657122+0000", 2024-09-17T07:29:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657123+0000", 2024-09-17T07:29:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.41e-07 2024-09-17T07:29:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657123+0000", 2024-09-17T07:29:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7400000000000002e-07 2024-09-17T07:29:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657133+0000", 2024-09-17T07:29:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8279999999999998e-06 2024-09-17T07:29:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657146+0000", 2024-09-17T07:29:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2978e-05 2024-09-17T07:29:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669167+0000", 2024-09-17T07:29:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012020786 2024-09-17T07:29:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669171+0000", 2024-09-17T07:29:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8149999999999999e-06 2024-09-17T07:29:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18295 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657164+0000", 2024-09-17T07:29:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.382412709, 2024-09-17T07:29:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012026273000000001, 2024-09-17T07:29:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18295 2024-09-17T07:29:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657164+0000", 2024-09-17T07:29:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657164+0000", 2024-09-17T07:29:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657164+0000", 2024-09-17T07:29:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7199999999999999e-07 2024-09-17T07:29:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657167+0000", 2024-09-17T07:29:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4700000000000001e-06 2024-09-17T07:29:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657178+0000", 2024-09-17T07:29:05.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1066999999999999e-05 2024-09-17T07:29:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657183+0000", 2024-09-17T07:29:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.843e-06 2024-09-17T07:29:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669186+0000", 2024-09-17T07:29:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012003290999999999 2024-09-17T07:29:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669190+0000", 2024-09-17T07:29:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0300000000000004e-06 2024-09-17T07:29:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18296 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657194+0000", 2024-09-17T07:29:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.3823823120000001, 2024-09-17T07:29:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012015122, 2024-09-17T07:29:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.106 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18296 2024-09-17T07:29:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657194+0000", 2024-09-17T07:29:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657194+0000", 2024-09-17T07:29:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657195+0000", 2024-09-17T07:29:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8299999999999997e-07 2024-09-17T07:29:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657202+0000", 2024-09-17T07:29:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.216e-06 2024-09-17T07:29:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657208+0000", 2024-09-17T07:29:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4450000000000001e-06 2024-09-17T07:29:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657213+0000", 2024-09-17T07:29:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6630000000000001e-06 2024-09-17T07:29:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669205+0000", 2024-09-17T07:29:05.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011992361 2024-09-17T07:29:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669209+0000", 2024-09-17T07:29:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9539999999999998e-06 2024-09-17T07:29:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18297 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657220+0000", 2024-09-17T07:29:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.3823569199999999, 2024-09-17T07:29:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012008522000000001, 2024-09-17T07:29:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.111 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.112 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18297 2024-09-17T07:29:05.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.112 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657220+0000", 2024-09-17T07:29:05.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657220+0000", 2024-09-17T07:29:05.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657220+0000", 2024-09-17T07:29:05.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5900000000000002e-07 2024-09-17T07:29:05.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657222+0000", 2024-09-17T07:29:05.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9089999999999998e-06 2024-09-17T07:29:05.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657228+0000", 2024-09-17T07:29:05.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8640000000000001e-06 2024-09-17T07:29:05.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657233+0000", 2024-09-17T07:29:05.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6469999999999997e-06 2024-09-17T07:29:05.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669222+0000", 2024-09-17T07:29:05.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011989641000000001 2024-09-17T07:29:05.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669228+0000", 2024-09-17T07:29:05.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0020000000000003e-06 2024-09-17T07:29:05.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.120 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.121 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18298 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.121 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657238+0000", 2024-09-17T07:29:05.121 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.3823382500000001, 2024-09-17T07:29:05.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012009072000000001, 2024-09-17T07:29:05.121 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.121 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.122 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.122 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18298 2024-09-17T07:29:05.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.123 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657238+0000", 2024-09-17T07:29:05.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657238+0000", 2024-09-17T07:29:05.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657239+0000", 2024-09-17T07:29:05.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9000000000000002e-07 2024-09-17T07:29:05.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657241+0000", 2024-09-17T07:29:05.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:29:05.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657246+0000", 2024-09-17T07:29:05.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7880000000000003e-06 2024-09-17T07:29:05.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657252+0000", 2024-09-17T07:29:05.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4870000000000002e-06 2024-09-17T07:29:05.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669244+0000", 2024-09-17T07:29:05.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011991793000000001 2024-09-17T07:29:05.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669247+0000", 2024-09-17T07:29:05.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8130000000000002e-06 2024-09-17T07:29:05.130 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.130 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.130 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.131 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18299 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.131 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657258+0000", 2024-09-17T07:29:05.131 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.382318712, 2024-09-17T07:29:05.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012008586999999999, 2024-09-17T07:29:05.131 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.131 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.132 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.132 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18299 2024-09-17T07:29:05.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.133 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657258+0000", 2024-09-17T07:29:05.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657258+0000", 2024-09-17T07:29:05.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657258+0000", 2024-09-17T07:29:05.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7e-07 2024-09-17T07:29:05.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657260+0000", 2024-09-17T07:29:05.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.733e-06 2024-09-17T07:29:05.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657266+0000", 2024-09-17T07:29:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6840000000000001e-06 2024-09-17T07:29:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657270+0000", 2024-09-17T07:29:05.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5929999999999999e-06 2024-09-17T07:29:05.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669262+0000", 2024-09-17T07:29:05.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011992252 2024-09-17T07:29:05.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669266+0000", 2024-09-17T07:29:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9550000000000002e-06 2024-09-17T07:29:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.141 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18300 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.141 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657276+0000", 2024-09-17T07:29:05.141 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.3823006499999999, 2024-09-17T07:29:05.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012010353, 2024-09-17T07:29:05.141 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.142 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.142 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.142 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.142 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.142 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.143 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18300 2024-09-17T07:29:05.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.143 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657276+0000", 2024-09-17T07:29:05.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657276+0000", 2024-09-17T07:29:05.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657276+0000", 2024-09-17T07:29:05.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3799999999999998e-07 2024-09-17T07:29:05.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657278+0000", 2024-09-17T07:29:05.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6619999999999999e-06 2024-09-17T07:29:05.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657284+0000", 2024-09-17T07:29:05.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2469999999999997e-06 2024-09-17T07:29:05.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657289+0000", 2024-09-17T07:29:05.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4039999999999997e-06 2024-09-17T07:29:05.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669282+0000", 2024-09-17T07:29:05.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011993859000000001 2024-09-17T07:29:05.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669286+0000", 2024-09-17T07:29:05.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:29:05.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.150 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.151 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18301 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.151 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657295+0000", 2024-09-17T07:29:05.151 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.3822819960000001, 2024-09-17T07:29:05.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.012010817999999999, 2024-09-17T07:29:05.151 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.151 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18301 2024-09-17T07:29:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.152 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657295+0000", 2024-09-17T07:29:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657295+0000", 2024-09-17T07:29:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657295+0000", 2024-09-17T07:29:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:29:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657297+0000", 2024-09-17T07:29:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.672e-06 2024-09-17T07:29:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657302+0000", 2024-09-17T07:29:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6960000000000002e-06 2024-09-17T07:29:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657306+0000", 2024-09-17T07:29:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9820000000000002e-06 2024-09-17T07:29:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669302+0000", 2024-09-17T07:29:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011995212 2024-09-17T07:29:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669305+0000", 2024-09-17T07:29:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8809999999999998e-06 2024-09-17T07:29:05.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18303 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657355+0000", 2024-09-17T07:29:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.382221623, 2024-09-17T07:29:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011988591, 2024-09-17T07:29:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18303 2024-09-17T07:29:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657355+0000", 2024-09-17T07:29:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657355+0000", 2024-09-17T07:29:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657355+0000", 2024-09-17T07:29:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-07 2024-09-17T07:29:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657356+0000", 2024-09-17T07:29:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.136e-06 2024-09-17T07:29:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657360+0000", 2024-09-17T07:29:05.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2600000000000001e-06 2024-09-17T07:29:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657367+0000", 2024-09-17T07:29:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1450000000000002e-06 2024-09-17T07:29:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669339+0000", 2024-09-17T07:29:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011972251999999999 2024-09-17T07:29:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669344+0000", 2024-09-17T07:29:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4680000000000003e-06 2024-09-17T07:29:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:18304 3.1a 3.5d37047a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.657370+0000", 2024-09-17T07:29:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.3822063449999999, 2024-09-17T07:29:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011988831, 2024-09-17T07:29:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.160 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-17T07:29:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 18304 2024-09-17T07:29:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657370+0000", 2024-09-17T07:29:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657370+0000", 2024-09-17T07:29:05.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657370+0000", 2024-09-17T07:29:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3799999999999999e-07 2024-09-17T07:29:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657372+0000", 2024-09-17T07:29:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.285e-06 2024-09-17T07:29:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657375+0000", 2024-09-17T07:29:05.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-06 2024-09-17T07:29:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.657377+0000", 2024-09-17T07:29:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3980000000000002e-06 2024-09-17T07:29:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669356+0000", 2024-09-17T07:29:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.011978173 2024-09-17T07:29:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:05.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.669359+0000", 2024-09-17T07:29:05.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.467e-06 2024-09-17T07:29:05.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.164 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.164 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.164 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:05.165 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:29:05.166 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:05.166 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:05.165+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:05.242 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone11 from 0xe273bc4 (overlap 0x20b2aad) to 0x9cf333 2024-09-17T07:29:05.266 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:05.266+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:05.324 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:05.324 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:05.334 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:05.334 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:05.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.334 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.334 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:05.334 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.073917040000000003, 2024-09-17T07:29:05.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073932233, 2024-09-17T07:29:05.334 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.334 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.335 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.335 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:29:05.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.335 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:05.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:05.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:05.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:05.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:29:05.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:29:05.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:29:05.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:29:05.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:29:05.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:29:05.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.337 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.338 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20071 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.338 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.272429+0000", 2024-09-17T07:29:05.338 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.054107192999999998, 2024-09-17T07:29:05.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05415441, 2024-09-17T07:29:05.338 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.338 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.338 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.338 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20071 2024-09-17T07:29:05.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.338 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272429+0000", 2024-09-17T07:29:05.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272429+0000", 2024-09-17T07:29:05.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272429+0000", 2024-09-17T07:29:05.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.05e-07 2024-09-17T07:29:05.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272430+0000", 2024-09-17T07:29:05.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8000000000000004e-07 2024-09-17T07:29:05.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272434+0000", 2024-09-17T07:29:05.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2690000000000001e-06 2024-09-17T07:29:05.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272437+0000", 2024-09-17T07:29:05.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6290000000000002e-06 2024-09-17T07:29:05.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.341 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.341 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20169 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.341 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.297587+0000", 2024-09-17T07:29:05.341 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028949325000000001, 2024-09-17T07:29:05.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029023184, 2024-09-17T07:29:05.341 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.341 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.341 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.342 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20169 2024-09-17T07:29:05.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.342 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297587+0000", 2024-09-17T07:29:05.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297587+0000", 2024-09-17T07:29:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297587+0000", 2024-09-17T07:29:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-09-17T07:29:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297591+0000", 2024-09-17T07:29:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.659e-06 2024-09-17T07:29:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297598+0000", 2024-09-17T07:29:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3000000000000004e-06 2024-09-17T07:29:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297604+0000", 2024-09-17T07:29:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0920000000000003e-06 2024-09-17T07:29:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20245 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.320911+0000", 2024-09-17T07:29:05.344 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0056253800000000001, 2024-09-17T07:29:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0057233579999999996, 2024-09-17T07:29:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20245 2024-09-17T07:29:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320911+0000", 2024-09-17T07:29:05.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320911+0000", 2024-09-17T07:29:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320911+0000", 2024-09-17T07:29:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:29:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320912+0000", 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320915+0000", 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.835e-06 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320918+0000", 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7049999999999999e-06 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.348 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.348 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:05.348 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.073906686999999999, 2024-09-17T07:29:05.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074029136999999995, 2024-09-17T07:29:05.348 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.348 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.348 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.348 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:29:05.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.349 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:05.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:05.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:05.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:05.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:29:05.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:29:05.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:29:05.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:29:05.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:29:05.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:29:05.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.351 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.351 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20094 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.351 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.279545+0000", 2024-09-17T07:29:05.351 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.046991500999999998, 2024-09-17T07:29:05.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047139233000000003, 2024-09-17T07:29:05.351 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.351 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.352 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.352 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20094 2024-09-17T07:29:05.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.352 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279545+0000", 2024-09-17T07:29:05.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279545+0000", 2024-09-17T07:29:05.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279546+0000", 2024-09-17T07:29:05.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0839999999999999e-06 2024-09-17T07:29:05.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279550+0000", 2024-09-17T07:29:05.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4669999999999998e-06 2024-09-17T07:29:05.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279559+0000", 2024-09-17T07:29:05.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1719999999999996e-06 2024-09-17T07:29:05.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279564+0000", 2024-09-17T07:29:05.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9980000000000003e-06 2024-09-17T07:29:05.354 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.354 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.354 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.355 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20170 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.355 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.297612+0000", 2024-09-17T07:29:05.355 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028923912999999999, 2024-09-17T07:29:05.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029097475000000001, 2024-09-17T07:29:05.355 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.355 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.355 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.355 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.355 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.355 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.355 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20170 2024-09-17T07:29:05.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.356 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297612+0000", 2024-09-17T07:29:05.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297612+0000", 2024-09-17T07:29:05.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297613+0000", 2024-09-17T07:29:05.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.58e-07 2024-09-17T07:29:05.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297615+0000", 2024-09-17T07:29:05.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0219999999999999e-06 2024-09-17T07:29:05.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297622+0000", 2024-09-17T07:29:05.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9870000000000002e-06 2024-09-17T07:29:05.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297628+0000", 2024-09-17T07:29:05.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1689999999999997e-06 2024-09-17T07:29:05.358 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.358 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.358 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.358 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20246 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.358 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.320921+0000", 2024-09-17T07:29:05.358 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0056156660000000001, 2024-09-17T07:29:05.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0058196309999999996, 2024-09-17T07:29:05.358 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20246 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320921+0000", 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320921+0000", 2024-09-17T07:29:05.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320921+0000", 2024-09-17T07:29:05.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-07 2024-09-17T07:29:05.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320923+0000", 2024-09-17T07:29:05.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.874e-06 2024-09-17T07:29:05.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320926+0000", 2024-09-17T07:29:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.242e-06 2024-09-17T07:29:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320929+0000", 2024-09-17T07:29:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.644e-06 2024-09-17T07:29:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.073891235999999999, 2024-09-17T07:29:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074120440999999995, 2024-09-17T07:29:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:29:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:05.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:05.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:29:05.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:29:05.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:29:05.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:29:05.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:29:05.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:29:05.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:29:05.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.364 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.365 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20095 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.365 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.279577+0000", 2024-09-17T07:29:05.365 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.046959287000000002, 2024-09-17T07:29:05.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047214672999999999, 2024-09-17T07:29:05.365 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.365 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.365 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.365 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20095 2024-09-17T07:29:05.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.365 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279577+0000", 2024-09-17T07:29:05.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279577+0000", 2024-09-17T07:29:05.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279577+0000", 2024-09-17T07:29:05.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:29:05.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279579+0000", 2024-09-17T07:29:05.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:29:05.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279583+0000", 2024-09-17T07:29:05.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6399999999999996e-06 2024-09-17T07:29:05.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279586+0000", 2024-09-17T07:29:05.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0769999999999999e-06 2024-09-17T07:29:05.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.368 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.368 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20171 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.368 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.297635+0000", 2024-09-17T07:29:05.368 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028901455, 2024-09-17T07:29:05.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029180640000000001, 2024-09-17T07:29:05.368 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.368 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.368 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.369 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20171 2024-09-17T07:29:05.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.369 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297635+0000", 2024-09-17T07:29:05.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297635+0000", 2024-09-17T07:29:05.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297635+0000", 2024-09-17T07:29:05.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:29:05.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297637+0000", 2024-09-17T07:29:05.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0140000000000001e-06 2024-09-17T07:29:05.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297644+0000", 2024-09-17T07:29:05.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7689999999999999e-06 2024-09-17T07:29:05.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297650+0000", 2024-09-17T07:29:05.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6790000000000002e-06 2024-09-17T07:29:05.371 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.372 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.372 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.372 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20247 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.372 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.320937+0000", 2024-09-17T07:29:05.372 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.005599578, 2024-09-17T07:29:05.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059021660000000004, 2024-09-17T07:29:05.372 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.372 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.372 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.372 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.372 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.372 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.373 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20247 2024-09-17T07:29:05.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.373 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320937+0000", 2024-09-17T07:29:05.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320937+0000", 2024-09-17T07:29:05.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320937+0000", 2024-09-17T07:29:05.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8999999999999998e-07 2024-09-17T07:29:05.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320938+0000", 2024-09-17T07:29:05.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1039999999999999e-06 2024-09-17T07:29:05.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320943+0000", 2024-09-17T07:29:05.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.994e-06 2024-09-17T07:29:05.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320948+0000", 2024-09-17T07:29:05.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.955e-06 2024-09-17T07:29:05.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.375 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.375 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20096 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.376 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.279591+0000", 2024-09-17T07:29:05.376 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.046945648, 2024-09-17T07:29:05.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047272143000000003, 2024-09-17T07:29:05.376 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.376 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.376 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.376 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20096 2024-09-17T07:29:05.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.376 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279591+0000", 2024-09-17T07:29:05.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279591+0000", 2024-09-17T07:29:05.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279591+0000", 2024-09-17T07:29:05.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:29:05.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279592+0000", 2024-09-17T07:29:05.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1039999999999999e-06 2024-09-17T07:29:05.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279595+0000", 2024-09-17T07:29:05.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.247e-06 2024-09-17T07:29:05.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279598+0000", 2024-09-17T07:29:05.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7140000000000002e-06 2024-09-17T07:29:05.379 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.379 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.379 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.380 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20172 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.380 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.297658+0000", 2024-09-17T07:29:05.380 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028878562999999999, 2024-09-17T07:29:05.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029228595, 2024-09-17T07:29:05.380 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.380 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.380 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.380 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20172 2024-09-17T07:29:05.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.381 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297658+0000", 2024-09-17T07:29:05.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297658+0000", 2024-09-17T07:29:05.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297658+0000", 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297660+0000", 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1280000000000002e-06 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297668+0000", 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5170000000000002e-06 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297674+0000", 2024-09-17T07:29:05.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0530000000000003e-06 2024-09-17T07:29:05.383 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.383 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.383 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.383 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20248 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.383 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.320955+0000", 2024-09-17T07:29:05.383 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0055813870000000002, 2024-09-17T07:29:05.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0059550690000000003, 2024-09-17T07:29:05.383 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.383 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.383 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.384 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.384 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.384 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.384 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20248 2024-09-17T07:29:05.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.384 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320955+0000", 2024-09-17T07:29:05.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320955+0000", 2024-09-17T07:29:05.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320955+0000", 2024-09-17T07:29:05.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-17T07:29:05.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320957+0000", 2024-09-17T07:29:05.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.978e-06 2024-09-17T07:29:05.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320967+0000", 2024-09-17T07:29:05.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0200999999999999e-05 2024-09-17T07:29:05.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320972+0000", 2024-09-17T07:29:05.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8169999999999998e-06 2024-09-17T07:29:05.386 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.386 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.386 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.386 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20097 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.387 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.279601+0000", 2024-09-17T07:29:05.387 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.046935117999999998, 2024-09-17T07:29:05.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047337122000000002, 2024-09-17T07:29:05.387 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.387 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.387 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.387 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20097 2024-09-17T07:29:05.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.387 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279601+0000", 2024-09-17T07:29:05.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279601+0000", 2024-09-17T07:29:05.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279601+0000", 2024-09-17T07:29:05.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2000000000000001e-07 2024-09-17T07:29:05.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279602+0000", 2024-09-17T07:29:05.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0699999999999999e-06 2024-09-17T07:29:05.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279605+0000", 2024-09-17T07:29:05.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.959e-06 2024-09-17T07:29:05.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279608+0000", 2024-09-17T07:29:05.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4849999999999999e-06 2024-09-17T07:29:05.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20173 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.297683+0000", 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028853554999999999, 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029279396999999999, 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20173 2024-09-17T07:29:05.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297683+0000", 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297683+0000", 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297683+0000", 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:29:05.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297685+0000", 2024-09-17T07:29:05.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9759999999999998e-06 2024-09-17T07:29:05.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297693+0000", 2024-09-17T07:29:05.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5739999999999998e-06 2024-09-17T07:29:05.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297699+0000", 2024-09-17T07:29:05.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1979999999999998e-06 2024-09-17T07:29:05.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.393 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.393 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20249 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.393 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.320979+0000", 2024-09-17T07:29:05.393 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.005557429, 2024-09-17T07:29:05.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060104119999999997, 2024-09-17T07:29:05.393 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.393 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.394 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.394 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20249 2024-09-17T07:29:05.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.394 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320979+0000", 2024-09-17T07:29:05.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320979+0000", 2024-09-17T07:29:05.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320979+0000", 2024-09-17T07:29:05.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:29:05.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320981+0000", 2024-09-17T07:29:05.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.227e-06 2024-09-17T07:29:05.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320988+0000", 2024-09-17T07:29:05.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0219999999999999e-06 2024-09-17T07:29:05.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320993+0000", 2024-09-17T07:29:05.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6859999999999997e-06 2024-09-17T07:29:05.396 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.396 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.396 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.397 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20021 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.397 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262484+0000", 2024-09-17T07:29:05.397 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.064052415000000001, 2024-09-17T07:29:05.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.064529988999999996, 2024-09-17T07:29:05.397 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.397 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.397 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.397 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.397 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.397 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.397 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20021 2024-09-17T07:29:05.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.397 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262484+0000", 2024-09-17T07:29:05.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262484+0000", 2024-09-17T07:29:05.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262485+0000", 2024-09-17T07:29:05.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.23e-07 2024-09-17T07:29:05.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262488+0000", 2024-09-17T07:29:05.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8099999999999999e-06 2024-09-17T07:29:05.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262496+0000", 2024-09-17T07:29:05.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8939999999999993e-06 2024-09-17T07:29:05.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262501+0000", 2024-09-17T07:29:05.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8080000000000004e-06 2024-09-17T07:29:05.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.400 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.400 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.400 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20098 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.400 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.279615+0000", 2024-09-17T07:29:05.400 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.046921523, 2024-09-17T07:29:05.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047422833999999997, 2024-09-17T07:29:05.400 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.400 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.400 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.401 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20098 2024-09-17T07:29:05.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.401 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279615+0000", 2024-09-17T07:29:05.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279615+0000", 2024-09-17T07:29:05.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279615+0000", 2024-09-17T07:29:05.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6e-07 2024-09-17T07:29:05.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279616+0000", 2024-09-17T07:29:05.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.164e-06 2024-09-17T07:29:05.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279625+0000", 2024-09-17T07:29:05.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4470000000000006e-06 2024-09-17T07:29:05.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279628+0000", 2024-09-17T07:29:05.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0639999999999998e-06 2024-09-17T07:29:05.403 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.403 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.403 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.403 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20174 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.403 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.297707+0000", 2024-09-17T07:29:05.403 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028829298, 2024-09-17T07:29:05.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029354838000000001, 2024-09-17T07:29:05.403 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.404 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.404 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.404 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.404 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20174 2024-09-17T07:29:05.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.404 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297707+0000", 2024-09-17T07:29:05.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297707+0000", 2024-09-17T07:29:05.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297707+0000", 2024-09-17T07:29:05.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9200000000000002e-07 2024-09-17T07:29:05.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297709+0000", 2024-09-17T07:29:05.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0590000000000001e-06 2024-09-17T07:29:05.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297717+0000", 2024-09-17T07:29:05.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4669999999999998e-06 2024-09-17T07:29:05.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297723+0000", 2024-09-17T07:29:05.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1489999999999999e-06 2024-09-17T07:29:05.406 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.406 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.406 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.406 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20250 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.406 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.321007+0000", 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0055294690000000004, 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060788639999999998, 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20250 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321007+0000", 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321007+0000", 2024-09-17T07:29:05.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321007+0000", 2024-09-17T07:29:05.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9799999999999999e-07 2024-09-17T07:29:05.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321009+0000", 2024-09-17T07:29:05.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5999999999999999e-06 2024-09-17T07:29:05.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321013+0000", 2024-09-17T07:29:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5789999999999997e-06 2024-09-17T07:29:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321016+0000", 2024-09-17T07:29:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9660000000000001e-06 2024-09-17T07:29:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20022 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262511+0000", 2024-09-17T07:29:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.064025429999999994, 2024-09-17T07:29:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.064598532, 2024-09-17T07:29:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20022 2024-09-17T07:29:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262511+0000", 2024-09-17T07:29:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262511+0000", 2024-09-17T07:29:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262511+0000", 2024-09-17T07:29:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8500000000000002e-07 2024-09-17T07:29:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262512+0000", 2024-09-17T07:29:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3239999999999999e-06 2024-09-17T07:29:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262518+0000", 2024-09-17T07:29:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:29:05.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262522+0000", 2024-09-17T07:29:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.185e-06 2024-09-17T07:29:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20099 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.279632+0000", 2024-09-17T07:29:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.046904513000000002, 2024-09-17T07:29:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047503072, 2024-09-17T07:29:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.414 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20099 2024-09-17T07:29:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279632+0000", 2024-09-17T07:29:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279632+0000", 2024-09-17T07:29:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279632+0000", 2024-09-17T07:29:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.41e-07 2024-09-17T07:29:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279633+0000", 2024-09-17T07:29:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.513e-06 2024-09-17T07:29:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279644+0000", 2024-09-17T07:29:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0125999999999999e-05 2024-09-17T07:29:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279650+0000", 2024-09-17T07:29:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5339999999999996e-06 2024-09-17T07:29:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20175 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.417 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.297737+0000", 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028799657999999999, 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029422291, 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20175 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297737+0000", 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297737+0000", 2024-09-17T07:29:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297737+0000", 2024-09-17T07:29:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-17T07:29:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297748+0000", 2024-09-17T07:29:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0522e-05 2024-09-17T07:29:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297756+0000", 2024-09-17T07:29:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0790000000000001e-06 2024-09-17T07:29:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297761+0000", 2024-09-17T07:29:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000003e-06 2024-09-17T07:29:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.420 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20251 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.321020+0000", 2024-09-17T07:29:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0055167460000000003, 2024-09-17T07:29:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0061629279999999998, 2024-09-17T07:29:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20251 2024-09-17T07:29:05.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321020+0000", 2024-09-17T07:29:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321020+0000", 2024-09-17T07:29:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321020+0000", 2024-09-17T07:29:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321021+0000", 2024-09-17T07:29:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7100000000000011e-07 2024-09-17T07:29:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321024+0000", 2024-09-17T07:29:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.083e-06 2024-09-17T07:29:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321027+0000", 2024-09-17T07:29:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4699999999999998e-06 2024-09-17T07:29:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20023 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262526+0000", 2024-09-17T07:29:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.064010195000000006, 2024-09-17T07:29:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.064680221999999996, 2024-09-17T07:29:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.424 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20023 2024-09-17T07:29:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262526+0000", 2024-09-17T07:29:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262526+0000", 2024-09-17T07:29:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262526+0000", 2024-09-17T07:29:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262528+0000", 2024-09-17T07:29:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2580000000000001e-06 2024-09-17T07:29:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262532+0000", 2024-09-17T07:29:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3059999999999999e-06 2024-09-17T07:29:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262535+0000", 2024-09-17T07:29:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7140000000000002e-06 2024-09-17T07:29:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20100 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.279657+0000", 2024-09-17T07:29:05.427 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.046878853999999998, 2024-09-17T07:29:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047572634000000003, 2024-09-17T07:29:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20100 2024-09-17T07:29:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279657+0000", 2024-09-17T07:29:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279657+0000", 2024-09-17T07:29:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279658+0000", 2024-09-17T07:29:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2600000000000002e-07 2024-09-17T07:29:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279660+0000", 2024-09-17T07:29:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9989999999999998e-06 2024-09-17T07:29:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279666+0000", 2024-09-17T07:29:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:29:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279670+0000", 2024-09-17T07:29:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4850000000000003e-06 2024-09-17T07:29:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.430 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20176 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.297770+0000", 2024-09-17T07:29:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028766080999999999, 2024-09-17T07:29:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029483338000000001, 2024-09-17T07:29:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20176 2024-09-17T07:29:05.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297770+0000", 2024-09-17T07:29:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297770+0000", 2024-09-17T07:29:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297771+0000", 2024-09-17T07:29:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.89e-07 2024-09-17T07:29:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297773+0000", 2024-09-17T07:29:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0999999999999998e-06 2024-09-17T07:29:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297780+0000", 2024-09-17T07:29:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2030000000000003e-06 2024-09-17T07:29:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297786+0000", 2024-09-17T07:29:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3779999999999998e-06 2024-09-17T07:29:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20252 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.321032+0000", 2024-09-17T07:29:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0055040480000000001, 2024-09-17T07:29:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062447270000000003, 2024-09-17T07:29:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20252 2024-09-17T07:29:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321032+0000", 2024-09-17T07:29:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321032+0000", 2024-09-17T07:29:05.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321033+0000", 2024-09-17T07:29:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.46e-07 2024-09-17T07:29:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321034+0000", 2024-09-17T07:29:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.544e-06 2024-09-17T07:29:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321040+0000", 2024-09-17T07:29:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.19e-06 2024-09-17T07:29:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.321045+0000", 2024-09-17T07:29:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:29:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20024 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262613+0000", 2024-09-17T07:29:05.437 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.063923477000000006, 2024-09-17T07:29:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.064687857000000001, 2024-09-17T07:29:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20024 2024-09-17T07:29:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262613+0000", 2024-09-17T07:29:05.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262613+0000", 2024-09-17T07:29:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262613+0000", 2024-09-17T07:29:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1600000000000001e-07 2024-09-17T07:29:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262616+0000", 2024-09-17T07:29:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7690000000000001e-06 2024-09-17T07:29:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262622+0000", 2024-09-17T07:29:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1859999999999997e-06 2024-09-17T07:29:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262628+0000", 2024-09-17T07:29:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3859999999999998e-06 2024-09-17T07:29:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.440 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20101 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.279676+0000", 2024-09-17T07:29:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.046860201999999997, 2024-09-17T07:29:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047648612, 2024-09-17T07:29:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.441 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20101 2024-09-17T07:29:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279676+0000", 2024-09-17T07:29:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279676+0000", 2024-09-17T07:29:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279676+0000", 2024-09-17T07:29:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-09-17T07:29:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279678+0000", 2024-09-17T07:29:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9300000000000002e-06 2024-09-17T07:29:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279688+0000", 2024-09-17T07:29:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4520000000000003e-06 2024-09-17T07:29:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279697+0000", 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7809999999999994e-06 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20177 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.297794+0000", 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028742113, 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029554736000000002, 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20177 2024-09-17T07:29:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297794+0000", 2024-09-17T07:29:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297794+0000", 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297795+0000", 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297796+0000", 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.857e-06 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297804+0000", 2024-09-17T07:29:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4279999999999999e-06 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297810+0000", 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.161e-06 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20025 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262638+0000", 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.063898185999999996, 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.064741075999999995, 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20025 2024-09-17T07:29:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262638+0000", 2024-09-17T07:29:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262638+0000", 2024-09-17T07:29:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262638+0000", 2024-09-17T07:29:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:29:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262640+0000", 2024-09-17T07:29:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0040000000000002e-06 2024-09-17T07:29:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262646+0000", 2024-09-17T07:29:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7250000000000002e-06 2024-09-17T07:29:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262653+0000", 2024-09-17T07:29:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.567e-06 2024-09-17T07:29:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20102 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.279711+0000", 2024-09-17T07:29:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.046825701999999997, 2024-09-17T07:29:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047750318, 2024-09-17T07:29:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20102 2024-09-17T07:29:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279711+0000", 2024-09-17T07:29:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279711+0000", 2024-09-17T07:29:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279711+0000", 2024-09-17T07:29:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:29:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279713+0000", 2024-09-17T07:29:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:29:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279720+0000", 2024-09-17T07:29:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5289999999999997e-06 2024-09-17T07:29:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279726+0000", 2024-09-17T07:29:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4609999999999996e-06 2024-09-17T07:29:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20178 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.297818+0000", 2024-09-17T07:29:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028718413000000002, 2024-09-17T07:29:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029691693000000002, 2024-09-17T07:29:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20178 2024-09-17T07:29:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297818+0000", 2024-09-17T07:29:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297818+0000", 2024-09-17T07:29:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297818+0000", 2024-09-17T07:29:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5400000000000002e-07 2024-09-17T07:29:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297820+0000", 2024-09-17T07:29:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9879999999999999e-06 2024-09-17T07:29:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297828+0000", 2024-09-17T07:29:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6279999999999996e-06 2024-09-17T07:29:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297834+0000", 2024-09-17T07:29:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4760000000000003e-06 2024-09-17T07:29:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20273 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.324464+0000", 2024-09-17T07:29:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0020724820000000001, 2024-09-17T07:29:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030902949999999998, 2024-09-17T07:29:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20273 2024-09-17T07:29:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324464+0000", 2024-09-17T07:29:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324464+0000", 2024-09-17T07:29:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324464+0000", 2024-09-17T07:29:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0800000000000004e-07 2024-09-17T07:29:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324467+0000", 2024-09-17T07:29:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.056e-06 2024-09-17T07:29:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324475+0000", 2024-09-17T07:29:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0229999999999992e-06 2024-09-17T07:29:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324480+0000", 2024-09-17T07:29:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5369999999999999e-06 2024-09-17T07:29:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20026 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262659+0000", 2024-09-17T07:29:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.063876951000000001, 2024-09-17T07:29:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.064928992000000005, 2024-09-17T07:29:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20026 2024-09-17T07:29:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262659+0000", 2024-09-17T07:29:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262659+0000", 2024-09-17T07:29:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262660+0000", 2024-09-17T07:29:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:29:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262662+0000", 2024-09-17T07:29:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9810000000000002e-06 2024-09-17T07:29:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262673+0000", 2024-09-17T07:29:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1251e-05 2024-09-17T07:29:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262680+0000", 2024-09-17T07:29:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7100000000000001e-06 2024-09-17T07:29:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20103 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.279739+0000", 2024-09-17T07:29:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.046797685999999998, 2024-09-17T07:29:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047875083999999998, 2024-09-17T07:29:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20103 2024-09-17T07:29:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279739+0000", 2024-09-17T07:29:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279739+0000", 2024-09-17T07:29:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279739+0000", 2024-09-17T07:29:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4299999999999998e-07 2024-09-17T07:29:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279742+0000", 2024-09-17T07:29:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3019999999999999e-06 2024-09-17T07:29:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279748+0000", 2024-09-17T07:29:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1199999999999999e-06 2024-09-17T07:29:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279754+0000", 2024-09-17T07:29:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2789999999999998e-06 2024-09-17T07:29:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20179 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.297842+0000", 2024-09-17T07:29:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028693947000000001, 2024-09-17T07:29:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029797035999999999, 2024-09-17T07:29:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20179 2024-09-17T07:29:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297842+0000", 2024-09-17T07:29:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297842+0000", 2024-09-17T07:29:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297843+0000", 2024-09-17T07:29:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:29:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297845+0000", 2024-09-17T07:29:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1849999999999998e-06 2024-09-17T07:29:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297852+0000", 2024-09-17T07:29:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6190000000000002e-06 2024-09-17T07:29:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297859+0000", 2024-09-17T07:29:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4080000000000003e-06 2024-09-17T07:29:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20274 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.324489+0000", 2024-09-17T07:29:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.002047021, 2024-09-17T07:29:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031837089999999998, 2024-09-17T07:29:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20274 2024-09-17T07:29:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324489+0000", 2024-09-17T07:29:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324489+0000", 2024-09-17T07:29:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324490+0000", 2024-09-17T07:29:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9799999999999999e-07 2024-09-17T07:29:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324491+0000", 2024-09-17T07:29:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3549999999999999e-06 2024-09-17T07:29:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324495+0000", 2024-09-17T07:29:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.476e-06 2024-09-17T07:29:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324498+0000", 2024-09-17T07:29:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8009999999999999e-06 2024-09-17T07:29:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20027 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262687+0000", 2024-09-17T07:29:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.063849047000000006, 2024-09-17T07:29:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.065015795000000001, 2024-09-17T07:29:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20027 2024-09-17T07:29:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262687+0000", 2024-09-17T07:29:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262687+0000", 2024-09-17T07:29:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262688+0000", 2024-09-17T07:29:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:29:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262690+0000", 2024-09-17T07:29:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2730000000000001e-06 2024-09-17T07:29:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262696+0000", 2024-09-17T07:29:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9379999999999997e-06 2024-09-17T07:29:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262701+0000", 2024-09-17T07:29:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.775e-06 2024-09-17T07:29:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20104 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.279766+0000", 2024-09-17T07:29:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.046770752999999998, 2024-09-17T07:29:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047977022000000001, 2024-09-17T07:29:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20104 2024-09-17T07:29:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279766+0000", 2024-09-17T07:29:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279766+0000", 2024-09-17T07:29:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279766+0000", 2024-09-17T07:29:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-07 2024-09-17T07:29:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279768+0000", 2024-09-17T07:29:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1339999999999998e-06 2024-09-17T07:29:05.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279774+0000", 2024-09-17T07:29:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1190000000000002e-06 2024-09-17T07:29:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279782+0000", 2024-09-17T07:29:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6790000000000005e-06 2024-09-17T07:29:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20275 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.324502+0000", 2024-09-17T07:29:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0020341640000000002, 2024-09-17T07:29:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0032874760000000001, 2024-09-17T07:29:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20275 2024-09-17T07:29:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324502+0000", 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324502+0000", 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324502+0000", 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324503+0000", 2024-09-17T07:29:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.035e-06 2024-09-17T07:29:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324507+0000", 2024-09-17T07:29:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1200000000000004e-06 2024-09-17T07:29:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324510+0000", 2024-09-17T07:29:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5679999999999998e-06 2024-09-17T07:29:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20028 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262709+0000", 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.063827526999999995, 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.065123756000000005, 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20028 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262709+0000", 2024-09-17T07:29:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262709+0000", 2024-09-17T07:29:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262709+0000", 2024-09-17T07:29:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0200000000000003e-07 2024-09-17T07:29:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262711+0000", 2024-09-17T07:29:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0190000000000001e-06 2024-09-17T07:29:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262718+0000", 2024-09-17T07:29:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1409999999999999e-06 2024-09-17T07:29:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262723+0000", 2024-09-17T07:29:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9629999999999997e-06 2024-09-17T07:29:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20105 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.279794+0000", 2024-09-17T07:29:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.046742116, 2024-09-17T07:29:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048082662999999998, 2024-09-17T07:29:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20105 2024-09-17T07:29:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279794+0000", 2024-09-17T07:29:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279794+0000", 2024-09-17T07:29:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279795+0000", 2024-09-17T07:29:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.89e-07 2024-09-17T07:29:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279797+0000", 2024-09-17T07:29:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2120000000000002e-06 2024-09-17T07:29:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279805+0000", 2024-09-17T07:29:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3899999999999993e-06 2024-09-17T07:29:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279812+0000", 2024-09-17T07:29:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8210000000000004e-06 2024-09-17T07:29:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20200 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.303909+0000", 2024-09-17T07:29:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022626932999999998, 2024-09-17T07:29:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.02401354, 2024-09-17T07:29:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20200 2024-09-17T07:29:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303909+0000", 2024-09-17T07:29:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303909+0000", 2024-09-17T07:29:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303910+0000", 2024-09-17T07:29:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1600000000000001e-07 2024-09-17T07:29:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303914+0000", 2024-09-17T07:29:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8580000000000002e-06 2024-09-17T07:29:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303922+0000", 2024-09-17T07:29:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0509999999999997e-06 2024-09-17T07:29:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303927+0000", 2024-09-17T07:29:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:29:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20276 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.324513+0000", 2024-09-17T07:29:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0020227829999999998, 2024-09-17T07:29:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034582620000000001, 2024-09-17T07:29:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20276 2024-09-17T07:29:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324513+0000", 2024-09-17T07:29:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324513+0000", 2024-09-17T07:29:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324514+0000", 2024-09-17T07:29:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4999999999999999e-07 2024-09-17T07:29:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324515+0000", 2024-09-17T07:29:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9900000000000009e-07 2024-09-17T07:29:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324519+0000", 2024-09-17T07:29:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9480000000000001e-06 2024-09-17T07:29:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324521+0000", 2024-09-17T07:29:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5749999999999999e-06 2024-09-17T07:29:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20029 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.063806247999999996, 2024-09-17T07:29:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.065274141999999993, 2024-09-17T07:29:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20029 2024-09-17T07:29:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262735+0000", 2024-09-17T07:29:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.955e-06 2024-09-17T07:29:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262737+0000", 2024-09-17T07:29:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0090000000000002e-06 2024-09-17T07:29:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262743+0000", 2024-09-17T07:29:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2890000000000003e-06 2024-09-17T07:29:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262748+0000", 2024-09-17T07:29:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20106 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.279821+0000", 2024-09-17T07:29:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.046715472000000001, 2024-09-17T07:29:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048209084999999999, 2024-09-17T07:29:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20106 2024-09-17T07:29:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279821+0000", 2024-09-17T07:29:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279821+0000", 2024-09-17T07:29:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279821+0000", 2024-09-17T07:29:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4400000000000001e-07 2024-09-17T07:29:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279824+0000", 2024-09-17T07:29:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3199999999999998e-06 2024-09-17T07:29:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279830+0000", 2024-09-17T07:29:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8549999999999996e-06 2024-09-17T07:29:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279833+0000", 2024-09-17T07:29:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0350000000000002e-06 2024-09-17T07:29:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20201 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.303937+0000", 2024-09-17T07:29:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022599013000000001, 2024-09-17T07:29:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024116366, 2024-09-17T07:29:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20201 2024-09-17T07:29:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303937+0000", 2024-09-17T07:29:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303937+0000", 2024-09-17T07:29:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303938+0000", 2024-09-17T07:29:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8599999999999999e-07 2024-09-17T07:29:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303939+0000", 2024-09-17T07:29:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4759999999999999e-06 2024-09-17T07:29:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303944+0000", 2024-09-17T07:29:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5800000000000002e-06 2024-09-17T07:29:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303947+0000", 2024-09-17T07:29:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:29:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20277 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.324524+0000", 2024-09-17T07:29:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0020117619999999998, 2024-09-17T07:29:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035569550000000001, 2024-09-17T07:29:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20277 2024-09-17T07:29:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324524+0000", 2024-09-17T07:29:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324524+0000", 2024-09-17T07:29:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324525+0000", 2024-09-17T07:29:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:29:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324536+0000", 2024-09-17T07:29:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1174e-05 2024-09-17T07:29:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324541+0000", 2024-09-17T07:29:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1120000000000004e-06 2024-09-17T07:29:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324544+0000", 2024-09-17T07:29:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9849999999999998e-06 2024-09-17T07:29:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.063775076999999999, 2024-09-17T07:29:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.065348692999999999, 2024-09-17T07:29:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:29:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:29:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:29:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:29:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:29:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:29:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:29:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:29:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:29:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20107 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.279838+0000", 2024-09-17T07:29:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.046697807000000001, 2024-09-17T07:29:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048295381999999998, 2024-09-17T07:29:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20107 2024-09-17T07:29:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279838+0000", 2024-09-17T07:29:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279838+0000", 2024-09-17T07:29:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279839+0000", 2024-09-17T07:29:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6899999999999999e-07 2024-09-17T07:29:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279840+0000", 2024-09-17T07:29:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2640000000000001e-06 2024-09-17T07:29:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279844+0000", 2024-09-17T07:29:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1169999999999997e-06 2024-09-17T07:29:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.279847+0000", 2024-09-17T07:29:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8049999999999998e-06 2024-09-17T07:29:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20202 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.303951+0000", 2024-09-17T07:29:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022585612000000001, 2024-09-17T07:29:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024217796, 2024-09-17T07:29:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20202 2024-09-17T07:29:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303951+0000", 2024-09-17T07:29:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303951+0000", 2024-09-17T07:29:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303951+0000", 2024-09-17T07:29:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.04e-07 2024-09-17T07:29:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303952+0000", 2024-09-17T07:29:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1000000000000001e-06 2024-09-17T07:29:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303955+0000", 2024-09-17T07:29:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0570000000000001e-06 2024-09-17T07:29:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303958+0000", 2024-09-17T07:29:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6070000000000002e-06 2024-09-17T07:29:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20278 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.324549+0000", 2024-09-17T07:29:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.00198745, 2024-09-17T07:29:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036626530000000001, 2024-09-17T07:29:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20278 2024-09-17T07:29:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324549+0000", 2024-09-17T07:29:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324549+0000", 2024-09-17T07:29:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324549+0000", 2024-09-17T07:29:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6e-07 2024-09-17T07:29:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324550+0000", 2024-09-17T07:29:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.049e-06 2024-09-17T07:29:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324553+0000", 2024-09-17T07:29:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-06 2024-09-17T07:29:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324556+0000", 2024-09-17T07:29:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.441e-06 2024-09-17T07:29:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.063753365000000006, 2024-09-17T07:29:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.065474362999999994, 2024-09-17T07:29:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:29:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:29:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:29:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:29:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:29:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:29:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:29:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:29:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20130 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.287704+0000", 2024-09-17T07:29:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.038832066999999998, 2024-09-17T07:29:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040599568000000003, 2024-09-17T07:29:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20130 2024-09-17T07:29:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287704+0000", 2024-09-17T07:29:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287704+0000", 2024-09-17T07:29:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287706+0000", 2024-09-17T07:29:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.423e-06 2024-09-17T07:29:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287710+0000", 2024-09-17T07:29:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599999999999998e-06 2024-09-17T07:29:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287719+0000", 2024-09-17T07:29:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1600000000000004e-06 2024-09-17T07:29:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287739+0000", 2024-09-17T07:29:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0287000000000001e-05 2024-09-17T07:29:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20203 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.303961+0000", 2024-09-17T07:29:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022575477, 2024-09-17T07:29:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024391427, 2024-09-17T07:29:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20203 2024-09-17T07:29:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303961+0000", 2024-09-17T07:29:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303961+0000", 2024-09-17T07:29:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303961+0000", 2024-09-17T07:29:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2000000000000001e-07 2024-09-17T07:29:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303962+0000", 2024-09-17T07:29:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0500000000000002e-07 2024-09-17T07:29:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303966+0000", 2024-09-17T07:29:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.754e-06 2024-09-17T07:29:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303968+0000", 2024-09-17T07:29:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.525e-06 2024-09-17T07:29:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20279 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.324559+0000", 2024-09-17T07:29:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.001977281, 2024-09-17T07:29:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003839622, 2024-09-17T07:29:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20279 2024-09-17T07:29:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324559+0000", 2024-09-17T07:29:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324559+0000", 2024-09-17T07:29:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324559+0000", 2024-09-17T07:29:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:29:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324561+0000", 2024-09-17T07:29:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:29:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324564+0000", 2024-09-17T07:29:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1590000000000002e-06 2024-09-17T07:29:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324566+0000", 2024-09-17T07:29:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4420000000000001e-06 2024-09-17T07:29:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.063741166000000002, 2024-09-17T07:29:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.065638786000000005, 2024-09-17T07:29:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:29:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:29:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:29:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:29:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:29:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:29:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:29:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20131 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.287754+0000", 2024-09-17T07:29:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.038782364, 2024-09-17T07:29:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040704007, 2024-09-17T07:29:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20131 2024-09-17T07:29:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287754+0000", 2024-09-17T07:29:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287754+0000", 2024-09-17T07:29:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287754+0000", 2024-09-17T07:29:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3599999999999999e-07 2024-09-17T07:29:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287756+0000", 2024-09-17T07:29:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9069999999999999e-06 2024-09-17T07:29:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287761+0000", 2024-09-17T07:29:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5029999999999999e-06 2024-09-17T07:29:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287764+0000", 2024-09-17T07:29:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2619999999999999e-06 2024-09-17T07:29:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20204 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.303971+0000", 2024-09-17T07:29:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022565038999999999, 2024-09-17T07:29:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024511918000000001, 2024-09-17T07:29:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20204 2024-09-17T07:29:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303971+0000", 2024-09-17T07:29:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303971+0000", 2024-09-17T07:29:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303971+0000", 2024-09-17T07:29:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.05e-07 2024-09-17T07:29:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303972+0000", 2024-09-17T07:29:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9400000000000004e-07 2024-09-17T07:29:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303976+0000", 2024-09-17T07:29:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7419999999999999e-06 2024-09-17T07:29:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303979+0000", 2024-09-17T07:29:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.559e-06 2024-09-17T07:29:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20280 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.324569+0000", 2024-09-17T07:29:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0019668929999999999, 2024-09-17T07:29:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0039375579999999999, 2024-09-17T07:29:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20280 2024-09-17T07:29:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324569+0000", 2024-09-17T07:29:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324569+0000", 2024-09-17T07:29:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324570+0000", 2024-09-17T07:29:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2000000000000001e-07 2024-09-17T07:29:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324571+0000", 2024-09-17T07:29:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6599999999999994e-07 2024-09-17T07:29:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324574+0000", 2024-09-17T07:29:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0070000000000002e-06 2024-09-17T07:29:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324576+0000", 2024-09-17T07:29:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4679999999999999e-06 2024-09-17T07:29:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.063730307999999999, 2024-09-17T07:29:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.065746886000000004, 2024-09-17T07:29:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:29:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:29:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:29:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:29:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:29:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:29:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:29:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:29:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20132 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.287768+0000", 2024-09-17T07:29:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.038768097000000001, 2024-09-17T07:29:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.040831043999999997, 2024-09-17T07:29:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20132 2024-09-17T07:29:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287768+0000", 2024-09-17T07:29:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287768+0000", 2024-09-17T07:29:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287768+0000", 2024-09-17T07:29:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8500000000000002e-07 2024-09-17T07:29:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287770+0000", 2024-09-17T07:29:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.164e-06 2024-09-17T07:29:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287773+0000", 2024-09-17T07:29:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2799999999999999e-06 2024-09-17T07:29:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287776+0000", 2024-09-17T07:29:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8490000000000002e-06 2024-09-17T07:29:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20205 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.303982+0000", 2024-09-17T07:29:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022554478999999999, 2024-09-17T07:29:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024664266000000001, 2024-09-17T07:29:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20205 2024-09-17T07:29:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303982+0000", 2024-09-17T07:29:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303982+0000", 2024-09-17T07:29:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303982+0000", 2024-09-17T07:29:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303983+0000", 2024-09-17T07:29:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0920000000000001e-06 2024-09-17T07:29:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303987+0000", 2024-09-17T07:29:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8399999999999997e-06 2024-09-17T07:29:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303990+0000", 2024-09-17T07:29:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8720000000000002e-06 2024-09-17T07:29:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20281 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.324582+0000", 2024-09-17T07:29:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.001953906, 2024-09-17T07:29:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041093509999999998, 2024-09-17T07:29:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20281 2024-09-17T07:29:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324582+0000", 2024-09-17T07:29:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324582+0000", 2024-09-17T07:29:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324583+0000", 2024-09-17T07:29:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7300000000000002e-07 2024-09-17T07:29:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324584+0000", 2024-09-17T07:29:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4160000000000001e-06 2024-09-17T07:29:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324588+0000", 2024-09-17T07:29:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3800000000000004e-06 2024-09-17T07:29:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324591+0000", 2024-09-17T07:29:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7530000000000002e-06 2024-09-17T07:29:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.063716047999999997, 2024-09-17T07:29:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.065906200999999998, 2024-09-17T07:29:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:29:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:29:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:29:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:29:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:29:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:29:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:29:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:29:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20133 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.287782+0000", 2024-09-17T07:29:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.038754019000000001, 2024-09-17T07:29:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041065865, 2024-09-17T07:29:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20133 2024-09-17T07:29:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287782+0000", 2024-09-17T07:29:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287782+0000", 2024-09-17T07:29:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287782+0000", 2024-09-17T07:29:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1899999999999999e-07 2024-09-17T07:29:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287784+0000", 2024-09-17T07:29:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.358e-06 2024-09-17T07:29:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287791+0000", 2024-09-17T07:29:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5360000000000004e-06 2024-09-17T07:29:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287794+0000", 2024-09-17T07:29:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.038e-06 2024-09-17T07:29:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20206 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.303993+0000", 2024-09-17T07:29:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022543497999999999, 2024-09-17T07:29:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024902179, 2024-09-17T07:29:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20206 2024-09-17T07:29:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303993+0000", 2024-09-17T07:29:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303993+0000", 2024-09-17T07:29:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303993+0000", 2024-09-17T07:29:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0800000000000001e-07 2024-09-17T07:29:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303994+0000", 2024-09-17T07:29:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.009e-06 2024-09-17T07:29:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.303997+0000", 2024-09-17T07:29:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0070000000000002e-06 2024-09-17T07:29:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304000+0000", 2024-09-17T07:29:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.734e-06 2024-09-17T07:29:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20282 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.324614+0000", 2024-09-17T07:29:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.001922163, 2024-09-17T07:29:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0043285290000000002, 2024-09-17T07:29:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20282 2024-09-17T07:29:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324614+0000", 2024-09-17T07:29:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324614+0000", 2024-09-17T07:29:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324615+0000", 2024-09-17T07:29:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4799999999999999e-07 2024-09-17T07:29:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324616+0000", 2024-09-17T07:29:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.398e-06 2024-09-17T07:29:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324619+0000", 2024-09-17T07:29:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.523e-06 2024-09-17T07:29:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324623+0000", 2024-09-17T07:29:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1209999999999998e-06 2024-09-17T07:29:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20057 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.272229+0000", 2024-09-17T07:29:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.054307218999999997, 2024-09-17T07:29:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.056757283999999998, 2024-09-17T07:29:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20057 2024-09-17T07:29:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272229+0000", 2024-09-17T07:29:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272229+0000", 2024-09-17T07:29:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272231+0000", 2024-09-17T07:29:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8330000000000001e-06 2024-09-17T07:29:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272236+0000", 2024-09-17T07:29:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:29:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272247+0000", 2024-09-17T07:29:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0784999999999999e-05 2024-09-17T07:29:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272253+0000", 2024-09-17T07:29:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9399999999999999e-06 2024-09-17T07:29:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20134 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.287798+0000", 2024-09-17T07:29:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.038738021999999997, 2024-09-17T07:29:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041236320999999999, 2024-09-17T07:29:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20134 2024-09-17T07:29:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287798+0000", 2024-09-17T07:29:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287798+0000", 2024-09-17T07:29:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287798+0000", 2024-09-17T07:29:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.22e-07 2024-09-17T07:29:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287806+0000", 2024-09-17T07:29:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0639999999999996e-06 2024-09-17T07:29:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287809+0000", 2024-09-17T07:29:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.698e-06 2024-09-17T07:29:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287814+0000", 2024-09-17T07:29:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5199999999999999e-06 2024-09-17T07:29:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20207 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.304003+0000", 2024-09-17T07:29:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022533541000000001, 2024-09-17T07:29:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025077450000000001, 2024-09-17T07:29:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20207 2024-09-17T07:29:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304003+0000", 2024-09-17T07:29:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304003+0000", 2024-09-17T07:29:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304003+0000", 2024-09-17T07:29:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0900000000000001e-07 2024-09-17T07:29:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304004+0000", 2024-09-17T07:29:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4300000000000001e-07 2024-09-17T07:29:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304008+0000", 2024-09-17T07:29:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6449999999999998e-06 2024-09-17T07:29:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304010+0000", 2024-09-17T07:29:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7700000000000002e-06 2024-09-17T07:29:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20283 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.324626+0000", 2024-09-17T07:29:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.001910082, 2024-09-17T07:29:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045011399999999998, 2024-09-17T07:29:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20283 2024-09-17T07:29:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324626+0000", 2024-09-17T07:29:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324626+0000", 2024-09-17T07:29:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324626+0000", 2024-09-17T07:29:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0800000000000001e-07 2024-09-17T07:29:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324627+0000", 2024-09-17T07:29:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7000000000000003e-07 2024-09-17T07:29:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324630+0000", 2024-09-17T07:29:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9969999999999999e-06 2024-09-17T07:29:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324633+0000", 2024-09-17T07:29:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6809999999999998e-06 2024-09-17T07:29:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20059 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.272270+0000", 2024-09-17T07:29:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.054266657000000003, 2024-09-17T07:29:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.056903269999999999, 2024-09-17T07:29:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20059 2024-09-17T07:29:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272270+0000", 2024-09-17T07:29:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272270+0000", 2024-09-17T07:29:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272270+0000", 2024-09-17T07:29:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:29:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272272+0000", 2024-09-17T07:29:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7129999999999999e-06 2024-09-17T07:29:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272277+0000", 2024-09-17T07:29:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2569999999999998e-06 2024-09-17T07:29:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272280+0000", 2024-09-17T07:29:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4790000000000001e-06 2024-09-17T07:29:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20135 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.287818+0000", 2024-09-17T07:29:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.038718491000000001, 2024-09-17T07:29:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041402912, 2024-09-17T07:29:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20135 2024-09-17T07:29:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287818+0000", 2024-09-17T07:29:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287818+0000", 2024-09-17T07:29:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287818+0000", 2024-09-17T07:29:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6399999999999998e-07 2024-09-17T07:29:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287819+0000", 2024-09-17T07:29:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.046e-06 2024-09-17T07:29:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287822+0000", 2024-09-17T07:29:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1540000000000002e-06 2024-09-17T07:29:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287825+0000", 2024-09-17T07:29:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6060000000000001e-06 2024-09-17T07:29:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20208 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.304013+0000", 2024-09-17T07:29:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022523053000000001, 2024-09-17T07:29:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025252427000000001, 2024-09-17T07:29:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20208 2024-09-17T07:29:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304013+0000", 2024-09-17T07:29:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304013+0000", 2024-09-17T07:29:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304013+0000", 2024-09-17T07:29:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304014+0000", 2024-09-17T07:29:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2699999999999998e-07 2024-09-17T07:29:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304017+0000", 2024-09-17T07:29:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8569999999999999e-06 2024-09-17T07:29:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304020+0000", 2024-09-17T07:29:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.48e-06 2024-09-17T07:29:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20284 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.324636+0000", 2024-09-17T07:29:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.001900313, 2024-09-17T07:29:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.004676772, 2024-09-17T07:29:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20284 2024-09-17T07:29:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324636+0000", 2024-09-17T07:29:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324636+0000", 2024-09-17T07:29:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324636+0000", 2024-09-17T07:29:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8700000000000002e-07 2024-09-17T07:29:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324637+0000", 2024-09-17T07:29:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4499999999999995e-07 2024-09-17T07:29:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324640+0000", 2024-09-17T07:29:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9050000000000001e-06 2024-09-17T07:29:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324643+0000", 2024-09-17T07:29:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.508e-06 2024-09-17T07:29:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20060 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.272285+0000", 2024-09-17T07:29:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.054251501000000001, 2024-09-17T07:29:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.057073341, 2024-09-17T07:29:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20060 2024-09-17T07:29:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272285+0000", 2024-09-17T07:29:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272285+0000", 2024-09-17T07:29:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272285+0000", 2024-09-17T07:29:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2100000000000001e-07 2024-09-17T07:29:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272286+0000", 2024-09-17T07:29:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.15e-06 2024-09-17T07:29:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272289+0000", 2024-09-17T07:29:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3289999999999998e-06 2024-09-17T07:29:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272292+0000", 2024-09-17T07:29:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.734e-06 2024-09-17T07:29:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20136 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.287828+0000", 2024-09-17T07:29:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.038708336000000003, 2024-09-17T07:29:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041579285000000001, 2024-09-17T07:29:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20136 2024-09-17T07:29:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287828+0000", 2024-09-17T07:29:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287828+0000", 2024-09-17T07:29:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287828+0000", 2024-09-17T07:29:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0800000000000001e-07 2024-09-17T07:29:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287830+0000", 2024-09-17T07:29:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8500000000000001e-06 2024-09-17T07:29:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287833+0000", 2024-09-17T07:29:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0410000000000002e-06 2024-09-17T07:29:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287836+0000", 2024-09-17T07:29:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4379999999999999e-06 2024-09-17T07:29:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20209 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.304022+0000", 2024-09-17T07:29:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022513785000000001, 2024-09-17T07:29:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025430444, 2024-09-17T07:29:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20209 2024-09-17T07:29:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304022+0000", 2024-09-17T07:29:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304022+0000", 2024-09-17T07:29:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304023+0000", 2024-09-17T07:29:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304028+0000", 2024-09-17T07:29:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2689999999999999e-06 2024-09-17T07:29:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304031+0000", 2024-09-17T07:29:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4659999999999999e-06 2024-09-17T07:29:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304034+0000", 2024-09-17T07:29:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6570000000000001e-06 2024-09-17T07:29:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20285 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.324645+0000", 2024-09-17T07:29:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0018908449999999999, 2024-09-17T07:29:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0048556600000000004, 2024-09-17T07:29:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20285 2024-09-17T07:29:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324645+0000", 2024-09-17T07:29:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324645+0000", 2024-09-17T07:29:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324646+0000", 2024-09-17T07:29:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324646+0000", 2024-09-17T07:29:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4300000000000002e-07 2024-09-17T07:29:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324649+0000", 2024-09-17T07:29:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9960000000000002e-06 2024-09-17T07:29:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324652+0000", 2024-09-17T07:29:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3980000000000002e-06 2024-09-17T07:29:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20061 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.272296+0000", 2024-09-17T07:29:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.054240710999999997, 2024-09-17T07:29:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.057252440000000002, 2024-09-17T07:29:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20061 2024-09-17T07:29:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272296+0000", 2024-09-17T07:29:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272296+0000", 2024-09-17T07:29:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272296+0000", 2024-09-17T07:29:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272297+0000", 2024-09-17T07:29:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0330000000000001e-06 2024-09-17T07:29:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272300+0000", 2024-09-17T07:29:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1470000000000001e-06 2024-09-17T07:29:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272303+0000", 2024-09-17T07:29:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9210000000000001e-06 2024-09-17T07:29:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20137 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.287840+0000", 2024-09-17T07:29:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.038696673000000001, 2024-09-17T07:29:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041755805999999999, 2024-09-17T07:29:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20137 2024-09-17T07:29:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287840+0000", 2024-09-17T07:29:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287840+0000", 2024-09-17T07:29:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287840+0000", 2024-09-17T07:29:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.04e-07 2024-09-17T07:29:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287841+0000", 2024-09-17T07:29:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7300000000000004e-07 2024-09-17T07:29:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287844+0000", 2024-09-17T07:29:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1099999999999999e-06 2024-09-17T07:29:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287847+0000", 2024-09-17T07:29:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.734e-06 2024-09-17T07:29:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20210 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.304038+0000", 2024-09-17T07:29:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022498321000000002, 2024-09-17T07:29:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025597106000000001, 2024-09-17T07:29:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20210 2024-09-17T07:29:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304038+0000", 2024-09-17T07:29:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304038+0000", 2024-09-17T07:29:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304038+0000", 2024-09-17T07:29:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.36e-07 2024-09-17T07:29:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304044+0000", 2024-09-17T07:29:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0460000000000002e-06 2024-09-17T07:29:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304049+0000", 2024-09-17T07:29:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3780000000000003e-06 2024-09-17T07:29:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304051+0000", 2024-09-17T07:29:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7609999999999999e-06 2024-09-17T07:29:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20062 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.272306+0000", 2024-09-17T07:29:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.054230440999999997, 2024-09-17T07:29:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.057356413000000002, 2024-09-17T07:29:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20062 2024-09-17T07:29:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272306+0000", 2024-09-17T07:29:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272306+0000", 2024-09-17T07:29:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272306+0000", 2024-09-17T07:29:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2100000000000001e-07 2024-09-17T07:29:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272316+0000", 2024-09-17T07:29:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8619999999999998e-06 2024-09-17T07:29:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272323+0000", 2024-09-17T07:29:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.311e-06 2024-09-17T07:29:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272327+0000", 2024-09-17T07:29:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4209999999999999e-06 2024-09-17T07:29:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20138 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.287850+0000", 2024-09-17T07:29:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.038686679000000002, 2024-09-17T07:29:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041839903999999997, 2024-09-17T07:29:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20138 2024-09-17T07:29:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287850+0000", 2024-09-17T07:29:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287850+0000", 2024-09-17T07:29:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287850+0000", 2024-09-17T07:29:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2700000000000001e-07 2024-09-17T07:29:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287851+0000", 2024-09-17T07:29:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0599999999999999e-07 2024-09-17T07:29:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287854+0000", 2024-09-17T07:29:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9440000000000001e-06 2024-09-17T07:29:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287856+0000", 2024-09-17T07:29:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6010000000000002e-06 2024-09-17T07:29:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20211 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.304056+0000", 2024-09-17T07:29:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022479969999999998, 2024-09-17T07:29:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025656866, 2024-09-17T07:29:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20211 2024-09-17T07:29:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304056+0000", 2024-09-17T07:29:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304056+0000", 2024-09-17T07:29:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304057+0000", 2024-09-17T07:29:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5600000000000002e-07 2024-09-17T07:29:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304058+0000", 2024-09-17T07:29:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0729999999999999e-06 2024-09-17T07:29:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304062+0000", 2024-09-17T07:29:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0790000000000002e-06 2024-09-17T07:29:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304064+0000", 2024-09-17T07:29:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.571e-06 2024-09-17T07:29:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20063 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.272331+0000", 2024-09-17T07:29:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.054205359000000002, 2024-09-17T07:29:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.057418085000000001, 2024-09-17T07:29:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20063 2024-09-17T07:29:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272331+0000", 2024-09-17T07:29:05.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272331+0000", 2024-09-17T07:29:05.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272331+0000", 2024-09-17T07:29:05.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-17T07:29:05.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272332+0000", 2024-09-17T07:29:05.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2049999999999999e-06 2024-09-17T07:29:05.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272336+0000", 2024-09-17T07:29:05.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2160000000000002e-06 2024-09-17T07:29:05.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272338+0000", 2024-09-17T07:29:05.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4849999999999999e-06 2024-09-17T07:29:05.646 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.646 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.646 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.646 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20139 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.646 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.287859+0000", 2024-09-17T07:29:05.646 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.038677059999999999, 2024-09-17T07:29:05.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041914514999999999, 2024-09-17T07:29:05.646 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.646 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.646 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.647 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.647 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.647 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.647 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20139 2024-09-17T07:29:05.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.647 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287859+0000", 2024-09-17T07:29:05.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287859+0000", 2024-09-17T07:29:05.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287859+0000", 2024-09-17T07:29:05.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.22e-07 2024-09-17T07:29:05.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287860+0000", 2024-09-17T07:29:05.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.47e-07 2024-09-17T07:29:05.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287863+0000", 2024-09-17T07:29:05.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9550000000000001e-06 2024-09-17T07:29:05.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287866+0000", 2024-09-17T07:29:05.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5440000000000001e-06 2024-09-17T07:29:05.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.649 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.649 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20212 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.650 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.304067+0000", 2024-09-17T07:29:05.650 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022468993, 2024-09-17T07:29:05.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025732167, 2024-09-17T07:29:05.650 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.650 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.650 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.650 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.650 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.650 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.650 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20212 2024-09-17T07:29:05.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.650 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304067+0000", 2024-09-17T07:29:05.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304067+0000", 2024-09-17T07:29:05.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304067+0000", 2024-09-17T07:29:05.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.04e-07 2024-09-17T07:29:05.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304068+0000", 2024-09-17T07:29:05.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7300000000000004e-07 2024-09-17T07:29:05.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304072+0000", 2024-09-17T07:29:05.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.794e-06 2024-09-17T07:29:05.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304075+0000", 2024-09-17T07:29:05.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6579999999999998e-06 2024-09-17T07:29:05.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.653 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.653 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20064 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.653 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.272341+0000", 2024-09-17T07:29:05.653 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.054195108999999998, 2024-09-17T07:29:05.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.057482092999999998, 2024-09-17T07:29:05.653 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.653 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.653 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.654 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.654 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20064 2024-09-17T07:29:05.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.654 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272341+0000", 2024-09-17T07:29:05.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272341+0000", 2024-09-17T07:29:05.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272341+0000", 2024-09-17T07:29:05.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3699999999999999e-07 2024-09-17T07:29:05.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272342+0000", 2024-09-17T07:29:05.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:29:05.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272345+0000", 2024-09-17T07:29:05.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0110000000000001e-06 2024-09-17T07:29:05.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272348+0000", 2024-09-17T07:29:05.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6630000000000002e-06 2024-09-17T07:29:05.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.656 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.656 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20140 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.656 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.287869+0000", 2024-09-17T07:29:05.656 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.038667502999999999, 2024-09-17T07:29:05.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.041979453999999999, 2024-09-17T07:29:05.656 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.657 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.657 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.657 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20140 2024-09-17T07:29:05.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.657 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287869+0000", 2024-09-17T07:29:05.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287869+0000", 2024-09-17T07:29:05.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287869+0000", 2024-09-17T07:29:05.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:29:05.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287870+0000", 2024-09-17T07:29:05.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.015e-06 2024-09-17T07:29:05.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287873+0000", 2024-09-17T07:29:05.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9890000000000001e-06 2024-09-17T07:29:05.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287875+0000", 2024-09-17T07:29:05.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4530000000000001e-06 2024-09-17T07:29:05.659 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.659 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.659 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.660 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20213 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.660 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.304078+0000", 2024-09-17T07:29:05.660 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.022458513999999999, 2024-09-17T07:29:05.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025794577999999999, 2024-09-17T07:29:05.660 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.660 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.660 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.660 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.660 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.660 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.660 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20213 2024-09-17T07:29:05.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.660 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304078+0000", 2024-09-17T07:29:05.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304078+0000", 2024-09-17T07:29:05.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304078+0000", 2024-09-17T07:29:05.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:05.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304079+0000", 2024-09-17T07:29:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8100000000000001e-07 2024-09-17T07:29:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304082+0000", 2024-09-17T07:29:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5099999999999999e-06 2024-09-17T07:29:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.304085+0000", 2024-09-17T07:29:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9610000000000001e-06 2024-09-17T07:29:05.662 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20065 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.272369+0000", 2024-09-17T07:29:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.054167426999999997, 2024-09-17T07:29:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.057528568000000002, 2024-09-17T07:29:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.663 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20065 2024-09-17T07:29:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272369+0000", 2024-09-17T07:29:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272369+0000", 2024-09-17T07:29:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272369+0000", 2024-09-17T07:29:05.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4700000000000002e-07 2024-09-17T07:29:05.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272371+0000", 2024-09-17T07:29:05.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5379999999999999e-06 2024-09-17T07:29:05.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272374+0000", 2024-09-17T07:29:05.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.529e-06 2024-09-17T07:29:05.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272378+0000", 2024-09-17T07:29:05.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3340000000000002e-06 2024-09-17T07:29:05.666 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.666 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.666 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.666 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20141 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.666 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.287878+0000", 2024-09-17T07:29:05.666 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.038657879999999999, 2024-09-17T07:29:05.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042042748999999997, 2024-09-17T07:29:05.666 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.666 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.667 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.667 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20141 2024-09-17T07:29:05.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.667 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287878+0000", 2024-09-17T07:29:05.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287878+0000", 2024-09-17T07:29:05.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287879+0000", 2024-09-17T07:29:05.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:05.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287882+0000", 2024-09-17T07:29:05.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5140000000000002e-06 2024-09-17T07:29:05.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287885+0000", 2024-09-17T07:29:05.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8789999999999999e-06 2024-09-17T07:29:05.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287888+0000", 2024-09-17T07:29:05.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6019999999999998e-06 2024-09-17T07:29:05.669 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.669 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.669 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.669 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20239 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.670 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.320755+0000", 2024-09-17T07:29:05.670 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0057817470000000003, 2024-09-17T07:29:05.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0091917430000000005, 2024-09-17T07:29:05.670 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.670 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.670 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.670 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20239 2024-09-17T07:29:05.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.670 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320755+0000", 2024-09-17T07:29:05.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320755+0000", 2024-09-17T07:29:05.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320756+0000", 2024-09-17T07:29:05.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3790000000000001e-06 2024-09-17T07:29:05.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320762+0000", 2024-09-17T07:29:05.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7810000000000002e-06 2024-09-17T07:29:05.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320780+0000", 2024-09-17T07:29:05.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8372e-05 2024-09-17T07:29:05.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320789+0000", 2024-09-17T07:29:05.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.551e-06 2024-09-17T07:29:05.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.673 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.673 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20066 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.673 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.272381+0000", 2024-09-17T07:29:05.673 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.054155080000000001, 2024-09-17T07:29:05.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.057589030999999999, 2024-09-17T07:29:05.673 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.673 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.673 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.674 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20066 2024-09-17T07:29:05.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.674 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272381+0000", 2024-09-17T07:29:05.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272381+0000", 2024-09-17T07:29:05.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272381+0000", 2024-09-17T07:29:05.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:05.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272382+0000", 2024-09-17T07:29:05.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0860000000000001e-06 2024-09-17T07:29:05.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272385+0000", 2024-09-17T07:29:05.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9490000000000001e-06 2024-09-17T07:29:05.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272388+0000", 2024-09-17T07:29:05.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259999999999999e-06 2024-09-17T07:29:05.676 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.676 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.676 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.676 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20142 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.676 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.287891+0000", 2024-09-17T07:29:05.676 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.038645601000000002, 2024-09-17T07:29:05.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042103142000000003, 2024-09-17T07:29:05.677 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.677 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.677 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.677 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.677 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.677 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.677 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20142 2024-09-17T07:29:05.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.677 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287891+0000", 2024-09-17T07:29:05.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287891+0000", 2024-09-17T07:29:05.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287891+0000", 2024-09-17T07:29:05.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-07 2024-09-17T07:29:05.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287892+0000", 2024-09-17T07:29:05.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2800000000000005e-07 2024-09-17T07:29:05.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287896+0000", 2024-09-17T07:29:05.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3540000000000002e-06 2024-09-17T07:29:05.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287899+0000", 2024-09-17T07:29:05.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6299999999999998e-06 2024-09-17T07:29:05.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.679 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.680 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20240 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.680 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.320810+0000", 2024-09-17T07:29:05.680 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0057261669999999999, 2024-09-17T07:29:05.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0092086419999999995, 2024-09-17T07:29:05.680 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.680 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.680 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.680 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20240 2024-09-17T07:29:05.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.681 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320810+0000", 2024-09-17T07:29:05.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320810+0000", 2024-09-17T07:29:05.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320811+0000", 2024-09-17T07:29:05.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0299999999999999e-07 2024-09-17T07:29:05.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320814+0000", 2024-09-17T07:29:05.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0709999999999999e-06 2024-09-17T07:29:05.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320822+0000", 2024-09-17T07:29:05.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0600000000000008e-06 2024-09-17T07:29:05.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320828+0000", 2024-09-17T07:29:05.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7960000000000001e-06 2024-09-17T07:29:05.683 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.683 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.683 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.683 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20067 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.683 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.272391+0000", 2024-09-17T07:29:05.683 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.054145318999999997, 2024-09-17T07:29:05.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.057651314000000002, 2024-09-17T07:29:05.683 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.683 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.684 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.684 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.684 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.684 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.684 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20067 2024-09-17T07:29:05.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.684 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272391+0000", 2024-09-17T07:29:05.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272391+0000", 2024-09-17T07:29:05.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272391+0000", 2024-09-17T07:29:05.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1500000000000001e-07 2024-09-17T07:29:05.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272392+0000", 2024-09-17T07:29:05.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3600000000000002e-07 2024-09-17T07:29:05.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272395+0000", 2024-09-17T07:29:05.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9189999999999999e-06 2024-09-17T07:29:05.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272397+0000", 2024-09-17T07:29:05.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4760000000000001e-06 2024-09-17T07:29:05.686 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.686 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.686 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.687 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20143 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.687 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.287903+0000", 2024-09-17T07:29:05.687 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.038633013000000001, 2024-09-17T07:29:05.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042164460000000001, 2024-09-17T07:29:05.687 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.687 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.687 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.687 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20143 2024-09-17T07:29:05.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.687 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287903+0000", 2024-09-17T07:29:05.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287903+0000", 2024-09-17T07:29:05.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287903+0000", 2024-09-17T07:29:05.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:05.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287904+0000", 2024-09-17T07:29:05.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6099999999999999e-07 2024-09-17T07:29:05.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287908+0000", 2024-09-17T07:29:05.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0090000000000001e-06 2024-09-17T07:29:05.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.287911+0000", 2024-09-17T07:29:05.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5490000000000001e-06 2024-09-17T07:29:05.690 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.690 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.690 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.690 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20241 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.690 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.320835+0000", 2024-09-17T07:29:05.690 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0057014559999999997, 2024-09-17T07:29:05.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0092563909999999992, 2024-09-17T07:29:05.690 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.690 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.690 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20241 2024-09-17T07:29:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320835+0000", 2024-09-17T07:29:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320835+0000", 2024-09-17T07:29:05.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320835+0000", 2024-09-17T07:29:05.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-17T07:29:05.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320837+0000", 2024-09-17T07:29:05.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:29:05.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320844+0000", 2024-09-17T07:29:05.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1810000000000004e-06 2024-09-17T07:29:05.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320849+0000", 2024-09-17T07:29:05.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9930000000000003e-06 2024-09-17T07:29:05.693 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.693 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.693 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.693 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20068 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.693 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.272400+0000", 2024-09-17T07:29:05.693 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.054135853999999997, 2024-09-17T07:29:05.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.057715672000000003, 2024-09-17T07:29:05.694 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.694 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.694 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.694 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.694 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.694 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.694 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20068 2024-09-17T07:29:05.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.694 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272400+0000", 2024-09-17T07:29:05.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272400+0000", 2024-09-17T07:29:05.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272401+0000", 2024-09-17T07:29:05.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:05.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272402+0000", 2024-09-17T07:29:05.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3399999999999997e-07 2024-09-17T07:29:05.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272404+0000", 2024-09-17T07:29:05.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.92e-06 2024-09-17T07:29:05.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272407+0000", 2024-09-17T07:29:05.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5469999999999999e-06 2024-09-17T07:29:05.696 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.696 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.696 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.697 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20166 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.697 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.297476+0000", 2024-09-17T07:29:05.697 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.029060334, 2024-09-17T07:29:05.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032663874000000002, 2024-09-17T07:29:05.697 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.697 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.697 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.697 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.697 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.697 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.697 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20166 2024-09-17T07:29:05.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.697 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297476+0000", 2024-09-17T07:29:05.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297476+0000", 2024-09-17T07:29:05.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297477+0000", 2024-09-17T07:29:05.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.53e-06 2024-09-17T07:29:05.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297483+0000", 2024-09-17T07:29:05.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0170000000000002e-06 2024-09-17T07:29:05.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297499+0000", 2024-09-17T07:29:05.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5500000000000001e-05 2024-09-17T07:29:05.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297507+0000", 2024-09-17T07:29:05.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4160000000000004e-06 2024-09-17T07:29:05.700 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.700 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.700 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.700 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20242 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.700 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.320855+0000", 2024-09-17T07:29:05.700 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0056810369999999999, 2024-09-17T07:29:05.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0093096310000000005, 2024-09-17T07:29:05.700 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.701 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.701 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.701 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20242 2024-09-17T07:29:05.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.701 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320855+0000", 2024-09-17T07:29:05.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320855+0000", 2024-09-17T07:29:05.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320856+0000", 2024-09-17T07:29:05.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7399999999999999e-07 2024-09-17T07:29:05.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320857+0000", 2024-09-17T07:29:05.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.759e-06 2024-09-17T07:29:05.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320864+0000", 2024-09-17T07:29:05.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7660000000000001e-06 2024-09-17T07:29:05.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320869+0000", 2024-09-17T07:29:05.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9620000000000001e-06 2024-09-17T07:29:05.703 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.703 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.703 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.704 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20069 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.704 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.272410+0000", 2024-09-17T07:29:05.704 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.054126301000000002, 2024-09-17T07:29:05.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.057778637000000001, 2024-09-17T07:29:05.704 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.704 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.704 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.704 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20069 2024-09-17T07:29:05.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.704 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272410+0000", 2024-09-17T07:29:05.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272410+0000", 2024-09-17T07:29:05.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272410+0000", 2024-09-17T07:29:05.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:29:05.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272411+0000", 2024-09-17T07:29:05.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.003e-06 2024-09-17T07:29:05.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272414+0000", 2024-09-17T07:29:05.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8710000000000001e-06 2024-09-17T07:29:05.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272416+0000", 2024-09-17T07:29:05.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4480000000000001e-06 2024-09-17T07:29:05.707 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.707 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.707 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.707 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20167 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.707 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.297533+0000", 2024-09-17T07:29:05.707 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.029002856, 2024-09-17T07:29:05.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032679780999999998, 2024-09-17T07:29:05.707 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.707 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.707 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.708 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20167 2024-09-17T07:29:05.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.708 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297533+0000", 2024-09-17T07:29:05.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297533+0000", 2024-09-17T07:29:05.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297534+0000", 2024-09-17T07:29:05.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5199999999999996e-07 2024-09-17T07:29:05.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297537+0000", 2024-09-17T07:29:05.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.756e-06 2024-09-17T07:29:05.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297546+0000", 2024-09-17T07:29:05.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7900000000000005e-06 2024-09-17T07:29:05.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297553+0000", 2024-09-17T07:29:05.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9059999999999996e-06 2024-09-17T07:29:05.710 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.710 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.710 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.710 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20243 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.710 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.320875+0000", 2024-09-17T07:29:05.711 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0056617100000000004, 2024-09-17T07:29:05.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0093624559999999999, 2024-09-17T07:29:05.711 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.711 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.711 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.711 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.711 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.711 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.711 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20243 2024-09-17T07:29:05.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.711 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320875+0000", 2024-09-17T07:29:05.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320875+0000", 2024-09-17T07:29:05.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320875+0000", 2024-09-17T07:29:05.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3000000000000002e-07 2024-09-17T07:29:05.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320876+0000", 2024-09-17T07:29:05.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5400000000000001e-06 2024-09-17T07:29:05.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320883+0000", 2024-09-17T07:29:05.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5139999999999998e-06 2024-09-17T07:29:05.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320888+0000", 2024-09-17T07:29:05.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8679999999999998e-06 2024-09-17T07:29:05.713 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.713 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.713 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.714 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20070 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.714 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.272419+0000", 2024-09-17T07:29:05.714 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.054116781000000003, 2024-09-17T07:29:05.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.057842402000000001, 2024-09-17T07:29:05.714 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.714 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.714 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.714 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.714 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.714 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.714 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20070 2024-09-17T07:29:05.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.715 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272419+0000", 2024-09-17T07:29:05.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272419+0000", 2024-09-17T07:29:05.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272420+0000", 2024-09-17T07:29:05.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:29:05.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272421+0000", 2024-09-17T07:29:05.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0599999999999999e-07 2024-09-17T07:29:05.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272424+0000", 2024-09-17T07:29:05.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9519999999999999e-06 2024-09-17T07:29:05.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.272426+0000", 2024-09-17T07:29:05.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.486e-06 2024-09-17T07:29:05.717 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.717 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.717 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.717 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20168 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.717 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.297562+0000", 2024-09-17T07:29:05.717 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.028973997000000001, 2024-09-17T07:29:05.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032723466999999999, 2024-09-17T07:29:05.717 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.717 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.718 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.718 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.718 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.718 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.718 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20168 2024-09-17T07:29:05.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.718 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297562+0000", 2024-09-17T07:29:05.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297562+0000", 2024-09-17T07:29:05.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297563+0000", 2024-09-17T07:29:05.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.34e-07 2024-09-17T07:29:05.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297565+0000", 2024-09-17T07:29:05.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6560000000000001e-06 2024-09-17T07:29:05.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297573+0000", 2024-09-17T07:29:05.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2230000000000001e-06 2024-09-17T07:29:05.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.297579+0000", 2024-09-17T07:29:05.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1789999999999996e-06 2024-09-17T07:29:05.720 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.720 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.720 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.721 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20244 3.1c 3.a9a18d1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:05.721 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.320893+0000", 2024-09-17T07:29:05.721 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.0056430669999999999, 2024-09-17T07:29:05.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0094171519999999998, 2024-09-17T07:29:05.721 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:05.721 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:05.721 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:05.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:05.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:05.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:05.721 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20244 2024-09-17T07:29:05.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.721 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:05.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:05.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320893+0000", 2024-09-17T07:29:05.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:05.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320893+0000", 2024-09-17T07:29:05.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:05.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:05.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320894+0000", 2024-09-17T07:29:05.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:29:05.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:05.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320895+0000", 2024-09-17T07:29:05.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5990000000000001e-06 2024-09-17T07:29:05.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:05.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320902+0000", 2024-09-17T07:29:05.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5860000000000001e-06 2024-09-17T07:29:05.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:05.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:05.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:05.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.320906+0000", 2024-09-17T07:29:05.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.78e-06 2024-09-17T07:29:05.723 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.724 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:05.724 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.724 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:05.724 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-09-17T07:29:05.724 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 114 2024-09-17T07:29:05.724 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:05.725 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:05.725 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:05.726 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:05.726 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:05.727 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:05.727 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:05.727 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:05.727 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:05.728 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:05.728 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:05.728 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:05.729 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:05.729 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:05.729 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:05.730 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:05.730 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:05.732 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-17T07:29:05.732 INFO:teuthology.orchestra.run.smithi045.stdout:1088 read 0x2e92d40 thru 0x2e9a4df (0x77a0 bytes) 2024-09-17T07:29:05.732 INFO:teuthology.orchestra.run.smithi045.stdout:1089 trunc from 0xe273bc4 to 0xa1ea52f 2024-09-17T07:29:05.732 INFO:teuthology.orchestra.run.smithi045.stdout:1090 writesame 0x1a31bbf thru 0x1a411ac data_size 0x15(0xf5ee bytes) 2024-09-17T07:29:05.732 INFO:teuthology.orchestra.run.smithi045.stdout:1091 writesame 0x754e2ad thru 0x7553c84 data_size 0x64(0x59d8 bytes) 2024-09-17T07:29:05.732 INFO:teuthology.orchestra.run.smithi045.stdout:1093 write 0x216572a thru 0x216817a (0x2a51 bytes) 2024-09-17T07:29:05.732 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:05.732 INFO:teuthology.orchestra.run.smithi045.stdout:1096 read 0x137356f thru 0x13790b1 (0x5b43 bytes) 2024-09-17T07:29:05.732 INFO:teuthology.orchestra.run.smithi045.stdout:1097 write 0x3badc07 thru 0x3bbd6c8 (0xfac2 bytes) 2024-09-17T07:29:05.732 INFO:teuthology.orchestra.run.smithi045.stdout:1099 write 0xc838b5e thru 0xc840180 (0x7623 bytes) 2024-09-17T07:29:05.732 INFO:teuthology.orchestra.run.smithi045.stdout:1100 writesame 0xe197d78 thru 0xe199cfc data_size 0x1(0x1f85 bytes) 2024-09-17T07:29:05.733 INFO:teuthology.orchestra.run.smithi045.stdout:1101 punch from 0x88c6613 to 0x88c9096, (0x2a83 bytes) 2024-09-17T07:29:05.733 INFO:teuthology.orchestra.run.smithi045.stdout:1102 trunc from 0xe199cfd to 0x54a1447 2024-09-17T07:29:05.733 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:05.733 INFO:teuthology.orchestra.run.smithi045.stdout:1104 write 0x217179a thru 0x217d5fd (0xbe64 bytes) 2024-09-17T07:29:05.733 INFO:teuthology.orchestra.run.smithi045.stdout:1105 read 0x45ee28d thru 0x45f7c57 (0x99cb bytes) 2024-09-17T07:29:05.733 INFO:teuthology.orchestra.run.smithi045.stdout:1106 write 0xb660fd3 thru 0xb663ca9 (0x2cd7 bytes) 2024-09-17T07:29:05.733 INFO:teuthology.orchestra.run.smithi045.stdout:1108 write 0x54cd3b1 thru 0x54daf50 (0xdba0 bytes) 2024-09-17T07:29:05.733 INFO:teuthology.orchestra.run.smithi045.stdout:1110 write 0xd163c34 thru 0xd16ed56 (0xb123 bytes) 2024-09-17T07:29:05.733 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:05.725+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:05.736 INFO:teuthology.orchestra.run.smithi045.stdout:1111 trunc from 0xd16ed57 to 0x55a6b02 2024-09-17T07:29:05.748 INFO:teuthology.orchestra.run.smithi045.stdout:1112 trunc from 0x55a6b02 to 0x50c3956 2024-09-17T07:29:05.765 INFO:teuthology.orchestra.run.smithi045.stdout:1113 read 0x165d39e thru 0x166a8e5 (0xd548 bytes) 2024-09-17T07:29:05.766 INFO:teuthology.orchestra.run.smithi045.stdout:1114 punch from 0x1181901 to 0x11823c1, (0xac0 bytes) 2024-09-17T07:29:05.766 INFO:teuthology.orchestra.run.smithi045.stdout:1116 write 0x275b081 thru 0x275b633 (0x5b3 bytes) 2024-09-17T07:29:05.769 INFO:teuthology.orchestra.run.smithi045.stdout:1117 writesame 0x25a2caa thru 0x25b03c5 data_size 0x4e(0xd71c bytes) 2024-09-17T07:29:05.791 INFO:teuthology.orchestra.run.smithi045.stdout:1118 trunc from 0x50c3956 to 0xcf6850f 2024-09-17T07:29:05.795 INFO:teuthology.orchestra.run.smithi045.stdout:1119 writesame 0x35e47a6 thru 0x35e9031 data_size 0x2(0x488c bytes) 2024-09-17T07:29:05.809 INFO:teuthology.orchestra.run.smithi045.stdout:1121 trunc from 0xcf6850f to 0x5942ac8 2024-09-17T07:29:05.815 INFO:teuthology.orchestra.run.smithi045.stdout:1123 read 0x2bd5e67 thru 0x2bd6e61 (0xffb bytes) 2024-09-17T07:29:05.815 INFO:teuthology.orchestra.run.smithi045.stdout:1124 punch from 0x2eabcb9 to 0x2eb4195, (0x84dc bytes) 2024-09-17T07:29:05.821 INFO:teuthology.orchestra.run.smithi045.stdout:1125 trunc from 0x5942ac8 to 0x28b54e3 2024-09-17T07:29:05.826 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:05.826+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:05.846 INFO:teuthology.orchestra.run.smithi045.stdout:1129 read 0xd28d35 thru 0xd2e817 (0x5ae3 bytes) 2024-09-17T07:29:05.851 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:05.851 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:05.860 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:05.861 INFO:teuthology.orchestra.run.smithi045.stdout:1130 trunc from 0x28b54e3 to 0xec5bd22 2024-09-17T07:29:05.866 INFO:teuthology.orchestra.run.smithi045.stdout:1131 read 0x7febea6 thru 0x7ff1794 (0x58ef bytes) 2024-09-17T07:29:05.866 INFO:teuthology.orchestra.run.smithi045.stdout:1132 read 0x2466f5 thru 0x249e6e (0x377a bytes) 2024-09-17T07:29:05.867 INFO:teuthology.orchestra.run.smithi045.stdout:1134 read 0xd7b5a14 thru 0xd7bcf34 (0x7521 bytes) 2024-09-17T07:29:05.867 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:05.867 INFO:teuthology.orchestra.run.smithi045.stdout:1137 read 0x9a95ceb thru 0x9a9a61b (0x4931 bytes) 2024-09-17T07:29:05.867 INFO:teuthology.orchestra.run.smithi045.stdout:1138 read 0xca47908 thru 0xca4ff94 (0x868d bytes) 2024-09-17T07:29:05.878 INFO:teuthology.orchestra.run.smithi045.stdout:1139 trunc from 0xec5bd22 to 0x41ed17e 2024-09-17T07:29:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:1140 writesame 0xd6fb55d thru 0xd709b41 data_size 0x1(0xe5e5 bytes) 2024-09-17T07:29:05.901 INFO:teuthology.orchestra.run.smithi045.stdout:1141 write 0x40fd88b thru 0x4106828 (0x8f9e bytes) 2024-09-17T07:29:05.908 INFO:teuthology.orchestra.run.smithi045.stdout:1142 read 0x5a8a586 thru 0x5a973a6 (0xce21 bytes) 2024-09-17T07:29:05.909 INFO:teuthology.orchestra.run.smithi045.stdout:1143 punch from 0x73a3160 to 0x73a7c73, (0x4b13 bytes) 2024-09-17T07:29:05.909 INFO:teuthology.orchestra.run.smithi045.stdout:1144 read 0xa71a2d5 thru 0xa722366 (0x8092 bytes) 2024-09-17T07:29:05.909 INFO:teuthology.orchestra.run.smithi045.stdout:1145 write 0x94eeb5 thru 0x9555e3 (0x672f bytes) 2024-09-17T07:29:05.935 INFO:teuthology.orchestra.run.smithi045.stdout:1146 read 0xa0ec1f3 thru 0xa0f46ca (0x84d8 bytes) 2024-09-17T07:29:05.935 INFO:teuthology.orchestra.run.smithi045.stdout:1147 writesame 0x8e7e57c thru 0x8e8e4db data_size 0x90(0xff60 bytes) 2024-09-17T07:29:05.942 INFO:teuthology.orchestra.run.smithi045.stdout:1149 read 0x5e30ede thru 0x5e33ed7 (0x2ffa bytes) 2024-09-17T07:29:05.942 INFO:teuthology.orchestra.run.smithi045.stdout:1150 read 0xca7e910 thru 0xca8e3ff (0xfaf0 bytes) 2024-09-17T07:29:05.942 INFO:teuthology.orchestra.run.smithi045.stdout:1151 writesame 0x12d7d5d thru 0x12e1898 data_size 0xa(0x9b3c bytes) 2024-09-17T07:29:05.946 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:05.944+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:05.976 INFO:teuthology.orchestra.run.smithi045.stdout:1153 write 0x4d1495c thru 0x4d159ef (0x1094 bytes) 2024-09-17T07:29:05.980 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:05.985 INFO:teuthology.orchestra.run.smithi045.stdout:1156 writesame 0xe4e231e thru 0xe4e4d08 data_size 0x15(0x29eb bytes) 2024-09-17T07:29:05.985 INFO:teuthology.orchestra.run.smithi045.stdout:1160 read 0x42621a3 thru 0x4265e2b (0x3c89 bytes) 2024-09-17T07:29:05.991 INFO:teuthology.orchestra.run.smithi045.stdout:1161 trunc from 0xe4e4d09 to 0x8f15316 2024-09-17T07:29:05.998 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:06.005 INFO:teuthology.orchestra.run.smithi045.stdout:1163 write 0xa7e19d7 thru 0xa7e7880 (0x5eaa bytes) 2024-09-17T07:29:06.010 INFO:teuthology.orchestra.run.smithi045.stdout:1164 write 0x69fe0d6 thru 0x6a0bd97 (0xdcc2 bytes) 2024-09-17T07:29:06.018 INFO:teuthology.orchestra.run.smithi045.stdout:1165 write 0x42f6e39 thru 0x42faa8e (0x3c56 bytes) 2024-09-17T07:29:06.024 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:06.025 INFO:teuthology.orchestra.run.smithi045.stdout:1167 writesame 0x4be955a thru 0x4bed2a2 data_size 0x1(0x3d49 bytes) 2024-09-17T07:29:06.028 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:06.027+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:06.030 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:06.030 INFO:teuthology.orchestra.run.smithi045.stdout:1170 write 0x9d6e08c thru 0x9d7ca1e (0xe993 bytes) 2024-09-17T07:29:06.030 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:06.031 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:06.040 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:06.047 INFO:teuthology.orchestra.run.smithi045.stdout:1171 trunc from 0xa7e7881 to 0x1fe8855 2024-09-17T07:29:06.068 INFO:teuthology.orchestra.run.smithi045.stdout:1173 punch from 0x131703e to 0x131cda1, (0x5d63 bytes) 2024-09-17T07:29:06.074 INFO:teuthology.orchestra.run.smithi045.stdout:1174 trunc from 0x1fe8855 to 0x74250ed 2024-09-17T07:29:06.093 INFO:teuthology.orchestra.run.smithi045.stdout:1176 write 0xe64ca2c thru 0xe652186 (0x575b bytes) 2024-09-17T07:29:06.100 INFO:teuthology.orchestra.run.smithi045.stdout:1177 trunc from 0xe652187 to 0xcf8b5b2 2024-09-17T07:29:06.120 INFO:teuthology.orchestra.run.smithi045.stdout:1179 trunc from 0xcf8b5b2 to 0x3fee781 2024-09-17T07:29:06.127 INFO:teuthology.orchestra.run.smithi045.stdout:1181 clone 13 order 20 su 131072 sc 12 2024-09-17T07:29:06.129 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:06.128+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:06.176 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T07:29:06.190 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:06.190 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:06.199 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:06.239 INFO:teuthology.orchestra.run.smithi045.stdout:leaving image image_client.0-clone12 intact 2024-09-17T07:29:06.240 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:06.239+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:06.332 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:06.331+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:06.346 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:06.346 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:06.350 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-17T07:29:06.356 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:06.432 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:06.432+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:06.446 INFO:teuthology.orchestra.run.smithi045.stdout:1183 write 0xb9fa295 thru 0xba02ef0 (0x8c5c bytes) 2024-09-17T07:29:06.450 INFO:teuthology.orchestra.run.smithi045.stdout:1184 punch from 0x54e5ddd to 0x54eaa98, (0x4cbb bytes) 2024-09-17T07:29:06.451 INFO:teuthology.orchestra.run.smithi045.stdout:1185 write 0x9fa2161 thru 0x9fb123f (0xf0df bytes) 2024-09-17T07:29:06.456 INFO:teuthology.orchestra.run.smithi045.stdout:1189 trunc from 0xba02ef1 to 0xd7c39e2 2024-09-17T07:29:06.461 INFO:teuthology.orchestra.run.smithi045.stdout:1192 punch from 0x59552e6 to 0x595f12d, (0x9e47 bytes) 2024-09-17T07:29:06.462 INFO:teuthology.orchestra.run.smithi045.stdout:1193 writesame 0x83adaae thru 0x83b097a data_size 0x1(0x2ecd bytes) 2024-09-17T07:29:06.469 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:06.469 INFO:teuthology.orchestra.run.smithi045.stdout:1196 write 0x7dae9e3 thru 0x7dbc8f9 (0xdf17 bytes) 2024-09-17T07:29:06.470 INFO:teuthology.orchestra.run.smithi045.stdout:1197 writesame 0xb5ec384 thru 0xb5f3c95 data_size 0x1(0x7912 bytes) 2024-09-17T07:29:06.487 INFO:teuthology.orchestra.run.smithi045.stdout:1200 trunc from 0xd7c39e2 to 0xe1876a6 2024-09-17T07:29:06.493 INFO:teuthology.orchestra.run.smithi045.stdout:1201 write 0xe05a4a1 thru 0xe0643c3 (0x9f23 bytes) 2024-09-17T07:29:06.496 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:06.496 INFO:teuthology.orchestra.run.smithi045.stdout:1205 read 0x8f0b0b8 thru 0x8f0d125 (0x206e bytes) 2024-09-17T07:29:06.496 INFO:teuthology.orchestra.run.smithi045.stdout:1209 write 0x2cbaa51 thru 0x2cc6339 (0xb8e9 bytes) 2024-09-17T07:29:06.500 INFO:teuthology.orchestra.run.smithi045.stdout:1210 punch from 0x9e079d8 to 0x9e13c9e, (0xc2c6 bytes) 2024-09-17T07:29:06.501 INFO:teuthology.orchestra.run.smithi045.stdout:1211 punch from 0x25f3d6 to 0x268c0d, (0x9837 bytes) 2024-09-17T07:29:06.501 INFO:teuthology.orchestra.run.smithi045.stdout:1212 writesame 0x682f509 thru 0x6831848 data_size 0x30(0x2340 bytes) 2024-09-17T07:29:06.503 INFO:teuthology.orchestra.run.smithi045.stdout:1213 writesame 0x28a93b7 thru 0x28af8b9 data_size 0x1(0x6503 bytes) 2024-09-17T07:29:06.509 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:06.509 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:06.514 INFO:teuthology.orchestra.run.smithi045.stdout:1216 trunc from 0xe1876a6 to 0xe5d34e4 2024-09-17T07:29:06.519 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:06.520 INFO:teuthology.orchestra.run.smithi045.stdout:1218 writesame 0x8b6200b thru 0x8b65ad1 data_size 0x1(0x3ac7 bytes) 2024-09-17T07:29:06.527 INFO:teuthology.orchestra.run.smithi045.stdout:1219 read 0x1552a7d thru 0x15537cc (0xd50 bytes) 2024-09-17T07:29:06.529 INFO:teuthology.orchestra.run.smithi045.stdout:1220 read 0x981c19a thru 0x98276da (0xb541 bytes) 2024-09-17T07:29:06.529 INFO:teuthology.orchestra.run.smithi045.stdout:1221 punch from 0xd2edb15 to 0xd2f2580, (0x4a6b bytes) 2024-09-17T07:29:06.529 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:06.529 INFO:teuthology.orchestra.run.smithi045.stdout:1223 trunc from 0xe5d34e4 to 0xe8fccd7 2024-09-17T07:29:06.538 INFO:teuthology.orchestra.run.smithi045.stdout:1224 trunc from 0xe8fccd7 to 0xa4ab444 2024-09-17T07:29:06.559 INFO:teuthology.orchestra.run.smithi045.stdout:1225 write 0xcff2dcb thru 0xcff8d30 (0x5f66 bytes) 2024-09-17T07:29:06.562 INFO:teuthology.orchestra.run.smithi045.stdout:1226 write 0xa652f88 thru 0xa65954e (0x65c7 bytes) 2024-09-17T07:29:06.562 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:06.562+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:06.564 INFO:teuthology.orchestra.run.smithi045.stdout:1229 writesame 0x5a990f5 thru 0x5aa6824 data_size 0x58(0xd730 bytes) 2024-09-17T07:29:06.568 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:06.568 INFO:teuthology.orchestra.run.smithi045.stdout:1231 read 0xad9a205 thru 0xad9c336 (0x2132 bytes) 2024-09-17T07:29:06.568 INFO:teuthology.orchestra.run.smithi045.stdout:1232 write 0x92e40bc thru 0x92ed45e (0x93a3 bytes) 2024-09-17T07:29:06.571 INFO:teuthology.orchestra.run.smithi045.stdout:1233 write 0x16282e thru 0x171b8a (0xf35d bytes) 2024-09-17T07:29:06.584 INFO:teuthology.orchestra.run.smithi045.stdout:1234 punch from 0x7d5bb24 to 0x7d66730, (0xac0c bytes) 2024-09-17T07:29:06.584 INFO:teuthology.orchestra.run.smithi045.stdout:1235 read 0x83b482b thru 0x83b8ac3 (0x4299 bytes) 2024-09-17T07:29:06.585 INFO:teuthology.orchestra.run.smithi045.stdout:1236 read 0x63bee4d thru 0x63c7a6c (0x8c20 bytes) 2024-09-17T07:29:06.585 INFO:teuthology.orchestra.run.smithi045.stdout:1237 read 0x1bae9f9 thru 0x1baef75 (0x57d bytes) 2024-09-17T07:29:06.586 INFO:teuthology.orchestra.run.smithi045.stdout:1238 writesame 0x800a02d thru 0x8017e1d data_size 0x9(0xddf1 bytes) 2024-09-17T07:29:06.591 INFO:teuthology.orchestra.run.smithi045.stdout:1240 writesame 0xcc18a3e thru 0xcc2694c data_size 0x1(0xdf0f bytes) 2024-09-17T07:29:06.616 INFO:teuthology.orchestra.run.smithi045.stdout:1241 write 0xb67cad8 thru 0xb681319 (0x4842 bytes) 2024-09-17T07:29:06.619 INFO:teuthology.orchestra.run.smithi045.stdout:1244 punch from 0x6e83e85 to 0x6e88482, (0x45fd bytes) 2024-09-17T07:29:06.633 INFO:teuthology.orchestra.run.smithi045.stdout:1246 trunc from 0xcff8d31 to 0x1fc9620 2024-09-17T07:29:06.634 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:06.634 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:06.634 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:06.633+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:06.646 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:06.702 INFO:teuthology.orchestra.run.smithi045.stdout:1247 read 0xbfd058 thru 0xc093bd (0xc366 bytes) 2024-09-17T07:29:06.723 INFO:teuthology.orchestra.run.smithi045.stdout:1249 write 0xe7ce0d5 thru 0xe7d2101 (0x402d bytes) 2024-09-17T07:29:06.727 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:06.727 INFO:teuthology.orchestra.run.smithi045.stdout:1251 read 0x5974a4f thru 0x597f779 (0xad2b bytes) 2024-09-17T07:29:06.727 INFO:teuthology.orchestra.run.smithi045.stdout:1252 read 0x567d970 thru 0x5684512 (0x6ba3 bytes) 2024-09-17T07:29:06.727 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:06.727 INFO:teuthology.orchestra.run.smithi045.stdout:1254 write 0x4bf5b26 thru 0x4bf71a2 (0x167d bytes) 2024-09-17T07:29:06.729 INFO:teuthology.orchestra.run.smithi045.stdout:1255 write 0xc57ee51 thru 0xc58693d (0x7aed bytes) 2024-09-17T07:29:06.735 INFO:teuthology.orchestra.run.smithi045.stdout:1258 punch from 0x4a2c473 to 0x4a2fe88, (0x3a15 bytes) 2024-09-17T07:29:06.735 INFO:teuthology.orchestra.run.smithi045.stdout:1259 read 0x7439d54 thru 0x7444b57 (0xae04 bytes) 2024-09-17T07:29:06.735 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:06.734+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:06.737 INFO:teuthology.orchestra.run.smithi045.stdout:1260 punch from 0xc5b089a to 0xc5b31a9, (0x290f bytes) 2024-09-17T07:29:06.738 INFO:teuthology.orchestra.run.smithi045.stdout:1261 punch from 0x2c2c0e9 to 0x2c31de7, (0x5cfe bytes) 2024-09-17T07:29:06.738 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:06.738 INFO:teuthology.orchestra.run.smithi045.stdout:1263 read 0xbb38b5d thru 0xbb47248 (0xe6ec bytes) 2024-09-17T07:29:06.738 INFO:teuthology.orchestra.run.smithi045.stdout:1264 read 0x5b1733a thru 0x5b18481 (0x1148 bytes) 2024-09-17T07:29:06.738 INFO:teuthology.orchestra.run.smithi045.stdout:1267 punch from 0x3ac2a63 to 0x3ac6c7b, (0x4218 bytes) 2024-09-17T07:29:06.738 INFO:teuthology.orchestra.run.smithi045.stdout:1268 punch from 0x503473a to 0x503d04f, (0x8915 bytes) 2024-09-17T07:29:06.751 INFO:teuthology.orchestra.run.smithi045.stdout:1269 trunc from 0xe7d2102 to 0xc86e6 2024-09-17T07:29:06.808 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:06.808 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:06.817 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:06.833 INFO:teuthology.orchestra.run.smithi045.stdout:1270 read 0x23c0b thru 0x2a229 (0x661f bytes) 2024-09-17T07:29:06.834 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:06.846 INFO:teuthology.orchestra.run.smithi045.stdout:1272 write 0x5718571 thru 0x5720230 (0x7cc0 bytes) 2024-09-17T07:29:06.849 INFO:teuthology.orchestra.run.smithi045.stdout:1274 read 0x3ba588d thru 0x3bab23d (0x59b1 bytes) 2024-09-17T07:29:06.849 INFO:teuthology.orchestra.run.smithi045.stdout:1275 read 0x1cf1b9 thru 0x1d18f7 (0x273f bytes) 2024-09-17T07:29:06.850 INFO:teuthology.orchestra.run.smithi045.stdout:1276 read 0x1ce8bda thru 0x1ceebc4 (0x5feb bytes) 2024-09-17T07:29:06.861 INFO:teuthology.orchestra.run.smithi045.stdout:1277 write 0xa6697b3 thru 0xa6774ea (0xdd38 bytes) 2024-09-17T07:29:06.861 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:06.861+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:06.864 INFO:teuthology.orchestra.run.smithi045.stdout:1279 read 0x84e6477 thru 0x84ea7c5 (0x434f bytes) 2024-09-17T07:29:06.864 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:06.864 INFO:teuthology.orchestra.run.smithi045.stdout:1281 write 0x6118efe thru 0x61228d5 (0x99d8 bytes) 2024-09-17T07:29:06.867 INFO:teuthology.orchestra.run.smithi045.stdout:1282 read 0x72a72eb thru 0x72a8503 (0x1219 bytes) 2024-09-17T07:29:06.867 INFO:teuthology.orchestra.run.smithi045.stdout:1283 write 0x2bb13b1 thru 0x2bc0f43 (0xfb93 bytes) 2024-09-17T07:29:06.871 INFO:teuthology.orchestra.run.smithi045.stdout:1284 punch from 0x63adb3a to 0x63b7ed0, (0xa396 bytes) 2024-09-17T07:29:06.871 INFO:teuthology.orchestra.run.smithi045.stdout:1285 punch from 0x782a38a to 0x78324cb, (0x8141 bytes) 2024-09-17T07:29:06.871 INFO:teuthology.orchestra.run.smithi045.stdout:1286 write 0x2951b5b thru 0x295e89c (0xcd42 bytes) 2024-09-17T07:29:06.874 INFO:teuthology.orchestra.run.smithi045.stdout:1287 read 0x5283e22 thru 0x5284b2b (0xd0a bytes) 2024-09-17T07:29:06.874 INFO:teuthology.orchestra.run.smithi045.stdout:1288 write 0x2141fc thru 0x221c9f (0xdaa4 bytes) 2024-09-17T07:29:06.875 INFO:teuthology.orchestra.run.smithi045.stdout:1289 punch from 0xa76168 to 0xa79102, (0x2f9a bytes) 2024-09-17T07:29:06.878 INFO:teuthology.orchestra.run.smithi045.stdout:1291 trunc from 0xa6774eb to 0xd12198d 2024-09-17T07:29:06.892 INFO:teuthology.orchestra.run.smithi045.stdout:1293 writesame 0xdd55290 thru 0xdd582bd data_size 0x7(0x302e bytes) 2024-09-17T07:29:06.892 INFO:teuthology.orchestra.run.smithi045.stdout:1295 writesame 0x75b611e thru 0x75bb52a data_size 0x1(0x540d bytes) 2024-09-17T07:29:06.901 INFO:teuthology.orchestra.run.smithi045.stdout:1296 writesame 0xbafe87c thru 0xbb08cfb data_size 0x80(0xa480 bytes) 2024-09-17T07:29:06.905 INFO:teuthology.orchestra.run.smithi045.stdout:1297 read 0x8c4db9d thru 0x8c501bd (0x2621 bytes) 2024-09-17T07:29:06.905 INFO:teuthology.orchestra.run.smithi045.stdout:1298 writesame 0x68794c8 thru 0x687f261 data_size 0x1(0x5d9a bytes) 2024-09-17T07:29:06.914 INFO:teuthology.orchestra.run.smithi045.stdout:1299 write 0x15100f2 thru 0x1513f86 (0x3e95 bytes) 2024-09-17T07:29:06.917 INFO:teuthology.orchestra.run.smithi045.stdout:1300 read 0xb739a5f thru 0xb74598d (0xbf2f bytes) 2024-09-17T07:29:06.918 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:06.918 INFO:teuthology.orchestra.run.smithi045.stdout:1302 read 0x2236799 thru 0x2238544 (0x1dac bytes) 2024-09-17T07:29:06.918 INFO:teuthology.orchestra.run.smithi045.stdout:1304 read 0x9ea215b thru 0x9eac997 (0xa83d bytes) 2024-09-17T07:29:06.928 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:06.929 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:06.930 INFO:teuthology.orchestra.run.smithi045.stdout:1305 trunc from 0xdd582be to 0x351037b 2024-09-17T07:29:06.936 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:06.935+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:06.939 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:06.950 INFO:teuthology.orchestra.run.smithi045.stdout:1306 trunc from 0x351037b to 0xb659994 2024-09-17T07:29:06.956 INFO:teuthology.orchestra.run.smithi045.stdout:1307 read 0xa601de3 thru 0xa60243b (0x659 bytes) 2024-09-17T07:29:06.956 INFO:teuthology.orchestra.run.smithi045.stdout:1308 read 0x686f9fe thru 0x687add3 (0xb3d6 bytes) 2024-09-17T07:29:06.956 INFO:teuthology.orchestra.run.smithi045.stdout:1311 punch from 0x2b7503c to 0x2b7dd2e, (0x8cf2 bytes) 2024-09-17T07:29:06.956 INFO:teuthology.orchestra.run.smithi045.stdout:1312 write 0x1a29ca8 thru 0x1a2eb00 (0x4e59 bytes) 2024-09-17T07:29:06.959 INFO:teuthology.orchestra.run.smithi045.stdout:1313 punch from 0x88ab8d9 to 0x88bad14, (0xf43b bytes) 2024-09-17T07:29:06.959 INFO:teuthology.orchestra.run.smithi045.stdout:1314 read 0x500cf85 thru 0x5011ce9 (0x4d65 bytes) 2024-09-17T07:29:06.959 INFO:teuthology.orchestra.run.smithi045.stdout:1317 writesame 0x85d33ee thru 0x85dd496 data_size 0x1(0xa0a9 bytes) 2024-09-17T07:29:06.987 INFO:teuthology.orchestra.run.smithi045.stdout:1318 writesame 0xdadfd93 thru 0xdaed218 data_size 0x2(0xd486 bytes) 2024-09-17T07:29:06.987 INFO:teuthology.orchestra.run.smithi045.stdout:1320 read 0x3afa360 thru 0x3b043e0 (0xa081 bytes) 2024-09-17T07:29:06.988 INFO:teuthology.orchestra.run.smithi045.stdout:1322 read 0x8adeada thru 0x8ae2df9 (0x4320 bytes) 2024-09-17T07:29:06.988 INFO:teuthology.orchestra.run.smithi045.stdout:1323 read 0x6259e thru 0x7120c (0xec6f bytes) 2024-09-17T07:29:06.988 INFO:teuthology.orchestra.run.smithi045.stdout:1324 writesame 0x68d06f9 thru 0x68d9cb2 data_size 0x5(0x95ba bytes) 2024-09-17T07:29:06.993 INFO:teuthology.orchestra.run.smithi045.stdout:1325 punch from 0x46db031 to 0x46e12a0, (0x626f bytes) 2024-09-17T07:29:06.993 INFO:teuthology.orchestra.run.smithi045.stdout:1327 punch from 0x97b5276 to 0x97c4926, (0xf6b0 bytes) 2024-09-17T07:29:06.993 INFO:teuthology.orchestra.run.smithi045.stdout:1328 write 0xaa66a72 thru 0xaa69dfe (0x338d bytes) 2024-09-17T07:29:06.996 INFO:teuthology.orchestra.run.smithi045.stdout:1329 write 0x1618fdc thru 0x162356c (0xa591 bytes) 2024-09-17T07:29:06.999 INFO:teuthology.orchestra.run.smithi045.stdout:1330 write 0x4a52e3 thru 0x4a9180 (0x3e9e bytes) 2024-09-17T07:29:07.001 INFO:teuthology.orchestra.run.smithi045.stdout:1331 read 0xc277ebc thru 0xc280aa3 (0x8be8 bytes) 2024-09-17T07:29:07.001 INFO:teuthology.orchestra.run.smithi045.stdout:1332 writesame 0x8ae5bc1 thru 0x8af2588 data_size 0x2c(0xc9c8 bytes) 2024-09-17T07:29:07.004 INFO:teuthology.orchestra.run.smithi045.stdout:1333 read 0x7a5571d thru 0x7a5971a (0x3ffe bytes) 2024-09-17T07:29:07.012 INFO:teuthology.orchestra.run.smithi045.stdout:1334 trunc from 0xdaed219 to 0x664f365 2024-09-17T07:29:07.023 INFO:teuthology.orchestra.run.smithi045.stdout:1335 read 0x3531277 thru 0x353f53c (0xe2c6 bytes) 2024-09-17T07:29:07.028 INFO:teuthology.orchestra.run.smithi045.stdout:1336 writesame 0x6a6dd53 thru 0x6a6fbd5 data_size 0x1(0x1e83 bytes) 2024-09-17T07:29:07.028 INFO:teuthology.orchestra.run.smithi045.stdout:1338 writesame 0x51dea4c thru 0x51e9dd4 data_size 0x29(0xb389 bytes) 2024-09-17T07:29:07.032 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:07.036 INFO:teuthology.orchestra.run.smithi045.stdout:1340 trunc from 0x6a6fbd6 to 0x2b5d9f7 2024-09-17T07:29:07.037 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:07.036+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:07.048 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:07.048 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:07.048 INFO:teuthology.orchestra.run.smithi045.stdout:1343 writesame 0xe1f593 thru 0xe277f5 data_size 0x1(0x8263 bytes) 2024-09-17T07:29:07.058 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:07.058 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:07.060 INFO:teuthology.orchestra.run.smithi045.stdout:1344 write 0x177833c thru 0x177a6c7 (0x238c bytes) 2024-09-17T07:29:07.063 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:07.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:07.069 INFO:teuthology.orchestra.run.smithi045.stdout:1346 trunc from 0x2b5d9f7 to 0x7748c8e 2024-09-17T07:29:07.086 INFO:teuthology.orchestra.run.smithi045.stdout:1347 write 0xde1a8cc thru 0xde27027 (0xc75c bytes) 2024-09-17T07:29:07.089 INFO:teuthology.orchestra.run.smithi045.stdout:1349 read 0x277becc thru 0x277def3 (0x2028 bytes) 2024-09-17T07:29:07.089 INFO:teuthology.orchestra.run.smithi045.stdout:1350 writesame 0x42832e9 thru 0x42892b1 data_size 0x1f(0x5fc9 bytes) 2024-09-17T07:29:07.092 INFO:teuthology.orchestra.run.smithi045.stdout:1355 write 0xb13047f thru 0xb13d9a5 (0xd527 bytes) 2024-09-17T07:29:07.106 INFO:teuthology.orchestra.run.smithi045.stdout:1356 trunc from 0xde27028 to 0x45fc35f 2024-09-17T07:29:07.134 INFO:teuthology.orchestra.run.smithi045.stdout:1357 write 0xeb960ba thru 0xeb96ca5 (0xbec bytes) 2024-09-17T07:29:07.137 INFO:teuthology.orchestra.run.smithi045.stdout:1359 write 0x43df993 thru 0x43e5006 (0x5674 bytes) 2024-09-17T07:29:07.137 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:07.137+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:07.152 INFO:teuthology.orchestra.run.smithi045.stdout:1360 trunc from 0xeb96ca6 to 0x50872d5 2024-09-17T07:29:07.170 INFO:teuthology.orchestra.run.smithi045.stdout:1361 trunc from 0x50872d5 to 0xc1cf588 2024-09-17T07:29:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-17T07:29:07.179 INFO:teuthology.orchestra.run.smithi045.stdout:1362 read 0x234baac thru 0x2358de6 (0xd33b bytes) 2024-09-17T07:29:07.187 INFO:teuthology.orchestra.run.smithi045.stdout:1363 write 0xd446e58 thru 0xd4517bc (0xa965 bytes) 2024-09-17T07:29:07.190 INFO:teuthology.orchestra.run.smithi045.stdout:1367 clone 14 order 19 su 131072 sc 15 2024-09-17T07:29:07.192 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:07.193 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:07.201 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:07.238 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:07.238+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:07.339 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:07.338+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:07.349 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:07.349 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:07.358 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:07.439 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:07.439+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:07.537 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:07.537 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:07.537 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:07.540 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:07.540+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:07.642 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:07.641+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:07.644 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:07.644 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:07.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:07.731 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:29:07.742 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:07.742+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:07.760 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:07.760 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:07.770 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:07.843 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:07.843+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:07.859 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:07.859 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:07.876 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:07.876 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:07.893 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:07.893 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:07.893 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:07.910 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:07.910 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:07.921 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:07.921 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:07.921 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:07.921 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:07.927 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:07.927 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:07.929 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:07.944 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:07.944 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:07.945 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:07.944+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:07.961 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:07.961 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:07.964 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:07.965 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:08.036 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:08.037 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:08.037 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:08.037 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:08.037 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:08.045 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:08.046 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:08.045+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:08.146 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:08.146+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:08.153 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:08.153 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:08.153 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:08.153 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:08.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.154 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:1746 3.4 e30/23 3:21509c5f:::rbd_data.11832b928878.00000000000001bd:head v 30'14, mlcod=30'11)", 2024-09-17T07:29:08.154 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:50.791256+0000", 2024-09-17T07:29:08.154 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.360969955000002, 2024-09-17T07:29:08.154 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001583414, 2024-09-17T07:29:08.154 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.154 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.154 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.154 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.154 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.154 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791256+0000", 2024-09-17T07:29:08.154 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.155 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.155 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791256+0000", 2024-09-17T07:29:08.155 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.155 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.155 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791257+0000", 2024-09-17T07:29:08.155 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8599999999999997e-07 2024-09-17T07:29:08.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.155 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.155 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791290+0000", 2024-09-17T07:29:08.155 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3158999999999999e-05 2024-09-17T07:29:08.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.156 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.156 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791298+0000", 2024-09-17T07:29:08.156 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2700000000000004e-06 2024-09-17T07:29:08.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.156 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.156 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791305+0000", 2024-09-17T07:29:08.156 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8159999999999996e-06 2024-09-17T07:29:08.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.156 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.156 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791358+0000", 2024-09-17T07:29:08.156 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3300999999999997e-05 2024-09-17T07:29:08.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.157 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.157 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.791377+0000", 2024-09-17T07:29:08.157 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9062000000000001e-05 2024-09-17T07:29:08.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.157 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.157 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.792791+0000", 2024-09-17T07:29:08.157 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014139160000000001 2024-09-17T07:29:08.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.157 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.158 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:50.792839+0000", 2024-09-17T07:29:08.158 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8103999999999999e-05 2024-09-17T07:29:08.158 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.158 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.158 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.158 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:2183 3.4 e30/23 3:221ed066:::rbd_data.11832b928878.0000000000000073:head v 30'19, mlcod=30'17)", 2024-09-17T07:29:08.158 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:51.139568+0000", 2024-09-17T07:29:08.158 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.012657871999998, 2024-09-17T07:29:08.158 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00124438, 2024-09-17T07:29:08.158 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.158 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.158 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139568+0000", 2024-09-17T07:29:08.159 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139568+0000", 2024-09-17T07:29:08.159 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139569+0000", 2024-09-17T07:29:08.159 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3769999999999999e-06 2024-09-17T07:29:08.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.160 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139580+0000", 2024-09-17T07:29:08.160 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0229e-05 2024-09-17T07:29:08.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.160 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139601+0000", 2024-09-17T07:29:08.160 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1818999999999999e-05 2024-09-17T07:29:08.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.161 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139611+0000", 2024-09-17T07:29:08.161 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0121e-05 2024-09-17T07:29:08.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.161 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.161 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139676+0000", 2024-09-17T07:29:08.161 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4900999999999994e-05 2024-09-17T07:29:08.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.161 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.161 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139706+0000", 2024-09-17T07:29:08.161 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9762000000000001e-05 2024-09-17T07:29:08.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.161 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.162 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.140765+0000", 2024-09-17T07:29:08.162 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001059019 2024-09-17T07:29:08.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.162 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.162 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.140812+0000", 2024-09-17T07:29:08.162 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7151999999999998e-05 2024-09-17T07:29:08.162 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.162 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.162 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.163 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:2190 3.0 e30/23 3:02fd7b4f:::rbd_data.11832b928878.00000000000000b4:head v 30'16, mlcod=30'14)", 2024-09-17T07:29:08.163 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:29:08.163 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.012313044999999, 2024-09-17T07:29:08.163 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001228395, 2024-09-17T07:29:08.163 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.163 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.163 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.163 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.163 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.163 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:29:08.163 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:29:08.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139913+0000", 2024-09-17T07:29:08.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0399999999999996e-07 2024-09-17T07:29:08.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139918+0000", 2024-09-17T07:29:08.165 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8330000000000001e-06 2024-09-17T07:29:08.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.165 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.165 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139926+0000", 2024-09-17T07:29:08.165 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1720000000000007e-06 2024-09-17T07:29:08.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.165 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.165 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139932+0000", 2024-09-17T07:29:08.165 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1789999999999996e-06 2024-09-17T07:29:08.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.166 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.166 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.139988+0000", 2024-09-17T07:29:08.166 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5265e-05 2024-09-17T07:29:08.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.166 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.166 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.140004+0000", 2024-09-17T07:29:08.166 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6171000000000002e-05 2024-09-17T07:29:08.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.166 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.166 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.141100+0000", 2024-09-17T07:29:08.166 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010964519999999999 2024-09-17T07:29:08.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.167 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.167 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:51.141141+0000", 2024-09-17T07:29:08.167 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0818999999999997e-05 2024-09-17T07:29:08.167 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.167 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.167 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.167 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:4848 3.9 e33/23 3:97363921:::rbd_data.1183cff49e3a.0000000000000004:head v 33'102, mlcod=32'100)", 2024-09-17T07:29:08.167 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:08.167 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.912729978, 2024-09-17T07:29:08.167 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023715110000000002, 2024-09-17T07:29:08.167 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.168 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.168 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.168 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.168 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.168 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:08.168 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.168 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.168 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:08.168 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.169 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.169 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239497+0000", 2024-09-17T07:29:08.169 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.53e-06 2024-09-17T07:29:08.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.169 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.169 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240142+0000", 2024-09-17T07:29:08.169 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00064481300000000005 2024-09-17T07:29:08.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.169 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.169 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240160+0000", 2024-09-17T07:29:08.169 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8349999999999999e-05 2024-09-17T07:29:08.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.170 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.170 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240169+0000", 2024-09-17T07:29:08.170 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7690000000000002e-06 2024-09-17T07:29:08.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.170 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.170 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240246+0000", 2024-09-17T07:29:08.170 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6277000000000002e-05 2024-09-17T07:29:08.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.170 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.170 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240270+0000", 2024-09-17T07:29:08.170 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4575000000000001e-05 2024-09-17T07:29:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.241813+0000", 2024-09-17T07:29:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015432860000000001 2024-09-17T07:29:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.241867+0000", 2024-09-17T07:29:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3910999999999999e-05 2024-09-17T07:29:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:4849 3.9 e33/23 3:97363921:::rbd_data.1183cff49e3a.0000000000000004:head v 33'104, mlcod=32'100)", 2024-09-17T07:29:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.240572+0000", 2024-09-17T07:29:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.911653861, 2024-09-17T07:29:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015942700000000001, 2024-09-17T07:29:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240572+0000", 2024-09-17T07:29:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240572+0000", 2024-09-17T07:29:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240572+0000", 2024-09-17T07:29:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3799999999999997e-07 2024-09-17T07:29:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240577+0000", 2024-09-17T07:29:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3000000000000003e-06 2024-09-17T07:29:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240584+0000", 2024-09-17T07:29:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2300000000000002e-06 2024-09-17T07:29:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240588+0000", 2024-09-17T07:29:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7340000000000002e-06 2024-09-17T07:29:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240936+0000", 2024-09-17T07:29:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00034820700000000001 2024-09-17T07:29:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240955+0000", 2024-09-17T07:29:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8808e-05 2024-09-17T07:29:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.242116+0000", 2024-09-17T07:29:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011614590000000001 2024-09-17T07:29:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.242166+0000", 2024-09-17T07:29:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9994000000000003e-05 2024-09-17T07:29:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:5460 3.9 e33/23 3:90773955:::rbd_data.1183cff49e3a.000000000000000a:head v 33'106, mlcod=33'104)", 2024-09-17T07:29:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.625633391999999, 2024-09-17T07:29:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020825850000000001, 2024-09-17T07:29:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526593+0000", 2024-09-17T07:29:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.54e-07 2024-09-17T07:29:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527050+0000", 2024-09-17T07:29:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000456522 2024-09-17T07:29:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527061+0000", 2024-09-17T07:29:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1722000000000001e-05 2024-09-17T07:29:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527067+0000", 2024-09-17T07:29:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0599999999999998e-06 2024-09-17T07:29:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527112+0000", 2024-09-17T07:29:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999000000000001e-05 2024-09-17T07:29:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527128+0000", 2024-09-17T07:29:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6920999999999999e-05 2024-09-17T07:29:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528636+0000", 2024-09-17T07:29:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001507836 2024-09-17T07:29:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528675+0000", 2024-09-17T07:29:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8671000000000003e-05 2024-09-17T07:29:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:5461 3.9 e33/23 3:90773955:::rbd_data.1183cff49e3a.000000000000000a:head v 33'108, mlcod=33'104)", 2024-09-17T07:29:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.527712+0000", 2024-09-17T07:29:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.624514246, 2024-09-17T07:29:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012502800000000001, 2024-09-17T07:29:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527712+0000", 2024-09-17T07:29:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527712+0000", 2024-09-17T07:29:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527712+0000", 2024-09-17T07:29:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.89e-07 2024-09-17T07:29:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527716+0000", 2024-09-17T07:29:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1520000000000002e-06 2024-09-17T07:29:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527724+0000", 2024-09-17T07:29:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9689999999999994e-06 2024-09-17T07:29:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527728+0000", 2024-09-17T07:29:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8909999999999997e-06 2024-09-17T07:29:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527776+0000", 2024-09-17T07:29:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7778999999999998e-05 2024-09-17T07:29:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527798+0000", 2024-09-17T07:29:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2558999999999999e-05 2024-09-17T07:29:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528931+0000", 2024-09-17T07:29:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001132814 2024-09-17T07:29:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528962+0000", 2024-09-17T07:29:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0626999999999997e-05 2024-09-17T07:29:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:6005 3.0 e33/23 3:037cd38f:::rbd_data.1183cff49e3a.0000000000000012:head v 33'25, mlcod=32'23)", 2024-09-17T07:29:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.527600258, 2024-09-17T07:29:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019123899999999999, 2024-09-17T07:29:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624627+0000", 2024-09-17T07:29:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5600000000000001e-06 2024-09-17T07:29:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624777+0000", 2024-09-17T07:29:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015030800000000001 2024-09-17T07:29:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624792+0000", 2024-09-17T07:29:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5089000000000001e-05 2024-09-17T07:29:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624799+0000", 2024-09-17T07:29:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7939999999999997e-06 2024-09-17T07:29:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624855+0000", 2024-09-17T07:29:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5782999999999998e-05 2024-09-17T07:29:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624871+0000", 2024-09-17T07:29:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6192e-05 2024-09-17T07:29:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.626507+0000", 2024-09-17T07:29:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016354500000000001 2024-09-17T07:29:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.626538+0000", 2024-09-17T07:29:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1214000000000003e-05 2024-09-17T07:29:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:6683 3.9 e33/23 3:97ddc208:::rbd_data.1183cff49e3a.000000000000001e:head v 33'110, mlcod=33'108)", 2024-09-17T07:29:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.764442+0000", 2024-09-17T07:29:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.387783856, 2024-09-17T07:29:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017329470000000001, 2024-09-17T07:29:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.764442+0000", 2024-09-17T07:29:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.764442+0000", 2024-09-17T07:29:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.764443+0000", 2024-09-17T07:29:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:29:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765126+0000", 2024-09-17T07:29:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00068291100000000002 2024-09-17T07:29:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765146+0000", 2024-09-17T07:29:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9406999999999999e-05 2024-09-17T07:29:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765156+0000", 2024-09-17T07:29:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0037000000000001e-05 2024-09-17T07:29:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765209+0000", 2024-09-17T07:29:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3732e-05 2024-09-17T07:29:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765226+0000", 2024-09-17T07:29:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6787e-05 2024-09-17T07:29:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.766139+0000", 2024-09-17T07:29:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00091277899999999996 2024-09-17T07:29:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.766175+0000", 2024-09-17T07:29:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5942999999999998e-05 2024-09-17T07:29:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:7670 3.4 e33/23 3:20e02957:::rbd_data.1183cff49e3a.000000000000002d:head v 33'30, mlcod=32'28)", 2024-09-17T07:29:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.955123+0000", 2024-09-17T07:29:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.197102395, 2024-09-17T07:29:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0018080839999999999, 2024-09-17T07:29:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955123+0000", 2024-09-17T07:29:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955123+0000", 2024-09-17T07:29:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955125+0000", 2024-09-17T07:29:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3060000000000001e-06 2024-09-17T07:29:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955285+0000", 2024-09-17T07:29:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00016064700000000001 2024-09-17T07:29:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955303+0000", 2024-09-17T07:29:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7465999999999998e-05 2024-09-17T07:29:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955311+0000", 2024-09-17T07:29:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4780000000000008e-06 2024-09-17T07:29:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955373+0000", 2024-09-17T07:29:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1977000000000006e-05 2024-09-17T07:29:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955388+0000", 2024-09-17T07:29:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4260000000000001e-05 2024-09-17T07:29:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.956893+0000", 2024-09-17T07:29:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001505309 2024-09-17T07:29:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.956931+0000", 2024-09-17T07:29:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8640999999999998e-05 2024-09-17T07:29:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8595 3.4 e33/23 3:237b2a73:::rbd_data.1183cff49e3a.0000000000000049:head v 33'37, mlcod=33'35)", 2024-09-17T07:29:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.077583299, 2024-09-17T07:29:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020228939999999999, 2024-09-17T07:29:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074643+0000", 2024-09-17T07:29:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3099999999999997e-07 2024-09-17T07:29:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074834+0000", 2024-09-17T07:29:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000191137 2024-09-17T07:29:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074850+0000", 2024-09-17T07:29:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6101999999999999e-05 2024-09-17T07:29:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074859+0000", 2024-09-17T07:29:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2169999999999994e-06 2024-09-17T07:29:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074919+0000", 2024-09-17T07:29:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.003e-05 2024-09-17T07:29:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074944+0000", 2024-09-17T07:29:08.202 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5761e-05 2024-09-17T07:29:08.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.202 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.202 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076622+0000", 2024-09-17T07:29:08.202 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001677977 2024-09-17T07:29:08.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.202 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.202 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076665+0000", 2024-09-17T07:29:08.202 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2938999999999997e-05 2024-09-17T07:29:08.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.202 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.203 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8596 3.4 e33/23 3:237b2a73:::rbd_data.1183cff49e3a.0000000000000049:head v 33'39, mlcod=33'35)", 2024-09-17T07:29:08.203 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.075842+0000", 2024-09-17T07:29:08.203 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.076383310000001, 2024-09-17T07:29:08.203 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012443630000000001, 2024-09-17T07:29:08.203 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.203 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.203 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075842+0000", 2024-09-17T07:29:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075842+0000", 2024-09-17T07:29:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075843+0000", 2024-09-17T07:29:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1099999999999995e-07 2024-09-17T07:29:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075848+0000", 2024-09-17T07:29:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1139999999999997e-06 2024-09-17T07:29:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075858+0000", 2024-09-17T07:29:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5920000000000007e-06 2024-09-17T07:29:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075862+0000", 2024-09-17T07:29:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5580000000000002e-06 2024-09-17T07:29:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075917+0000", 2024-09-17T07:29:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4274000000000002e-05 2024-09-17T07:29:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.075938+0000", 2024-09-17T07:29:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1693000000000001e-05 2024-09-17T07:29:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.077049+0000", 2024-09-17T07:29:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011106079999999999 2024-09-17T07:29:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.077087+0000", 2024-09-17T07:29:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7913000000000001e-05 2024-09-17T07:29:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8721 3.4 e33/23 3:207ac2b1:::rbd_data.1183cff49e3a.000000000000004a:head v 33'40, mlcod=33'37)", 2024-09-17T07:29:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.079669+0000", 2024-09-17T07:29:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.072557218, 2024-09-17T07:29:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017112749999999999, 2024-09-17T07:29:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079669+0000", 2024-09-17T07:29:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079669+0000", 2024-09-17T07:29:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079670+0000", 2024-09-17T07:29:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1540000000000001e-06 2024-09-17T07:29:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079849+0000", 2024-09-17T07:29:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017954900000000001 2024-09-17T07:29:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079864+0000", 2024-09-17T07:29:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4973e-05 2024-09-17T07:29:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079872+0000", 2024-09-17T07:29:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5170000000000002e-06 2024-09-17T07:29:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079929+0000", 2024-09-17T07:29:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7083000000000002e-05 2024-09-17T07:29:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.079959+0000", 2024-09-17T07:29:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0474e-05 2024-09-17T07:29:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.081348+0000", 2024-09-17T07:29:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001388481 2024-09-17T07:29:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.081380+0000", 2024-09-17T07:29:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2044e-05 2024-09-17T07:29:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8872 3.0 e34/23 3:042b6c8e:::rbd_data.1183cff49e3a.0000000000000058:head v 34'31, mlcod=33'29)", 2024-09-17T07:29:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.322367+0000", 2024-09-17T07:29:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 12.829858343, 2024-09-17T07:29:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016244619999999999, 2024-09-17T07:29:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322367+0000", 2024-09-17T07:29:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322367+0000", 2024-09-17T07:29:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322368+0000", 2024-09-17T07:29:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.16e-07 2024-09-17T07:29:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322539+0000", 2024-09-17T07:29:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017174200000000001 2024-09-17T07:29:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322554+0000", 2024-09-17T07:29:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4637000000000001e-05 2024-09-17T07:29:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322563+0000", 2024-09-17T07:29:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2369999999999998e-06 2024-09-17T07:29:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322622+0000", 2024-09-17T07:29:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8928000000000002e-05 2024-09-17T07:29:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.322641+0000", 2024-09-17T07:29:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8601e-05 2024-09-17T07:29:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.323955+0000", 2024-09-17T07:29:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013146410000000001 2024-09-17T07:29:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.323992+0000", 2024-09-17T07:29:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.646e-05 2024-09-17T07:29:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:15683 3.0 e39/23 3:0518bbb2:::rbd_object_map.1183515b0721:head v 39'90, mlcod=39'84)", 2024-09-17T07:29:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:00.635895+0000", 2024-09-17T07:29:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 7.5163304560000004, 2024-09-17T07:29:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001403357, 2024-09-17T07:29:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635895+0000", 2024-09-17T07:29:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635895+0000", 2024-09-17T07:29:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635896+0000", 2024-09-17T07:29:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9999999999999995e-07 2024-09-17T07:29:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635905+0000", 2024-09-17T07:29:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6510000000000007e-06 2024-09-17T07:29:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635918+0000", 2024-09-17T07:29:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3168e-05 2024-09-17T07:29:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.635931+0000", 2024-09-17T07:29:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2723e-05 2024-09-17T07:29:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.220 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.220 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.636183+0000", 2024-09-17T07:29:08.220 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00025214999999999998 2024-09-17T07:29:08.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.220 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.220 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.636210+0000", 2024-09-17T07:29:08.220 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6667000000000001e-05 2024-09-17T07:29:08.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.637240+0000", 2024-09-17T07:29:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010305889999999999 2024-09-17T07:29:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:00.637299+0000", 2024-09-17T07:29:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8409000000000003e-05 2024-09-17T07:29:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17069 3.0 e41/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 41'110, mlcod=40'108)", 2024-09-17T07:29:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.4124735749999999, 2024-09-17T07:29:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011408908000000001, 2024-09-17T07:29:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739753+0000", 2024-09-17T07:29:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:29:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743532+0000", 2024-09-17T07:29:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037794510000000001 2024-09-17T07:29:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743557+0000", 2024-09-17T07:29:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5253e-05 2024-09-17T07:29:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743569+0000", 2024-09-17T07:29:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1161999999999999e-05 2024-09-17T07:29:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743650+0000", 2024-09-17T07:29:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1087999999999993e-05 2024-09-17T07:29:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743674+0000", 2024-09-17T07:29:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4562e-05 2024-09-17T07:29:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751095+0000", 2024-09-17T07:29:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074211140000000004 2024-09-17T07:29:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751161+0000", 2024-09-17T07:29:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5707e-05 2024-09-17T07:29:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17240 3.0 e42/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 42'113, mlcod=41'110)", 2024-09-17T07:29:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.9268150909999999, 2024-09-17T07:29:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064392440000000002, 2024-09-17T07:29:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:29:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.228 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.228 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225418+0000", 2024-09-17T07:29:08.228 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8360000000000003e-06 2024-09-17T07:29:08.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.228 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.228 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225426+0000", 2024-09-17T07:29:08.228 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2670000000000006e-06 2024-09-17T07:29:08.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225434+0000", 2024-09-17T07:29:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0560000000000005e-06 2024-09-17T07:29:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225514+0000", 2024-09-17T07:29:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9979000000000001e-05 2024-09-17T07:29:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225535+0000", 2024-09-17T07:29:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0965e-05 2024-09-17T07:29:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231797+0000", 2024-09-17T07:29:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062621129999999997 2024-09-17T07:29:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231850+0000", 2024-09-17T07:29:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2611999999999997e-05 2024-09-17T07:29:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20054 3.9 e44/23 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:head v 44'131, mlcod=44'129)", 2024-09-17T07:29:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.884663975, 2024-09-17T07:29:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028510570000000002, 2024-09-17T07:29:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267563+0000", 2024-09-17T07:29:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.584e-06 2024-09-17T07:29:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268269+0000", 2024-09-17T07:29:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00070567399999999999 2024-09-17T07:29:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268287+0000", 2024-09-17T07:29:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.84e-05 2024-09-17T07:29:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268297+0000", 2024-09-17T07:29:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9059999999999997e-06 2024-09-17T07:29:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268368+0000", 2024-09-17T07:29:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0562999999999995e-05 2024-09-17T07:29:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268393+0000", 2024-09-17T07:29:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4958999999999999e-05 2024-09-17T07:29:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270330+0000", 2024-09-17T07:29:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001936644 2024-09-17T07:29:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270413+0000", 2024-09-17T07:29:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3326999999999997e-05 2024-09-17T07:29:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20055 3.9 e44/23 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:head v 44'133, mlcod=44'129)", 2024-09-17T07:29:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.268886+0000", 2024-09-17T07:29:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.8833399040000001, 2024-09-17T07:29:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019232920000000001, 2024-09-17T07:29:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268886+0000", 2024-09-17T07:29:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268886+0000", 2024-09-17T07:29:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268887+0000", 2024-09-17T07:29:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.85e-07 2024-09-17T07:29:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268905+0000", 2024-09-17T07:29:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8263000000000001e-05 2024-09-17T07:29:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268916+0000", 2024-09-17T07:29:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0876000000000001e-05 2024-09-17T07:29:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268922+0000", 2024-09-17T07:29:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7579999999999998e-06 2024-09-17T07:29:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.269201+0000", 2024-09-17T07:29:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027979200000000001 2024-09-17T07:29:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.269224+0000", 2024-09-17T07:29:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2838999999999999e-05 2024-09-17T07:29:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270774+0000", 2024-09-17T07:29:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015495629999999999 2024-09-17T07:29:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270809+0000", 2024-09-17T07:29:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5416000000000001e-05 2024-09-17T07:29:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20163 3.9 e44/23 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:head v 44'134, mlcod=44'131)", 2024-09-17T07:29:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.294194+0000", 2024-09-17T07:29:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.858031553, 2024-09-17T07:29:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001874824, 2024-09-17T07:29:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294194+0000", 2024-09-17T07:29:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294194+0000", 2024-09-17T07:29:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294196+0000", 2024-09-17T07:29:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3599999999999999e-06 2024-09-17T07:29:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294663+0000", 2024-09-17T07:29:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00046696200000000002 2024-09-17T07:29:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294683+0000", 2024-09-17T07:29:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0829999999999999e-05 2024-09-17T07:29:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294697+0000", 2024-09-17T07:29:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3524e-05 2024-09-17T07:29:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294758+0000", 2024-09-17T07:29:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0618999999999997e-05 2024-09-17T07:29:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294776+0000", 2024-09-17T07:29:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8431e-05 2024-09-17T07:29:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.296042+0000", 2024-09-17T07:29:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012656329999999999 2024-09-17T07:29:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.296069+0000", 2024-09-17T07:29:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7464999999999999e-05 2024-09-17T07:29:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:08.244 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:08.245 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:08.246 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:08.246+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:08.268 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone13 from 0xd4517bd (overlap 0xc86e6) to 0xb4635 2024-09-17T07:29:08.348 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:08.347+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:08.362 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:08.362 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:08.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.362 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:26377 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b out=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.363 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.362123+0000", 2024-09-17T07:29:08.363 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.00021438499999999999, 2024-09-17T07:29:08.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021676, 2024-09-17T07:29:08.363 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.363 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.363 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.363 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.363 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.363 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.363 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 26377 2024-09-17T07:29:08.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.363 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.362123+0000", 2024-09-17T07:29:08.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.362123+0000", 2024-09-17T07:29:08.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.362124+0000", 2024-09-17T07:29:08.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8400000000000003e-07 2024-09-17T07:29:08.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.362129+0000", 2024-09-17T07:29:08.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8620000000000002e-06 2024-09-17T07:29:08.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.362142+0000", 2024-09-17T07:29:08.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3081000000000001e-05 2024-09-17T07:29:08.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.362165+0000", 2024-09-17T07:29:08.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3819e-05 2024-09-17T07:29:08.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.362237+0000", 2024-09-17T07:29:08.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1487000000000002e-05 2024-09-17T07:29:08.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.362283+0000", 2024-09-17T07:29:08.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5692000000000002e-05 2024-09-17T07:29:08.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.362340+0000", 2024-09-17T07:29:08.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6935000000000003e-05 2024-09-17T07:29:08.367 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.367 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.367 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.367 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.367 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-09-17T07:29:08.367 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 1 2024-09-17T07:29:08.367 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:08.371 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:08.381 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-17T07:29:08.452 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:08.451+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:08.523 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:08.524 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:08.533 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:08.574 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:08.573+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:08.627 INFO:teuthology.orchestra.run.smithi045.stdout:1370 writesame 0x7cf3db2 thru 0x7d00b16 data_size 0x11(0xcd65 bytes) 2024-09-17T07:29:08.648 INFO:teuthology.orchestra.run.smithi045.stdout:1371 trunc from 0xd4517bd to 0x28d2d56 2024-09-17T07:29:08.651 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:08.651+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:08.673 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:08.674 INFO:teuthology.orchestra.run.smithi045.stdout:1373 read 0x10e4949 thru 0x10ec570 (0x7c28 bytes) 2024-09-17T07:29:08.674 INFO:teuthology.orchestra.run.smithi045.stdout:1374 punch from 0x251947e to 0x2525d6f, (0xc8f1 bytes) 2024-09-17T07:29:08.674 INFO:teuthology.orchestra.run.smithi045.stdout:1376 trunc from 0x28d2d56 to 0x1ce437b 2024-09-17T07:29:08.691 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:6683 3.9 3:97ddc208:::rbd_data.1183cff49e3a.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.763526+0000", 2024-09-17T07:29:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.925731141, 2024-09-17T07:29:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032433677000000001, 2024-09-17T07:29:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6683 2024-09-17T07:29:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.763526+0000", 2024-09-17T07:29:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.763526+0000", 2024-09-17T07:29:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.763527+0000", 2024-09-17T07:29:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.78e-07 2024-09-17T07:29:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764048+0000", 2024-09-17T07:29:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00052137799999999997 2024-09-17T07:29:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764060+0000", 2024-09-17T07:29:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2014000000000001e-05 2024-09-17T07:29:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764072+0000", 2024-09-17T07:29:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1762e-05 2024-09-17T07:29:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764130+0000", 2024-09-17T07:29:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7759e-05 2024-09-17T07:29:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764194+0000", 2024-09-17T07:29:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4189999999999994e-05 2024-09-17T07:29:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-17T07:29:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764332+0000", 2024-09-17T07:29:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013821300000000001 2024-09-17T07:29:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795822+0000", 2024-09-17T07:29:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031489847000000001 2024-09-17T07:29:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795858+0000", 2024-09-17T07:29:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6452999999999998e-05 2024-09-17T07:29:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795917+0000", 2024-09-17T07:29:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8851999999999998e-05 2024-09-17T07:29:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795959+0000", 2024-09-17T07:29:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2231000000000003e-05 2024-09-17T07:29:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:6684 3.9 3:97ddc208:::rbd_data.1183cff49e3a.000000000000001e:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.764091+0000", 2024-09-17T07:29:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.925165958000001, 2024-09-17T07:29:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032300207999999997, 2024-09-17T07:29:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6684 2024-09-17T07:29:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764091+0000", 2024-09-17T07:29:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764091+0000", 2024-09-17T07:29:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764092+0000", 2024-09-17T07:29:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8700000000000005e-07 2024-09-17T07:29:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764094+0000", 2024-09-17T07:29:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9189999999999999e-06 2024-09-17T07:29:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764102+0000", 2024-09-17T07:29:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2710000000000003e-06 2024-09-17T07:29:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764118+0000", 2024-09-17T07:29:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6231000000000001e-05 2024-09-17T07:29:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764995+0000", 2024-09-17T07:29:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00087753699999999996 2024-09-17T07:29:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.765094+0000", 2024-09-17T07:29:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8027000000000002e-05 2024-09-17T07:29:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-17T07:29:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794998+0000", 2024-09-17T07:29:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029904612000000001 2024-09-17T07:29:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796267+0000", 2024-09-17T07:29:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001268911 2024-09-17T07:29:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796328+0000", 2024-09-17T07:29:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0594000000000003e-05 2024-09-17T07:29:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796356+0000", 2024-09-17T07:29:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8787999999999999e-05 2024-09-17T07:29:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796391+0000", 2024-09-17T07:29:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4631000000000001e-05 2024-09-17T07:29:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4483.0:6683 3.9 e33/23 ondisk, result = 0)", 2024-09-17T07:29:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:29:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.923018162, 2024-09-17T07:29:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029728505999999998, 2024-09-17T07:29:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:29:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:29:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:29:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9400000000000004e-07 2024-09-17T07:29:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766243+0000", 2024-09-17T07:29:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9419999999999997e-06 2024-09-17T07:29:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794812+0000", 2024-09-17T07:29:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028568324999999999 2024-09-17T07:29:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794822+0000", 2024-09-17T07:29:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0786e-05 2024-09-17T07:29:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795847+0000", 2024-09-17T07:29:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010248919999999999 2024-09-17T07:29:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795854+0000", 2024-09-17T07:29:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4919999999999998e-06 2024-09-17T07:29:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795967+0000", 2024-09-17T07:29:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113275 2024-09-17T07:29:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:6685 3.14 3:2a16c27c:::rbd_object_map.1183cff49e3a.0000000000000007:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.770642+0000", 2024-09-17T07:29:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.918615260999999, 2024-09-17T07:29:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027608490999999999, 2024-09-17T07:29:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6685 2024-09-17T07:29:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770642+0000", 2024-09-17T07:29:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770642+0000", 2024-09-17T07:29:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770643+0000", 2024-09-17T07:29:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.824e-06 2024-09-17T07:29:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770651+0000", 2024-09-17T07:29:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9680000000000006e-06 2024-09-17T07:29:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794795+0000", 2024-09-17T07:29:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024143286 2024-09-17T07:29:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794811+0000", 2024-09-17T07:29:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6112000000000001e-05 2024-09-17T07:29:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794961+0000", 2024-09-17T07:29:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014987 2024-09-17T07:29:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795048+0000", 2024-09-17T07:29:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7740999999999999e-05 2024-09-17T07:29:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795317+0000", 2024-09-17T07:29:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026868500000000001 2024-09-17T07:29:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.797274+0000", 2024-09-17T07:29:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019571660000000002 2024-09-17T07:29:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.798166+0000", 2024-09-17T07:29:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00089202400000000005 2024-09-17T07:29:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.798212+0000", 2024-09-17T07:29:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.596e-05 2024-09-17T07:29:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.798250+0000", 2024-09-17T07:29:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7855000000000003e-05 2024-09-17T07:29:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27151 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405408+0000", 2024-09-17T07:29:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.28384915999999999, 2024-09-17T07:29:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023090975, 2024-09-17T07:29:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27151 2024-09-17T07:29:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405408+0000", 2024-09-17T07:29:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405408+0000", 2024-09-17T07:29:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405408+0000", 2024-09-17T07:29:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-17T07:29:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405411+0000", 2024-09-17T07:29:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5490000000000001e-06 2024-09-17T07:29:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405417+0000", 2024-09-17T07:29:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7379999999999997e-06 2024-09-17T07:29:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405427+0000", 2024-09-17T07:29:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4409999999999999e-06 2024-09-17T07:29:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428213+0000", 2024-09-17T07:29:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.022786130000000002 2024-09-17T07:29:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428286+0000", 2024-09-17T07:29:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3261000000000002e-05 2024-09-17T07:29:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428499+0000", 2024-09-17T07:29:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021233600000000001 2024-09-17T07:29:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27152 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405434+0000", 2024-09-17T07:29:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.28382230600000002, 2024-09-17T07:29:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023382574, 2024-09-17T07:29:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27152 2024-09-17T07:29:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405434+0000", 2024-09-17T07:29:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405434+0000", 2024-09-17T07:29:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405435+0000", 2024-09-17T07:29:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7899999999999999e-07 2024-09-17T07:29:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405437+0000", 2024-09-17T07:29:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.108e-06 2024-09-17T07:29:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405444+0000", 2024-09-17T07:29:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4829999999999996e-06 2024-09-17T07:29:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405448+0000", 2024-09-17T07:29:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4669999999999998e-06 2024-09-17T07:29:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428527+0000", 2024-09-17T07:29:08.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023079183 2024-09-17T07:29:08.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428601+0000", 2024-09-17T07:29:08.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3385000000000003e-05 2024-09-17T07:29:08.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428817+0000", 2024-09-17T07:29:08.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000216469 2024-09-17T07:29:08.726 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.726 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.726 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.726 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27153 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.726 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405455+0000", 2024-09-17T07:29:08.726 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.28380215800000003, 2024-09-17T07:29:08.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023682372, 2024-09-17T07:29:08.726 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.726 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.726 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.726 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.727 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.727 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.727 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27153 2024-09-17T07:29:08.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.727 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405455+0000", 2024-09-17T07:29:08.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405455+0000", 2024-09-17T07:29:08.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405455+0000", 2024-09-17T07:29:08.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-09-17T07:29:08.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405457+0000", 2024-09-17T07:29:08.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1040000000000001e-06 2024-09-17T07:29:08.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405464+0000", 2024-09-17T07:29:08.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7050000000000002e-06 2024-09-17T07:29:08.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405469+0000", 2024-09-17T07:29:08.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9019999999999998e-06 2024-09-17T07:29:08.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428847+0000", 2024-09-17T07:29:08.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023378304999999999 2024-09-17T07:29:08.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428920+0000", 2024-09-17T07:29:08.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-05 2024-09-17T07:29:08.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429137+0000", 2024-09-17T07:29:08.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217386 2024-09-17T07:29:08.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.730 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.731 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27154 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.731 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405475+0000", 2024-09-17T07:29:08.731 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.28378175100000003, 2024-09-17T07:29:08.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023995657, 2024-09-17T07:29:08.731 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.731 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.731 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.731 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27154 2024-09-17T07:29:08.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.732 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405475+0000", 2024-09-17T07:29:08.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405475+0000", 2024-09-17T07:29:08.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405475+0000", 2024-09-17T07:29:08.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-17T07:29:08.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405477+0000", 2024-09-17T07:29:08.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.883e-06 2024-09-17T07:29:08.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405484+0000", 2024-09-17T07:29:08.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2990000000000002e-06 2024-09-17T07:29:08.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405488+0000", 2024-09-17T07:29:08.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6659999999999999e-06 2024-09-17T07:29:08.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429166+0000", 2024-09-17T07:29:08.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023677404999999999 2024-09-17T07:29:08.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429238+0000", 2024-09-17T07:29:08.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2570999999999993e-05 2024-09-17T07:29:08.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429471+0000", 2024-09-17T07:29:08.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023245300000000001 2024-09-17T07:29:08.735 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.735 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.735 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.735 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27155 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.735 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405494+0000", 2024-09-17T07:29:08.735 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.28376254000000001, 2024-09-17T07:29:08.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024292818000000001, 2024-09-17T07:29:08.736 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.736 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.736 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.736 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.736 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.736 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.736 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27155 2024-09-17T07:29:08.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.736 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405494+0000", 2024-09-17T07:29:08.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405494+0000", 2024-09-17T07:29:08.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405495+0000", 2024-09-17T07:29:08.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:08.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405497+0000", 2024-09-17T07:29:08.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-09-17T07:29:08.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405503+0000", 2024-09-17T07:29:08.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4849999999999997e-06 2024-09-17T07:29:08.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405508+0000", 2024-09-17T07:29:08.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1290000000000004e-06 2024-09-17T07:29:08.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429499+0000", 2024-09-17T07:29:08.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023990592000000002 2024-09-17T07:29:08.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429571+0000", 2024-09-17T07:29:08.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2199000000000004e-05 2024-09-17T07:29:08.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429787+0000", 2024-09-17T07:29:08.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000216126 2024-09-17T07:29:08.740 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.740 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.740 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.740 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27156 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.740 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405515+0000", 2024-09-17T07:29:08.740 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.28374214199999997, 2024-09-17T07:29:08.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024640863999999998, 2024-09-17T07:29:08.740 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.741 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.741 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.741 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.741 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.741 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.741 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27156 2024-09-17T07:29:08.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.741 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405515+0000", 2024-09-17T07:29:08.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405515+0000", 2024-09-17T07:29:08.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405515+0000", 2024-09-17T07:29:08.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-09-17T07:29:08.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405524+0000", 2024-09-17T07:29:08.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6619999999999994e-06 2024-09-17T07:29:08.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405531+0000", 2024-09-17T07:29:08.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8669999999999996e-06 2024-09-17T07:29:08.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405535+0000", 2024-09-17T07:29:08.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6539999999999998e-06 2024-09-17T07:29:08.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429814+0000", 2024-09-17T07:29:08.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024279207000000001 2024-09-17T07:29:08.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429886+0000", 2024-09-17T07:29:08.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1729000000000004e-05 2024-09-17T07:29:08.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430156+0000", 2024-09-17T07:29:08.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269335 2024-09-17T07:29:08.744 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.745 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.745 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.745 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27158 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.745 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405542+0000", 2024-09-17T07:29:08.745 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.283714833, 2024-09-17T07:29:08.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024918612999999999, 2024-09-17T07:29:08.745 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.745 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.745 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.746 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27158 2024-09-17T07:29:08.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.746 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405542+0000", 2024-09-17T07:29:08.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405542+0000", 2024-09-17T07:29:08.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405542+0000", 2024-09-17T07:29:08.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3799999999999998e-07 2024-09-17T07:29:08.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405544+0000", 2024-09-17T07:29:08.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.812e-06 2024-09-17T07:29:08.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405551+0000", 2024-09-17T07:29:08.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5989999999999998e-06 2024-09-17T07:29:08.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405556+0000", 2024-09-17T07:29:08.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8060000000000002e-06 2024-09-17T07:29:08.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430183+0000", 2024-09-17T07:29:08.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024627860000000001 2024-09-17T07:29:08.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430258+0000", 2024-09-17T07:29:08.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4516000000000003e-05 2024-09-17T07:29:08.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430461+0000", 2024-09-17T07:29:08.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020258200000000001 2024-09-17T07:29:08.749 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.749 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.749 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.749 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27160 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.749 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405562+0000", 2024-09-17T07:29:08.750 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.28369506100000003, 2024-09-17T07:29:08.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025242179999999999, 2024-09-17T07:29:08.750 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.750 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.750 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.750 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.750 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.750 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.750 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27160 2024-09-17T07:29:08.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.750 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405562+0000", 2024-09-17T07:29:08.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405562+0000", 2024-09-17T07:29:08.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405562+0000", 2024-09-17T07:29:08.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6199999999999998e-07 2024-09-17T07:29:08.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405564+0000", 2024-09-17T07:29:08.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.765e-06 2024-09-17T07:29:08.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405570+0000", 2024-09-17T07:29:08.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3439999999999997e-06 2024-09-17T07:29:08.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405575+0000", 2024-09-17T07:29:08.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7639999999999996e-06 2024-09-17T07:29:08.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430505+0000", 2024-09-17T07:29:08.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024930101999999999 2024-09-17T07:29:08.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430584+0000", 2024-09-17T07:29:08.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8479999999999994e-05 2024-09-17T07:29:08.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430804+0000", 2024-09-17T07:29:08.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000220263 2024-09-17T07:29:08.754 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.754 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.754 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.754 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27162 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.754 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405581+0000", 2024-09-17T07:29:08.754 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.28367555999999999, 2024-09-17T07:29:08.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025527253999999999, 2024-09-17T07:29:08.754 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.754 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.754 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.755 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.755 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.755 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27162 2024-09-17T07:29:08.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.755 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405581+0000", 2024-09-17T07:29:08.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405581+0000", 2024-09-17T07:29:08.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405582+0000", 2024-09-17T07:29:08.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-07 2024-09-17T07:29:08.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405584+0000", 2024-09-17T07:29:08.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:29:08.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405590+0000", 2024-09-17T07:29:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3820000000000001e-06 2024-09-17T07:29:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405595+0000", 2024-09-17T07:29:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:29:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430832+0000", 2024-09-17T07:29:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025237654000000002 2024-09-17T07:29:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430905+0000", 2024-09-17T07:29:08.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2234e-05 2024-09-17T07:29:08.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.431108+0000", 2024-09-17T07:29:08.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020389200000000001 2024-09-17T07:29:08.758 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.758 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.758 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.759 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27165 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.759 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405601+0000", 2024-09-17T07:29:08.759 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.28365605300000002, 2024-09-17T07:29:08.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025836962000000002, 2024-09-17T07:29:08.759 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.759 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.759 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.759 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.759 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.759 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.759 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27165 2024-09-17T07:29:08.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.759 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405601+0000", 2024-09-17T07:29:08.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405601+0000", 2024-09-17T07:29:08.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405601+0000", 2024-09-17T07:29:08.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:29:08.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405603+0000", 2024-09-17T07:29:08.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.658e-06 2024-09-17T07:29:08.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405609+0000", 2024-09-17T07:29:08.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1689999999999997e-06 2024-09-17T07:29:08.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405614+0000", 2024-09-17T07:29:08.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8590000000000004e-06 2024-09-17T07:29:08.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.431136+0000", 2024-09-17T07:29:08.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025522382999999999 2024-09-17T07:29:08.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.431207+0000", 2024-09-17T07:29:08.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1076999999999997e-05 2024-09-17T07:29:08.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.431438+0000", 2024-09-17T07:29:08.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023040499999999999 2024-09-17T07:29:08.763 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.763 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.763 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.763 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27167 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.763 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405620+0000", 2024-09-17T07:29:08.763 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.283636791, 2024-09-17T07:29:08.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026157786999999998, 2024-09-17T07:29:08.763 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.764 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.764 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.764 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27167 2024-09-17T07:29:08.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.764 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405620+0000", 2024-09-17T07:29:08.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405620+0000", 2024-09-17T07:29:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405620+0000", 2024-09-17T07:29:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:29:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405629+0000", 2024-09-17T07:29:08.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0760000000000003e-06 2024-09-17T07:29:08.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405635+0000", 2024-09-17T07:29:08.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5919999999999997e-06 2024-09-17T07:29:08.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405640+0000", 2024-09-17T07:29:08.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8470000000000003e-06 2024-09-17T07:29:08.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.431468+0000", 2024-09-17T07:29:08.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025828002999999999 2024-09-17T07:29:08.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.431556+0000", 2024-09-17T07:29:08.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8213000000000004e-05 2024-09-17T07:29:08.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.431778+0000", 2024-09-17T07:29:08.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022160600000000001 2024-09-17T07:29:08.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.767 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.768 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27624 3.9 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:e [read 163840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.768 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.412477+0000", 2024-09-17T07:29:08.768 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.27678016999999999, 2024-09-17T07:29:08.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023034016000000001, 2024-09-17T07:29:08.768 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.768 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.768 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.769 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27624 2024-09-17T07:29:08.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.769 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412477+0000", 2024-09-17T07:29:08.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412477+0000", 2024-09-17T07:29:08.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412477+0000", 2024-09-17T07:29:08.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:29:08.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412479+0000", 2024-09-17T07:29:08.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7999999999999999e-06 2024-09-17T07:29:08.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412485+0000", 2024-09-17T07:29:08.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5740000000000001e-06 2024-09-17T07:29:08.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412490+0000", 2024-09-17T07:29:08.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9049999999999996e-06 2024-09-17T07:29:08.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.435238+0000", 2024-09-17T07:29:08.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.022747499000000001 2024-09-17T07:29:08.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.435311+0000", 2024-09-17T07:29:08.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3558000000000003e-05 2024-09-17T07:29:08.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.435511+0000", 2024-09-17T07:29:08.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00019924499999999999 2024-09-17T07:29:08.772 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.772 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.772 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.772 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27628 3.9 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:e [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.773 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.412497+0000", 2024-09-17T07:29:08.773 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.27676025799999998, 2024-09-17T07:29:08.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023335651999999998, 2024-09-17T07:29:08.773 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.773 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.773 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.773 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.773 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.773 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.773 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27628 2024-09-17T07:29:08.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.773 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412497+0000", 2024-09-17T07:29:08.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412497+0000", 2024-09-17T07:29:08.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412497+0000", 2024-09-17T07:29:08.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3000000000000001e-07 2024-09-17T07:29:08.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412499+0000", 2024-09-17T07:29:08.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7230000000000001e-06 2024-09-17T07:29:08.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412505+0000", 2024-09-17T07:29:08.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3969999999999999e-06 2024-09-17T07:29:08.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412510+0000", 2024-09-17T07:29:08.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9429999999999999e-06 2024-09-17T07:29:08.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.435538+0000", 2024-09-17T07:29:08.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023028090000000001 2024-09-17T07:29:08.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.435606+0000", 2024-09-17T07:29:08.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8303e-05 2024-09-17T07:29:08.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.435832+0000", 2024-09-17T07:29:08.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022576599999999999 2024-09-17T07:29:08.777 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.777 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.777 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.777 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27629 3.9 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:e [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.777 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.412516+0000", 2024-09-17T07:29:08.777 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.27674122800000001, 2024-09-17T07:29:08.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023701032, 2024-09-17T07:29:08.778 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.778 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.778 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.778 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.778 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.778 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.778 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27629 2024-09-17T07:29:08.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.778 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412516+0000", 2024-09-17T07:29:08.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412516+0000", 2024-09-17T07:29:08.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412516+0000", 2024-09-17T07:29:08.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9400000000000001e-07 2024-09-17T07:29:08.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412518+0000", 2024-09-17T07:29:08.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6589999999999999e-06 2024-09-17T07:29:08.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412524+0000", 2024-09-17T07:29:08.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3960000000000003e-06 2024-09-17T07:29:08.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412529+0000", 2024-09-17T07:29:08.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9609999999999996e-06 2024-09-17T07:29:08.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.435858+0000", 2024-09-17T07:29:08.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.02332934 2024-09-17T07:29:08.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.435929+0000", 2024-09-17T07:29:08.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0838000000000002e-05 2024-09-17T07:29:08.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.436217+0000", 2024-09-17T07:29:08.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028744399999999998 2024-09-17T07:29:08.782 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.782 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.782 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.782 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27630 3.9 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:e [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.782 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.412535+0000", 2024-09-17T07:29:08.782 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.27672172900000003, 2024-09-17T07:29:08.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024043742999999999, 2024-09-17T07:29:08.782 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.782 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.782 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.783 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.783 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27630 2024-09-17T07:29:08.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.783 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412535+0000", 2024-09-17T07:29:08.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412535+0000", 2024-09-17T07:29:08.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412536+0000", 2024-09-17T07:29:08.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3300000000000003e-07 2024-09-17T07:29:08.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412537+0000", 2024-09-17T07:29:08.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.877e-06 2024-09-17T07:29:08.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412544+0000", 2024-09-17T07:29:08.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3110000000000002e-06 2024-09-17T07:29:08.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412551+0000", 2024-09-17T07:29:08.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4420000000000001e-06 2024-09-17T07:29:08.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.436243+0000", 2024-09-17T07:29:08.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023692273 2024-09-17T07:29:08.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.436313+0000", 2024-09-17T07:29:08.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9388999999999996e-05 2024-09-17T07:29:08.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.436579+0000", 2024-09-17T07:29:08.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026601800000000003 2024-09-17T07:29:08.786 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.786 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.786 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.787 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27631 3.9 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:e [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:08.787 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.412557+0000", 2024-09-17T07:29:08.787 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.276699325, 2024-09-17T07:29:08.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024539926, 2024-09-17T07:29:08.787 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:08.787 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:08.787 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:08.787 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:08.787 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:08.787 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:08.787 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27631 2024-09-17T07:29:08.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.787 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:08.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:08.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412557+0000", 2024-09-17T07:29:08.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:08.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412557+0000", 2024-09-17T07:29:08.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:08.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:08.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412558+0000", 2024-09-17T07:29:08.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4300000000000003e-07 2024-09-17T07:29:08.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:08.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.418230+0000", 2024-09-17T07:29:08.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0056723199999999998 2024-09-17T07:29:08.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:08.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.419206+0000", 2024-09-17T07:29:08.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00097553099999999997 2024-09-17T07:29:08.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:08.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.419213+0000", 2024-09-17T07:29:08.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:29:08.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:08.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.436606+0000", 2024-09-17T07:29:08.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.017392364 2024-09-17T07:29:08.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:08.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.436675+0000", 2024-09-17T07:29:08.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9086999999999998e-05 2024-09-17T07:29:08.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:08.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:08.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:08.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.437097+0000", 2024-09-17T07:29:08.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00042249799999999999 2024-09-17T07:29:08.791 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.791 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.791 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.791 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:08.791 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:08.791 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:08.793 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:08.795 INFO:teuthology.orchestra.run.smithi045.stdout:1378 trunc from 0x1ce437b to 0x223eec 2024-09-17T07:29:08.795 INFO:teuthology.orchestra.run.smithi045.stdout:1379 read 0x83d21 thru 0x83ffd (0x2dd bytes) 2024-09-17T07:29:08.795 INFO:teuthology.orchestra.run.smithi045.stdout:1380 read 0x214453 thru 0x2179e6 (0x3594 bytes) 2024-09-17T07:29:08.795 INFO:teuthology.orchestra.run.smithi045.stdout:1381 trunc from 0x223eec to 0x4dd8c8 2024-09-17T07:29:08.795 INFO:teuthology.orchestra.run.smithi045.stdout:1382 read 0x26abeb thru 0x27834c (0xd762 bytes) 2024-09-17T07:29:08.795 INFO:teuthology.orchestra.run.smithi045.stdout:1383 read 0x2c7bfd thru 0x2d7406 (0xf80a bytes) 2024-09-17T07:29:08.795 INFO:teuthology.orchestra.run.smithi045.stdout:1387 write 0xb159c11 thru 0xb164ee8 (0xb2d8 bytes) 2024-09-17T07:29:08.795 INFO:teuthology.orchestra.run.smithi045.stdout:1388 writesame 0xc58636b thru 0xc5910c8 data_size 0x27(0xad5e bytes) 2024-09-17T07:29:08.795 INFO:teuthology.orchestra.run.smithi045.stdout:1390 trunc from 0xc5910c9 to 0xdd2e128 2024-09-17T07:29:08.795 INFO:teuthology.orchestra.run.smithi045.stdout:1391 write 0x71d82b9 thru 0x71e42a6 (0xbfee bytes) 2024-09-17T07:29:08.795 INFO:teuthology.orchestra.run.smithi045.stdout:1392 punch from 0x7fa9524 to 0x7fadbb6, (0x4692 bytes) 2024-09-17T07:29:08.796 INFO:teuthology.orchestra.run.smithi045.stdout:1393 trunc from 0xdd2e128 to 0x58ade26 2024-09-17T07:29:08.796 INFO:teuthology.orchestra.run.smithi045.stdout:1395 punch from 0x1aef830 to 0x1afe9ee, (0xf1be bytes) 2024-09-17T07:29:08.796 INFO:teuthology.orchestra.run.smithi045.stdout:1396 punch from 0x1e78ef1 to 0x1e78ef8, (0x7 bytes) 2024-09-17T07:29:08.796 INFO:teuthology.orchestra.run.smithi045.stdout:1398 punch from 0x5522398 to 0x552df3c, (0xbba4 bytes) 2024-09-17T07:29:08.796 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:08.796 INFO:teuthology.orchestra.run.smithi045.stdout:1406 flatten 2024-09-17T07:29:08.796 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:08.792+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:08.798 INFO:teuthology.orchestra.run.smithi045.stdout:1407 read 0x1f79545 thru 0x1f7a73e (0x11fa bytes) 2024-09-17T07:29:08.798 INFO:teuthology.orchestra.run.smithi045.stdout:1408 read 0x4d3184f thru 0x4d32e65 (0x1617 bytes) 2024-09-17T07:29:08.798 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:08.798 INFO:teuthology.orchestra.run.smithi045.stdout:1411 punch from 0x2cbe682 to 0x2cbe6e5, (0x63 bytes) 2024-09-17T07:29:08.798 INFO:teuthology.orchestra.run.smithi045.stdout:1412 read 0x48ab2c thru 0x49920d (0xe6e2 bytes) 2024-09-17T07:29:08.805 INFO:teuthology.orchestra.run.smithi045.stdout:1414 write 0x62a1950 thru 0x62a2318 (0x9c9 bytes) 2024-09-17T07:29:08.819 INFO:teuthology.orchestra.run.smithi045.stdout:1419 write 0xb3c5a22 thru 0xb3c5af8 (0xd7 bytes) 2024-09-17T07:29:08.822 INFO:teuthology.orchestra.run.smithi045.stdout:1420 write 0x947bf75 thru 0x9489c79 (0xdd05 bytes) 2024-09-17T07:29:08.827 INFO:teuthology.orchestra.run.smithi045.stdout:1421 punch from 0x83bd3d2 to 0x83c7421, (0xa04f bytes) 2024-09-17T07:29:08.827 INFO:teuthology.orchestra.run.smithi045.stdout:1422 read 0x520b45e thru 0x5214e09 (0x99ac bytes) 2024-09-17T07:29:08.827 INFO:teuthology.orchestra.run.smithi045.stdout:1424 writesame 0x19b9677 thru 0x19be106 data_size 0x8(0x4a90 bytes) 2024-09-17T07:29:08.832 INFO:teuthology.orchestra.run.smithi045.stdout:1425 write 0x6299033 thru 0x629f224 (0x61f2 bytes) 2024-09-17T07:29:08.835 INFO:teuthology.orchestra.run.smithi045.stdout:1426 read 0x2db6a25 thru 0x2db78b5 (0xe91 bytes) 2024-09-17T07:29:08.835 INFO:teuthology.orchestra.run.smithi045.stdout:1428 read 0x5002228 thru 0x50099cd (0x77a6 bytes) 2024-09-17T07:29:08.842 INFO:teuthology.orchestra.run.smithi045.stdout:1429 writesame 0xc967ec8 thru 0xc96c704 data_size 0x1(0x483d bytes) 2024-09-17T07:29:08.849 INFO:teuthology.orchestra.run.smithi045.stdout:1430 writesame 0xd6d2c38 thru 0xd6d5b10 data_size 0x1(0x2ed9 bytes) 2024-09-17T07:29:08.849 INFO:teuthology.orchestra.run.smithi045.stdout:1431 write 0x8250654 thru 0x825b051 (0xa9fe bytes) 2024-09-17T07:29:08.851 INFO:teuthology.orchestra.run.smithi045.stdout:1432 read 0xc4f4e92 thru 0xc503aab (0xec1a bytes) 2024-09-17T07:29:08.852 INFO:teuthology.orchestra.run.smithi045.stdout:1438 writesame 0x5795448 thru 0x5797e70 data_size 0x1(0x2a29 bytes) 2024-09-17T07:29:08.858 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:08.858 INFO:teuthology.orchestra.run.smithi045.stdout:1440 read 0x4147685 thru 0x4147ac1 (0x43d bytes) 2024-09-17T07:29:08.858 INFO:teuthology.orchestra.run.smithi045.stdout:1441 write 0x15595bf thru 0x1560488 (0x6eca bytes) 2024-09-17T07:29:08.862 INFO:teuthology.orchestra.run.smithi045.stdout:1444 writesame 0x645c03e thru 0x646bd68 data_size 0x1(0xfd2b bytes) 2024-09-17T07:29:08.886 INFO:teuthology.orchestra.run.smithi045.stdout:1446 writesame 0xc809440 thru 0xc817994 data_size 0x1(0xe555 bytes) 2024-09-17T07:29:08.893 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:08.893+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:08.895 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-17T07:29:08.895 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:29:08.895 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-17T07:29:08.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-17T07:29:08.906 INFO:teuthology.orchestra.run.smithi045.stdout:1449 writesame 0x2b607a7 thru 0x2b63770 data_size 0x3(0x2fca bytes) 2024-09-17T07:29:08.911 INFO:teuthology.orchestra.run.smithi045.stdout:1451 read 0x2a651e9 thru 0x2a71b98 (0xc9b0 bytes) 2024-09-17T07:29:08.911 INFO:teuthology.orchestra.run.smithi045.stdout:1452 write 0xb59ebd5 thru 0xb59f6d3 (0xaff bytes) 2024-09-17T07:29:08.913 INFO:teuthology.orchestra.run.smithi045.stdout:1455 writesame 0xcf89d82 thru 0xcf9733c data_size 0x1f(0xd5bb bytes) 2024-09-17T07:29:08.927 INFO:teuthology.orchestra.run.smithi045.stdout:1456 trunc from 0xd6d5b11 to 0x4a024c1 2024-09-17T07:29:08.941 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:08.941 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:08.942 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:08.947 INFO:teuthology.orchestra.run.smithi045.stdout:1457 write 0xa1d282f thru 0xa1d7721 (0x4ef3 bytes) 2024-09-17T07:29:08.950 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:08.954 INFO:teuthology.orchestra.run.smithi045.stdout:1458 trunc from 0xa1d7722 to 0xd8e60ac 2024-09-17T07:29:08.958 INFO:teuthology.orchestra.run.smithi045.stdout:1460 writesame 0xc176d54 thru 0xc17c43d data_size 0x19(0x56ea bytes) 2024-09-17T07:29:08.969 INFO:teuthology.orchestra.run.smithi045.stdout:1461 trunc from 0xd8e60ac to 0x70d86cc 2024-09-17T07:29:08.977 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:08.990 INFO:teuthology.orchestra.run.smithi045.stdout:1463 writesame 0xdf08442 thru 0xdf0fa81 data_size 0x58(0x7640 bytes) 2024-09-17T07:29:08.991 INFO:teuthology.orchestra.run.smithi045.stdout:1464 punch from 0xd2bdc78 to 0xd2c09cd, (0x2d55 bytes) 2024-09-17T07:29:08.991 INFO:teuthology.orchestra.run.smithi045.stdout:1466 writesame 0x43526b3 thru 0x435d7a1 data_size 0x1(0xb0ef bytes) 2024-09-17T07:29:08.994 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:08.993+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:09.013 INFO:teuthology.orchestra.run.smithi045.stdout:1467 write 0x7ba3b21 thru 0x7ba656b (0x2a4b bytes) 2024-09-17T07:29:09.017 INFO:teuthology.orchestra.run.smithi045.stdout:1468 read 0xa20b49b thru 0xa20f5c8 (0x412e bytes) 2024-09-17T07:29:09.017 INFO:teuthology.orchestra.run.smithi045.stdout:1469 write 0xc1bfb15 thru 0xc1cf764 (0xfc50 bytes) 2024-09-17T07:29:09.020 INFO:teuthology.orchestra.run.smithi045.stdout:1470 write 0xa5ebaa0 thru 0xa5f3f83 (0x84e4 bytes) 2024-09-17T07:29:09.023 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:09.023 INFO:teuthology.orchestra.run.smithi045.stdout:1474 read 0x1cd50c1 thru 0x1cddd86 (0x8cc6 bytes) 2024-09-17T07:29:09.030 INFO:teuthology.orchestra.run.smithi045.stdout:1475 trunc from 0xdf0fa82 to 0x7e9ea53 2024-09-17T07:29:09.042 INFO:teuthology.orchestra.run.smithi045.stdout:1476 write 0x163964d thru 0x1643a71 (0xa425 bytes) 2024-09-17T07:29:09.045 INFO:teuthology.orchestra.run.smithi045.stdout:1477 writesame 0x4c1e890 thru 0x4c2c0b1 data_size 0x37(0xd822 bytes) 2024-09-17T07:29:09.049 INFO:teuthology.orchestra.run.smithi045.stdout:1479 write 0x57f51bd thru 0x57fcdfd (0x7c41 bytes) 2024-09-17T07:29:09.051 INFO:teuthology.orchestra.run.smithi045.stdout:1480 read 0x162109f thru 0x1624de4 (0x3d46 bytes) 2024-09-17T07:29:09.052 INFO:teuthology.orchestra.run.smithi045.stdout:1481 punch from 0x341c5a8 to 0x3421875, (0x52cd bytes) 2024-09-17T07:29:09.052 INFO:teuthology.orchestra.run.smithi045.stdout:1483 write 0x2216e0d thru 0x2221613 (0xa807 bytes) 2024-09-17T07:29:09.056 INFO:teuthology.orchestra.run.smithi045.stdout:1485 punch from 0x2a6235f to 0x2a7111c, (0xedbd bytes) 2024-09-17T07:29:09.060 INFO:teuthology.orchestra.run.smithi045.stdout:1488 trunc from 0x7e9ea53 to 0xbdbe070 2024-09-17T07:29:09.063 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:09.064 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:09.070 INFO:teuthology.orchestra.run.smithi045.stdout:1490 trunc from 0xbdbe070 to 0x7f9d346 2024-09-17T07:29:09.072 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:09.075 INFO:teuthology.orchestra.run.smithi045.stdout:1491 writesame 0xa5b92c thru 0xa6af0a data_size 0x1(0xf5df bytes) 2024-09-17T07:29:09.092 INFO:teuthology.orchestra.run.smithi045.stdout:1492 read 0xbd5754 thru 0xbd8072 (0x291f bytes) 2024-09-17T07:29:09.092 INFO:teuthology.orchestra.run.smithi045.stdout:1495 read 0x2e4601b thru 0x2e46c68 (0xc4e bytes) 2024-09-17T07:29:09.102 INFO:teuthology.orchestra.run.smithi045.stdout:1496 write 0xcf131a1 thru 0xcf1ad67 (0x7bc7 bytes) 2024-09-17T07:29:09.105 INFO:teuthology.orchestra.run.smithi045.stdout:1497 write 0x68ad8a thru 0x696d6d (0xbfe4 bytes) 2024-09-17T07:29:09.106 INFO:teuthology.orchestra.run.smithi045.stdout:1498 writesame 0xb94ad6a thru 0xb94f828 data_size 0x2b(0x4abf bytes) 2024-09-17T07:29:09.109 INFO:teuthology.orchestra.run.smithi045.stdout:1499 punch from 0x56b74ed to 0x56c6cf7, (0xf80a bytes) 2024-09-17T07:29:09.109 INFO:teuthology.orchestra.run.smithi045.stdout:1500 writesame 0x96fa319 thru 0x970489c data_size 0x84(0xa584 bytes) 2024-09-17T07:29:09.112 INFO:teuthology.orchestra.run.smithi045.stdout:1501 write 0x9221656 thru 0x9231106 (0xfab1 bytes) 2024-09-17T07:29:09.115 INFO:teuthology.orchestra.run.smithi045.stdout:1502 write 0x1543709 thru 0x154ac2a (0x7522 bytes) 2024-09-17T07:29:09.116 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:09.116+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:09.117 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:09.117 INFO:teuthology.orchestra.run.smithi045.stdout:1505 read 0xbbaee2a thru 0xbbb2e78 (0x404f bytes) 2024-09-17T07:29:09.117 INFO:teuthology.orchestra.run.smithi045.stdout:1506 write 0x3d7e331 thru 0x3d87ab9 (0x9789 bytes) 2024-09-17T07:29:09.120 INFO:teuthology.orchestra.run.smithi045.stdout:1507 write 0xca59e3f thru 0xca63bf6 (0x9db8 bytes) 2024-09-17T07:29:09.124 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:09.124 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:09.124 INFO:teuthology.orchestra.run.smithi045.stdout:1510 write 0x10a09c5 thru 0x10a4b26 (0x4162 bytes) 2024-09-17T07:29:09.126 INFO:teuthology.orchestra.run.smithi045.stdout:1511 trunc from 0xcf1ad68 to 0xe862ce6 2024-09-17T07:29:09.148 INFO:teuthology.orchestra.run.smithi045.stdout:1513 trunc from 0xe862ce6 to 0x10cc3 2024-09-17T07:29:09.190 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T07:29:09.191 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19985 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5275648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9377931419999999, 2024-09-17T07:29:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071819998999999995, 2024-09-17T07:29:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19985 2024-09-17T07:29:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252493+0000", 2024-09-17T07:29:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6300000000000004e-07 2024-09-17T07:29:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252496+0000", 2024-09-17T07:29:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-17T07:29:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252499+0000", 2024-09-17T07:29:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:29:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324036+0000", 2024-09-17T07:29:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071536587999999998 2024-09-17T07:29:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324118+0000", 2024-09-17T07:29:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2336999999999995e-05 2024-09-17T07:29:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324312+0000", 2024-09-17T07:29:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000193709 2024-09-17T07:29:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19986 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5505024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.93778199, 2024-09-17T07:29:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072083761999999996, 2024-09-17T07:29:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19986 2024-09-17T07:29:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:29:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252508+0000", 2024-09-17T07:29:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:29:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252511+0000", 2024-09-17T07:29:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.145e-06 2024-09-17T07:29:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252514+0000", 2024-09-17T07:29:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5569999999999998e-06 2024-09-17T07:29:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324326+0000", 2024-09-17T07:29:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071812681000000003 2024-09-17T07:29:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324372+0000", 2024-09-17T07:29:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5893000000000003e-05 2024-09-17T07:29:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324587+0000", 2024-09-17T07:29:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214608 2024-09-17T07:29:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19987 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5734400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9377683719999999, 2024-09-17T07:29:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072276676999999998, 2024-09-17T07:29:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19987 2024-09-17T07:29:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252518+0000", 2024-09-17T07:29:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-17T07:29:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252522+0000", 2024-09-17T07:29:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2440000000000004e-06 2024-09-17T07:29:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252525+0000", 2024-09-17T07:29:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5919999999999999e-06 2024-09-17T07:29:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324621+0000", 2024-09-17T07:29:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072096442999999996 2024-09-17T07:29:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324699+0000", 2024-09-17T07:29:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8061999999999999e-05 2024-09-17T07:29:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324793+0000", 2024-09-17T07:29:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4207000000000001e-05 2024-09-17T07:29:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19988 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9377574009999998, 2024-09-17T07:29:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072585798000000007, 2024-09-17T07:29:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19988 2024-09-17T07:29:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252529+0000", 2024-09-17T07:29:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:29:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252532+0000", 2024-09-17T07:29:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9579999999999999e-06 2024-09-17T07:29:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252534+0000", 2024-09-17T07:29:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5730000000000002e-06 2024-09-17T07:29:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324825+0000", 2024-09-17T07:29:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072290757999999997 2024-09-17T07:29:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324874+0000", 2024-09-17T07:29:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9360000000000002e-05 2024-09-17T07:29:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325113+0000", 2024-09-17T07:29:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000239025 2024-09-17T07:29:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19989 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.93774778, 2024-09-17T07:29:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072963494000000004, 2024-09-17T07:29:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19989 2024-09-17T07:29:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:29:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:29:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:29:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2900000000000002e-07 2024-09-17T07:29:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252542+0000", 2024-09-17T07:29:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8600000000000003e-06 2024-09-17T07:29:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252545+0000", 2024-09-17T07:29:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.605e-06 2024-09-17T07:29:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325144+0000", 2024-09-17T07:29:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072599460000000005 2024-09-17T07:29:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325223+0000", 2024-09-17T07:29:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8473999999999994e-05 2024-09-17T07:29:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325501+0000", 2024-09-17T07:29:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000278063 2024-09-17T07:29:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19990 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6422528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9377373090000001, 2024-09-17T07:29:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073241127000000003, 2024-09-17T07:29:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19990 2024-09-17T07:29:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252552+0000", 2024-09-17T07:29:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4719999999999997e-06 2024-09-17T07:29:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252557+0000", 2024-09-17T07:29:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3039999999999998e-06 2024-09-17T07:29:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252560+0000", 2024-09-17T07:29:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8830000000000002e-06 2024-09-17T07:29:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325529+0000", 2024-09-17T07:29:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072968880999999999 2024-09-17T07:29:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325574+0000", 2024-09-17T07:29:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466000000000001e-05 2024-09-17T07:29:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325789+0000", 2024-09-17T07:29:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021490999999999999 2024-09-17T07:29:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19991 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6651904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.937718007, 2024-09-17T07:29:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073720552999999994, 2024-09-17T07:29:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19991 2024-09-17T07:29:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4699999999999998e-07 2024-09-17T07:29:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252568+0000", 2024-09-17T07:29:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5300000000000002e-07 2024-09-17T07:29:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252572+0000", 2024-09-17T07:29:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9160000000000003e-06 2024-09-17T07:29:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252575+0000", 2024-09-17T07:29:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:29:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325820+0000", 2024-09-17T07:29:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073245597999999995 2024-09-17T07:29:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325901+0000", 2024-09-17T07:29:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0913000000000003e-05 2024-09-17T07:29:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326288+0000", 2024-09-17T07:29:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038646900000000002 2024-09-17T07:29:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19992 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6881280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.937707654, 2024-09-17T07:29:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074054683999999996, 2024-09-17T07:29:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19992 2024-09-17T07:29:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252578+0000", 2024-09-17T07:29:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:29:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252579+0000", 2024-09-17T07:29:09.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2500000000000004e-07 2024-09-17T07:29:09.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252582+0000", 2024-09-17T07:29:09.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-17T07:29:09.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252585+0000", 2024-09-17T07:29:09.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.542e-06 2024-09-17T07:29:09.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326315+0000", 2024-09-17T07:29:09.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073729921000000004 2024-09-17T07:29:09.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326359+0000", 2024-09-17T07:29:09.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4095999999999999e-05 2024-09-17T07:29:09.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326632+0000", 2024-09-17T07:29:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027314499999999997 2024-09-17T07:29:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19993 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7110656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9376973500000001, 2024-09-17T07:29:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074425142999999999, 2024-09-17T07:29:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19993 2024-09-17T07:29:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:29:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252589+0000", 2024-09-17T07:29:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.33e-07 2024-09-17T07:29:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252593+0000", 2024-09-17T07:29:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:29:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252595+0000", 2024-09-17T07:29:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.621e-06 2024-09-17T07:29:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326666+0000", 2024-09-17T07:29:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074070787999999999 2024-09-17T07:29:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326752+0000", 2024-09-17T07:29:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5748000000000006e-05 2024-09-17T07:29:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327013+0000", 2024-09-17T07:29:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026098400000000002 2024-09-17T07:29:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19994 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7340032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9376868910000002, 2024-09-17T07:29:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074765920999999999, 2024-09-17T07:29:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19994 2024-09-17T07:29:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:29:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252599+0000", 2024-09-17T07:29:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2099999999999995e-07 2024-09-17T07:29:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252603+0000", 2024-09-17T07:29:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:29:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252606+0000", 2024-09-17T07:29:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-09-17T07:29:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327047+0000", 2024-09-17T07:29:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074441050999999994 2024-09-17T07:29:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327121+0000", 2024-09-17T07:29:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4283999999999996e-05 2024-09-17T07:29:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327364+0000", 2024-09-17T07:29:09.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000243155 2024-09-17T07:29:09.237 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.237 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.237 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.237 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19995 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7569408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.237 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:09.237 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9376761939999998, 2024-09-17T07:29:09.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075172694999999998, 2024-09-17T07:29:09.237 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.237 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.237 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.238 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.238 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.238 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.238 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19995 2024-09-17T07:29:09.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.238 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:09.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:09.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:09.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:09.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252610+0000", 2024-09-17T07:29:09.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6799999999999999e-07 2024-09-17T07:29:09.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252614+0000", 2024-09-17T07:29:09.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000002e-06 2024-09-17T07:29:09.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252616+0000", 2024-09-17T07:29:09.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.419e-06 2024-09-17T07:29:09.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327407+0000", 2024-09-17T07:29:09.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074790617000000004 2024-09-17T07:29:09.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327498+0000", 2024-09-17T07:29:09.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0758000000000001e-05 2024-09-17T07:29:09.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327782+0000", 2024-09-17T07:29:09.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028395599999999998 2024-09-17T07:29:09.241 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.241 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.241 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.241 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7798784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.242 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:09.242 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9376658779999998, 2024-09-17T07:29:09.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075605832999999997, 2024-09-17T07:29:09.242 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.242 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.242 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.242 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:29:09.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.242 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:09.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:09.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:09.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:09.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:29:09.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:29:09.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:29:09.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:29:09.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:29:09.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:29:09.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327844+0000", 2024-09-17T07:29:09.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075217709999999993 2024-09-17T07:29:09.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327927+0000", 2024-09-17T07:29:09.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2628999999999998e-05 2024-09-17T07:29:09.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328225+0000", 2024-09-17T07:29:09.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029800600000000002 2024-09-17T07:29:09.246 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.246 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.246 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.246 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8028160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.246 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:09.246 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9376555250000003, 2024-09-17T07:29:09.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076061237000000004, 2024-09-17T07:29:09.246 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.246 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.246 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.247 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.247 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.247 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.247 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:29:09.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.247 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:09.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:09.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:09.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:09.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:29:09.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:29:09.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:29:09.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:29:09.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:29:09.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:29:09.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328268+0000", 2024-09-17T07:29:09.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075626323999999995 2024-09-17T07:29:09.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328352+0000", 2024-09-17T07:29:09.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3578999999999994e-05 2024-09-17T07:29:09.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328691+0000", 2024-09-17T07:29:09.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033915099999999999 2024-09-17T07:29:09.250 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.250 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.250 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.251 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8257536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.251 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:09.251 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9376400739999999, 2024-09-17T07:29:09.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076441903000000005, 2024-09-17T07:29:09.251 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.251 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.251 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.251 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.251 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.251 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.251 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:29:09.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.251 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:09.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:09.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:09.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:29:09.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:29:09.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:29:09.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:29:09.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:29:09.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:29:09.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:29:09.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328747+0000", 2024-09-17T07:29:09.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076094192000000005 2024-09-17T07:29:09.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328824+0000", 2024-09-17T07:29:09.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7096999999999997e-05 2024-09-17T07:29:09.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.329087+0000", 2024-09-17T07:29:09.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026249999999999998 2024-09-17T07:29:09.255 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.255 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.255 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.255 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20029 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.255 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:09.255 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9275550859999999, 2024-09-17T07:29:09.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071783696999999994, 2024-09-17T07:29:09.255 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.255 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.255 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.256 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20029 2024-09-17T07:29:09.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.256 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:09.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:09.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262735+0000", 2024-09-17T07:29:09.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.955e-06 2024-09-17T07:29:09.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262737+0000", 2024-09-17T07:29:09.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0090000000000002e-06 2024-09-17T07:29:09.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262743+0000", 2024-09-17T07:29:09.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2890000000000003e-06 2024-09-17T07:29:09.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262748+0000", 2024-09-17T07:29:09.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:09.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334163+0000", 2024-09-17T07:29:09.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071415336999999995 2024-09-17T07:29:09.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334243+0000", 2024-09-17T07:29:09.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9548000000000005e-05 2024-09-17T07:29:09.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334514+0000", 2024-09-17T07:29:09.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027086800000000001 2024-09-17T07:29:09.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.259 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.259 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7356416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.260 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:09.260 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9275239150000001, 2024-09-17T07:29:09.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072172616999999994, 2024-09-17T07:29:09.260 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.260 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.260 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.260 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:29:09.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.260 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:09.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:09.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:29:09.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:29:09.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:29:09.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:29:09.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:29:09.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:29:09.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:29:09.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:29:09.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334558+0000", 2024-09-17T07:29:09.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071780790999999997 2024-09-17T07:29:09.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334647+0000", 2024-09-17T07:29:09.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9448000000000001e-05 2024-09-17T07:29:09.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334934+0000", 2024-09-17T07:29:09.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028654599999999999 2024-09-17T07:29:09.264 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.264 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.264 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.264 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7585792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.264 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:09.264 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.927502203, 2024-09-17T07:29:09.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072606135000000002, 2024-09-17T07:29:09.264 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.264 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.264 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:29:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:09.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:09.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:29:09.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:29:09.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:29:09.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:29:09.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:29:09.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:29:09.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:29:09.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335116+0000", 2024-09-17T07:29:09.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072323897999999998 2024-09-17T07:29:09.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335166+0000", 2024-09-17T07:29:09.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9882000000000001e-05 2024-09-17T07:29:09.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335389+0000", 2024-09-17T07:29:09.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022334300000000001 2024-09-17T07:29:09.268 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.268 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.268 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.269 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7815168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.269 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:09.269 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.927490004, 2024-09-17T07:29:09.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073051230999999994, 2024-09-17T07:29:09.269 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.269 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.269 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.269 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.269 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.269 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.269 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:29:09.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.270 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:09.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:09.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:09.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:09.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:29:09.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:29:09.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:29:09.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:29:09.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:29:09.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:29:09.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335431+0000", 2024-09-17T07:29:09.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072627852000000007 2024-09-17T07:29:09.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335525+0000", 2024-09-17T07:29:09.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3708000000000005e-05 2024-09-17T07:29:09.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335846+0000", 2024-09-17T07:29:09.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032182000000000001 2024-09-17T07:29:09.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.273 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.273 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.273 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:09.273 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.927479146, 2024-09-17T07:29:09.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073284199999999994, 2024-09-17T07:29:09.273 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.274 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.274 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.274 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:29:09.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.274 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:09.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:09.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:09.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:29:09.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:29:09.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:29:09.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:29:09.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:29:09.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:29:09.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:29:09.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335863+0000", 2024-09-17T07:29:09.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073048921000000003 2024-09-17T07:29:09.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335904+0000", 2024-09-17T07:29:09.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1545000000000003e-05 2024-09-17T07:29:09.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336090+0000", 2024-09-17T07:29:09.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185787 2024-09-17T07:29:09.277 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.277 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.277 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.278 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8273920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:09.278 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:09.278 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.9274648860000001, 2024-09-17T07:29:09.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073670478999999997, 2024-09-17T07:29:09.278 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.278 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.278 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.279 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.279 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:29:09.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.279 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:09.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:09.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:09.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:29:09.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:29:09.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:29:09.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:29:09.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:29:09.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:29:09.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:29:09.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336126+0000", 2024-09-17T07:29:09.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073297254000000006 2024-09-17T07:29:09.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336211+0000", 2024-09-17T07:29:09.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4728000000000006e-05 2024-09-17T07:29:09.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336491+0000", 2024-09-17T07:29:09.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027979399999999998 2024-09-17T07:29:09.282 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.282 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.282 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.282 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.282 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.282 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:09.285 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:09.285 INFO:teuthology.orchestra.run.smithi045.stdout:1514 write 0x9ff2b66 thru 0x9ff95fe (0x6a99 bytes) 2024-09-17T07:29:09.285 INFO:teuthology.orchestra.run.smithi045.stdout:1516 write 0x2c3376c thru 0x2c39384 (0x5c19 bytes) 2024-09-17T07:29:09.286 INFO:teuthology.orchestra.run.smithi045.stdout:1517 read 0x2b7bb17 thru 0x2b893b7 (0xd8a1 bytes) 2024-09-17T07:29:09.286 INFO:teuthology.orchestra.run.smithi045.stdout:1518 read 0x3dcd3ad thru 0x3dd7dba (0xaa0e bytes) 2024-09-17T07:29:09.286 INFO:teuthology.orchestra.run.smithi045.stdout:1519 writesame 0x7455a12 thru 0x745e63e data_size 0x1(0x8c2d bytes) 2024-09-17T07:29:09.286 INFO:teuthology.orchestra.run.smithi045.stdout:1520 writesame 0x98ddf28 thru 0x98e19df data_size 0x4(0x3ab8 bytes) 2024-09-17T07:29:09.286 INFO:teuthology.orchestra.run.smithi045.stdout:moved 0 / 82 (0%) 2024-09-17T07:29:09.286 INFO:teuthology.orchestra.run.smithi045.stdout:avg 11.7143 2024-09-17T07:29:09.286 INFO:teuthology.orchestra.run.smithi045.stdout:stddev 3.81993 -> 3.81993 (expected baseline 3.16872) 2024-09-17T07:29:09.286 INFO:teuthology.orchestra.run.smithi045.stdout:min osd.2 with 5 -> 5 pgs (0.426829 -> 0.426829 * mean) 2024-09-17T07:29:09.286 INFO:teuthology.orchestra.run.smithi045.stdout:max osd.7 with 14 -> 14 pgs (1.19512 -> 1.19512 * mean) 2024-09-17T07:29:09.286 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:29:09.287 INFO:teuthology.orchestra.run.smithi045.stdout:oload 1000 2024-09-17T07:29:09.287 INFO:teuthology.orchestra.run.smithi045.stdout:max_change 3 2024-09-17T07:29:09.287 INFO:teuthology.orchestra.run.smithi045.stdout:max_change_osds 4 2024-09-17T07:29:09.287 INFO:teuthology.orchestra.run.smithi045.stdout:average_utilization 0.0004 2024-09-17T07:29:09.287 INFO:teuthology.orchestra.run.smithi045.stdout:overload_utilization 0.0041 2024-09-17T07:29:09.287 INFO:teuthology.orchestra.run.smithi045.stderr:no change 2024-09-17T07:29:09.287 INFO:teuthology.orchestra.run.smithi045.stdout:1521 trunc from 0x9ff95ff to 0x2831fb 2024-09-17T07:29:09.288 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-17T07:29:09.288 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-17T07:29:09.293 INFO:teuthology.orchestra.run.smithi045.stdout:1522 read 0x102df4 thru 0x104291 (0x149e bytes) 2024-09-17T07:29:09.293 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:09.293 INFO:teuthology.orchestra.run.smithi045.stdout:1525 read 0xdbf9 thru 0x10e33 (0x323b bytes) 2024-09-17T07:29:09.293 INFO:teuthology.orchestra.run.smithi045.stdout:1526 read 0x640ce thru 0x70346 (0xc279 bytes) 2024-09-17T07:29:09.293 INFO:teuthology.orchestra.run.smithi045.stdout:1527 write 0x13cdd24 thru 0x13d48c2 (0x6b9f bytes) 2024-09-17T07:29:09.293 INFO:teuthology.orchestra.run.smithi045.stdout:1530 write 0x3963598 thru 0x396c1d5 (0x8c3e bytes) 2024-09-17T07:29:09.294 INFO:teuthology.orchestra.run.smithi045.stdout:1532 writesame 0x96fac35 thru 0x970082e data_size 0x3d(0x5bfa bytes) 2024-09-17T07:29:09.294 INFO:teuthology.orchestra.run.smithi045.stdout:1534 punch from 0x665ee86 to 0x666ed59, (0xfed3 bytes) 2024-09-17T07:29:09.294 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:09.294 INFO:teuthology.orchestra.run.smithi045.stdout:1537 write 0x913e79e thru 0x914046a (0x1ccd bytes) 2024-09-17T07:29:09.294 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:09.283+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:09.295 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:09.295 INFO:teuthology.orchestra.run.smithi045.stdout:1541 read 0x8128dc2 thru 0x8134284 (0xb4c3 bytes) 2024-09-17T07:29:09.295 INFO:teuthology.orchestra.run.smithi045.stdout:1542 read 0x482a75a thru 0x482b5a2 (0xe49 bytes) 2024-09-17T07:29:09.295 INFO:teuthology.orchestra.run.smithi045.stdout:1543 writesame 0x5962ea7 thru 0x5968bbc data_size 0x5(0x5d16 bytes) 2024-09-17T07:29:09.295 INFO:teuthology.orchestra.run.smithi045.stdout:1544 read 0x8586423 thru 0x859432e (0xdf0c bytes) 2024-09-17T07:29:09.295 INFO:teuthology.orchestra.run.smithi045.stdout:1545 read 0x3f9b58a thru 0x3fa2718 (0x718f bytes) 2024-09-17T07:29:09.295 INFO:teuthology.orchestra.run.smithi045.stdout:1546 write 0x865cd59 thru 0x866043b (0x36e3 bytes) 2024-09-17T07:29:09.296 INFO:teuthology.orchestra.run.smithi045.stdout:1548 punch from 0x891c117 to 0x892879c, (0xc685 bytes) 2024-09-17T07:29:09.296 INFO:teuthology.orchestra.run.smithi045.stdout:1550 writesame 0x5ae7df4 thru 0x5ae9daf data_size 0x6(0x1fbc bytes) 2024-09-17T07:29:09.299 INFO:teuthology.orchestra.run.smithi045.stdout:1551 punch from 0x2158bf3 to 0x216041e, (0x782b bytes) 2024-09-17T07:29:09.300 INFO:teuthology.orchestra.run.smithi045.stdout:1552 read 0x2f06166 thru 0x2f0d9f8 (0x7893 bytes) 2024-09-17T07:29:09.311 INFO:teuthology.orchestra.run.smithi045.stdout:1553 write 0xe39ff47 thru 0xe3a5273 (0x532d bytes) 2024-09-17T07:29:09.316 INFO:teuthology.orchestra.run.smithi045.stdout:1557 punch from 0x14c711a to 0x14d0bbb, (0x9aa1 bytes) 2024-09-17T07:29:09.316 INFO:teuthology.orchestra.run.smithi045.stdout:1558 write 0xdfc2568 thru 0xdfc4238 (0x1cd1 bytes) 2024-09-17T07:29:09.319 INFO:teuthology.orchestra.run.smithi045.stdout:1559 write 0x8b792b8 thru 0x8b7fffd (0x6d46 bytes) 2024-09-17T07:29:09.322 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:09.322 INFO:teuthology.orchestra.run.smithi045.stdout:1561 write 0xa2c2110 thru 0xa2c257a (0x46b bytes) 2024-09-17T07:29:09.324 INFO:teuthology.orchestra.run.smithi045.stdout:1562 write 0x304d506 thru 0x305af6d (0xda68 bytes) 2024-09-17T07:29:09.343 INFO:teuthology.orchestra.run.smithi045.stdout:1564 trunc from 0xe3a5274 to 0x32b2f78 2024-09-17T07:29:09.356 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:09.369 INFO:teuthology.orchestra.run.smithi045.stdout:1567 trunc from 0x32b2f78 to 0xe885047 2024-09-17T07:29:09.385 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:09.384+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:09.388 INFO:teuthology.orchestra.run.smithi045.stdout:1568 trunc from 0xe885047 to 0x5d72ed7 2024-09-17T07:29:09.394 INFO:teuthology.orchestra.run.smithi045.stdout:1569 read 0x19f296 thru 0x1a471c (0x5487 bytes) 2024-09-17T07:29:09.395 INFO:teuthology.orchestra.run.smithi045.stdout:1570 read 0x117bbc7 thru 0x11876b2 (0xbaec bytes) 2024-09-17T07:29:09.395 INFO:teuthology.orchestra.run.smithi045.stdout:1571 write 0x48544aa thru 0x485534d (0xea4 bytes) 2024-09-17T07:29:09.398 INFO:teuthology.orchestra.run.smithi045.stdout:1572 read 0x4aea675 thru 0x4af054d (0x5ed9 bytes) 2024-09-17T07:29:09.398 INFO:teuthology.orchestra.run.smithi045.stdout:1574 read 0x638558 thru 0x63eece (0x6977 bytes) 2024-09-17T07:29:09.399 INFO:teuthology.orchestra.run.smithi045.stdout:1575 trunc from 0x5d72ed7 to 0x6d3562c 2024-09-17T07:29:09.409 INFO:teuthology.orchestra.run.smithi045.stdout:1577 trunc from 0x6d3562c to 0x3b76644 2024-09-17T07:29:09.419 INFO:teuthology.orchestra.run.smithi045.stdout:1579 read 0x14634cb thru 0x1470ff0 (0xdb26 bytes) 2024-09-17T07:29:09.419 INFO:teuthology.orchestra.run.smithi045.stdout:1580 punch from 0x57eccc to 0x58151a, (0x284e bytes) 2024-09-17T07:29:09.419 INFO:teuthology.orchestra.run.smithi045.stdout:1581 read 0x27dd0c6 thru 0x27e8687 (0xb5c2 bytes) 2024-09-17T07:29:09.423 INFO:teuthology.orchestra.run.smithi045.stdout:1582 trunc from 0x3b76644 to 0xb3a511 2024-09-17T07:29:09.430 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:09.430 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:09.430 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:09.439 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:09.448 INFO:teuthology.orchestra.run.smithi045.stdout:1583 trunc from 0xb3a511 to 0xb9eec7d 2024-09-17T07:29:09.454 INFO:teuthology.orchestra.run.smithi045.stdout:1585 write 0x8067ff8 thru 0x8077866 (0xf86f bytes) 2024-09-17T07:29:09.458 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:09.458 INFO:teuthology.orchestra.run.smithi045.stdout:1588 writesame 0xaaba6c7 thru 0xaac7438 data_size 0x1(0xcd72 bytes) 2024-09-17T07:29:09.470 INFO:teuthology.orchestra.run.smithi045.stdout:1589 write 0x18ba02b thru 0x18c2977 (0x894d bytes) 2024-09-17T07:29:09.480 INFO:teuthology.orchestra.run.smithi045.stdout:1590 trunc from 0xb9eec7d to 0x678c269 2024-09-17T07:29:09.485 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:09.485+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:09.491 INFO:teuthology.orchestra.run.smithi045.stdout:1591 writesame 0x1b96335 thru 0x1ba1cda data_size 0x59(0xb9a6 bytes) 2024-09-17T07:29:09.497 INFO:teuthology.orchestra.run.smithi045.stdout:1592 write 0x24f127d thru 0x24f53ce (0x4152 bytes) 2024-09-17T07:29:09.510 INFO:teuthology.orchestra.run.smithi045.stdout:1594 writesame 0xc9e38a1 thru 0xc9f033b data_size 0x71(0xca9b bytes) 2024-09-17T07:29:09.511 INFO:teuthology.orchestra.run.smithi045.stdout:1596 write 0x7313363 thru 0x7314a32 (0x16d0 bytes) 2024-09-17T07:29:09.513 INFO:teuthology.orchestra.run.smithi045.stdout:1597 punch from 0xbbfb999 to 0xbc09171, (0xd7d8 bytes) 2024-09-17T07:29:09.513 INFO:teuthology.orchestra.run.smithi045.stdout:1598 read 0x9f6e2f7 thru 0x9f71580 (0x328a bytes) 2024-09-17T07:29:09.513 INFO:teuthology.orchestra.run.smithi045.stdout:1599 read 0x10723b8 thru 0x107601d (0x3c66 bytes) 2024-09-17T07:29:09.513 INFO:teuthology.orchestra.run.smithi045.stdout:1600 clone 15 order 20 su 262144 sc 15 2024-09-17T07:29:09.583 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:09.584 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:09.584 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:09.584 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:09.584 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:09.586 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:09.585+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:09.592 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:09.629 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-17T07:29:09.628+0000 7f9bb64ca640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-17T07:29:09.629 INFO:teuthology.orchestra.run.smithi045.stdout:moved 5 / 82 (6.09756%) 2024-09-17T07:29:09.629 INFO:teuthology.orchestra.run.smithi045.stdout:avg 11.7143 2024-09-17T07:29:09.630 INFO:teuthology.orchestra.run.smithi045.stdout:stddev 3.81993 -> 3.63655 (expected baseline 3.16872) 2024-09-17T07:29:09.630 INFO:teuthology.orchestra.run.smithi045.stdout:min osd.2 with 5 -> 6 pgs (0.426829 -> 0.512195 * mean) 2024-09-17T07:29:09.630 INFO:teuthology.orchestra.run.smithi045.stdout:max osd.7 with 14 -> 15 pgs (1.19512 -> 1.28049 * mean) 2024-09-17T07:29:09.630 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:29:09.630 INFO:teuthology.orchestra.run.smithi045.stdout:oload 110 2024-09-17T07:29:09.630 INFO:teuthology.orchestra.run.smithi045.stdout:max_change 3 2024-09-17T07:29:09.630 INFO:teuthology.orchestra.run.smithi045.stdout:max_change_osds 4 2024-09-17T07:29:09.630 INFO:teuthology.orchestra.run.smithi045.stdout:average_utilization 0.0004 2024-09-17T07:29:09.630 INFO:teuthology.orchestra.run.smithi045.stdout:overload_utilization 0.0004 2024-09-17T07:29:09.631 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 weight 1.0000 -> 0.8154 2024-09-17T07:29:09.631 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 weight 1.0000 -> 0.8517 2024-09-17T07:29:09.631 INFO:teuthology.orchestra.run.smithi045.stderr:no change 2024-09-17T07:29:09.642 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T07:29:09.642 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-17T07:29:09.726 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:09.726+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:09.737 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:09.737 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:09.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:09.737 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:09.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.738 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19118 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1966080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.738 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905451+0000", 2024-09-17T07:29:09.738 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8304531150000001, 2024-09-17T07:29:09.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046375417000000002, 2024-09-17T07:29:09.738 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.738 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.738 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.739 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19118 2024-09-17T07:29:09.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.739 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905451+0000", 2024-09-17T07:29:09.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905451+0000", 2024-09-17T07:29:09.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905452+0000", 2024-09-17T07:29:09.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:29:09.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905453+0000", 2024-09-17T07:29:09.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.719e-06 2024-09-17T07:29:09.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905459+0000", 2024-09-17T07:29:09.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6999999999999996e-06 2024-09-17T07:29:09.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905464+0000", 2024-09-17T07:29:09.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7310000000000001e-06 2024-09-17T07:29:09.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.951558+0000", 2024-09-17T07:29:09.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046094176000000001 2024-09-17T07:29:09.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.951608+0000", 2024-09-17T07:29:09.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9846999999999999e-05 2024-09-17T07:29:09.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.951827+0000", 2024-09-17T07:29:09.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021888499999999999 2024-09-17T07:29:09.743 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.743 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.743 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.743 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19119 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2195456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.743 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905469+0000", 2024-09-17T07:29:09.744 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8304350679999999, 2024-09-17T07:29:09.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046589024999999999, 2024-09-17T07:29:09.744 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.744 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.744 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.744 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.744 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.744 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.744 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19119 2024-09-17T07:29:09.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.744 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905469+0000", 2024-09-17T07:29:09.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905469+0000", 2024-09-17T07:29:09.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905470+0000", 2024-09-17T07:29:09.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:29:09.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905471+0000", 2024-09-17T07:29:09.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.761e-06 2024-09-17T07:29:09.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905477+0000", 2024-09-17T07:29:09.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7919999999999998e-06 2024-09-17T07:29:09.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905482+0000", 2024-09-17T07:29:09.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-09-17T07:29:09.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.951850+0000", 2024-09-17T07:29:09.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046367617 2024-09-17T07:29:09.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.951894+0000", 2024-09-17T07:29:09.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4637e-05 2024-09-17T07:29:09.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952058+0000", 2024-09-17T07:29:09.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016400900000000001 2024-09-17T07:29:09.748 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.748 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.749 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.749 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19120 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2424832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.749 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905488+0000", 2024-09-17T07:29:09.749 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8304166219999995, 2024-09-17T07:29:09.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046863363999999998, 2024-09-17T07:29:09.749 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.749 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.749 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.750 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19120 2024-09-17T07:29:09.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.750 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905488+0000", 2024-09-17T07:29:09.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905488+0000", 2024-09-17T07:29:09.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905488+0000", 2024-09-17T07:29:09.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-17T07:29:09.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905490+0000", 2024-09-17T07:29:09.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:29:09.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905497+0000", 2024-09-17T07:29:09.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0520000000000004e-06 2024-09-17T07:29:09.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905502+0000", 2024-09-17T07:29:09.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:29:09.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952082+0000", 2024-09-17T07:29:09.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046580557000000002 2024-09-17T07:29:09.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952127+0000", 2024-09-17T07:29:09.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5300999999999999e-05 2024-09-17T07:29:09.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952351+0000", 2024-09-17T07:29:09.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022359300000000001 2024-09-17T07:29:09.753 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.753 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.753 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.753 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19121 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2654208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.753 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905508+0000", 2024-09-17T07:29:09.753 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8303965470000003, 2024-09-17T07:29:09.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047141743999999999, 2024-09-17T07:29:09.754 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.754 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.754 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.754 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19121 2024-09-17T07:29:09.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.754 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905508+0000", 2024-09-17T07:29:09.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905508+0000", 2024-09-17T07:29:09.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905508+0000", 2024-09-17T07:29:09.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2599999999999998e-07 2024-09-17T07:29:09.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905514+0000", 2024-09-17T07:29:09.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:29:09.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905520+0000", 2024-09-17T07:29:09.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5130000000000004e-06 2024-09-17T07:29:09.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905524+0000", 2024-09-17T07:29:09.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6e-06 2024-09-17T07:29:09.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952375+0000", 2024-09-17T07:29:09.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850476000000002 2024-09-17T07:29:09.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952418+0000", 2024-09-17T07:29:09.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3205999999999999e-05 2024-09-17T07:29:09.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952650+0000", 2024-09-17T07:29:09.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023145500000000001 2024-09-17T07:29:09.759 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.759 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.759 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.759 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19122 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2883584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.759 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905531+0000", 2024-09-17T07:29:09.759 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8303734800000004, 2024-09-17T07:29:09.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047364581000000003, 2024-09-17T07:29:09.760 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.760 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.760 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.760 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19122 2024-09-17T07:29:09.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.760 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905531+0000", 2024-09-17T07:29:09.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905531+0000", 2024-09-17T07:29:09.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905531+0000", 2024-09-17T07:29:09.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:29:09.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905533+0000", 2024-09-17T07:29:09.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8640000000000001e-06 2024-09-17T07:29:09.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905539+0000", 2024-09-17T07:29:09.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.857e-06 2024-09-17T07:29:09.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905543+0000", 2024-09-17T07:29:09.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4320000000000001e-06 2024-09-17T07:29:09.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952676+0000", 2024-09-17T07:29:09.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047132276000000001 2024-09-17T07:29:09.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952723+0000", 2024-09-17T07:29:09.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7333000000000002e-05 2024-09-17T07:29:09.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952895+0000", 2024-09-17T07:29:09.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017240300000000001 2024-09-17T07:29:09.764 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.764 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.764 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.764 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19123 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.764 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905549+0000", 2024-09-17T07:29:09.764 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8303550089999998, 2024-09-17T07:29:09.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047648507999999999, 2024-09-17T07:29:09.764 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.764 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.764 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.765 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.765 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.765 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.765 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19123 2024-09-17T07:29:09.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.765 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905549+0000", 2024-09-17T07:29:09.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905549+0000", 2024-09-17T07:29:09.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905550+0000", 2024-09-17T07:29:09.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:09.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905552+0000", 2024-09-17T07:29:09.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8369999999999999e-06 2024-09-17T07:29:09.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905557+0000", 2024-09-17T07:29:09.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7200000000000003e-06 2024-09-17T07:29:09.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905562+0000", 2024-09-17T07:29:09.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3749999999999996e-06 2024-09-17T07:29:09.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952920+0000", 2024-09-17T07:29:09.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04735872 2024-09-17T07:29:09.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952965+0000", 2024-09-17T07:29:09.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4950000000000002e-05 2024-09-17T07:29:09.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953198+0000", 2024-09-17T07:29:09.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000232527 2024-09-17T07:29:09.769 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.769 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.769 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.769 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.769 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:09.769 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8303367980000003, 2024-09-17T07:29:09.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:29:09.770 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.770 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.770 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.770 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.770 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.770 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.770 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:29:09.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.770 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:09.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:09.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:09.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:29:09.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:29:09.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:29:09.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:29:09.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:29:09.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:29:09.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:29:09.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:29:09.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:29:09.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:29:09.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:29:09.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:29:09.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:29:09.773 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.774 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.774 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.774 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:09.774 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8303181210000004, 2024-09-17T07:29:09.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:29:09.775 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.775 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.775 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.775 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:29:09.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.775 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:09.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:09.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:29:09.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:09.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:29:09.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:29:09.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:29:09.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:29:09.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:29:09.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:29:09.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:29:09.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:29:09.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:29:09.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:29:09.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:29:09.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:29:09.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.779 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.779 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.779 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:09.779 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8302997679999997, 2024-09-17T07:29:09.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:29:09.780 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.780 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.780 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.780 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:29:09.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.780 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:09.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:09.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:09.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:29:09.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:29:09.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:29:09.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:29:09.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:29:09.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:29:09.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:29:09.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:29:09.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:29:09.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:29:09.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:29:09.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:29:09.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:29:09.784 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.784 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.784 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.784 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.784 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:09.784 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.83028017, 2024-09-17T07:29:09.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:29:09.784 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.784 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.784 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.785 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:29:09.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.785 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:09.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:09.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:29:09.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:29:09.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:29:09.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:29:09.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:29:09.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:29:09.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:29:09.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:29:09.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:29:09.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:29:09.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:29:09.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:29:09.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:29:09.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:29:09.788 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.788 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.788 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.789 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.789 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:09.789 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.830260215, 2024-09-17T07:29:09.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:29:09.789 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.789 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.789 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.790 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:29:09.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.790 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:09.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:09.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:09.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:29:09.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:29:09.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:29:09.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:29:09.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:29:09.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:29:09.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:29:09.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:29:09.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:29:09.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:29:09.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:29:09.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:29:09.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:29:09.794 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.794 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.795 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.795 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:09.795 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8302415600000002, 2024-09-17T07:29:09.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:29:09.795 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.795 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.795 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.795 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.795 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.795 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.795 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:29:09.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.796 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:09.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:09.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:09.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:29:09.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:29:09.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:29:09.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:29:09.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:29:09.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:29:09.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:29:09.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:29:09.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:29:09.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:29:09.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:29:09.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:29:09.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:29:09.799 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.799 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.800 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.800 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.800 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:09.800 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.830223084, 2024-09-17T07:29:09.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:29:09.800 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.800 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.800 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.801 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.801 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.801 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.801 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:29:09.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.801 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:09.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:09.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:29:09.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:29:09.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:29:09.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:29:09.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:29:09.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:29:09.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:29:09.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:29:09.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:29:09.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:29:09.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:29:09.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:29:09.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:29:09.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:29:09.804 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.805 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.805 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.805 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.805 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:09.805 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8302034850000002, 2024-09-17T07:29:09.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:29:09.805 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.805 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.805 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.806 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.806 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:29:09.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.806 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:09.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:09.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:09.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:29:09.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:29:09.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:29:09.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:29:09.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:29:09.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:29:09.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:29:09.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:29:09.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:29:09.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:29:09.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:29:09.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:29:09.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:29:09.809 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.809 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.810 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.810 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:09.810 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8301849099999998, 2024-09-17T07:29:09.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:29:09.810 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.810 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.810 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.810 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.811 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:29:09.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.811 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:09.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:09.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:29:09.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:09.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:29:09.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:29:09.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:29:09.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:29:09.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:29:09.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:29:09.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:29:09.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:29:09.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:29:09.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:29:09.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:29:09.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:29:09.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.815 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.815 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.815 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:09.815 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8301624829999996, 2024-09-17T07:29:09.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:29:09.815 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.815 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.815 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.815 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.816 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:29:09.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.816 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:09.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:09.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:09.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:09.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:29:09.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:29:09.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:29:09.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:29:09.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:29:09.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:29:09.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:29:09.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:29:09.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:29:09.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:29:09.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:29:09.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:29:09.819 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.820 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.820 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.820 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.820 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:09.820 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8301346660000002, 2024-09-17T07:29:09.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:29:09.820 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.820 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.820 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.820 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.821 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:29:09.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.821 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:09.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:09.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:09.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:29:09.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:29:09.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:29:09.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:29:09.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:29:09.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:29:09.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:29:09.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:29:09.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:29:09.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:29:09.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:29:09.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:29:09.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:29:09.824 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.825 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.825 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.825 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.825 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:09.825 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.830112722, 2024-09-17T07:29:09.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:29:09.825 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.825 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.825 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.826 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.826 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:29:09.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.826 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:09.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:09.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:09.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:29:09.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:29:09.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:29:09.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:29:09.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:29:09.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:29:09.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:29:09.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:29:09.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:29:09.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:29:09.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:29:09.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:29:09.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:29:09.829 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.830 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.830 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.830 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.830 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:09.830 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.830091618, 2024-09-17T07:29:09.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:29:09.830 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.830 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.830 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.831 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:29:09.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.831 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:09.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:09.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:09.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:29:09.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:29:09.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:29:09.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:29:09.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:09.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:29:09.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:29:09.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:29:09.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:29:09.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:29:09.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:29:09.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:29:09.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:29:09.834 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.834 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.834 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.835 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:09.835 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:09.835 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 5.8300720149999998, 2024-09-17T07:29:09.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:29:09.835 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:09.835 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:09.835 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:09.835 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:09.835 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:09.835 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:09.835 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:29:09.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.835 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:09.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:09.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:09.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:09.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:09.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:09.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:09.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:29:09.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:29:09.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:09.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:29:09.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:29:09.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:09.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:29:09.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:29:09.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:09.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:29:09.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:29:09.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:09.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:29:09.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:29:09.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:09.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:29:09.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:29:09.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:09.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:09.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:09.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:29:09.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:29:09.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.839 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:09.839 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:09.840 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:09.841 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:09.842 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:09.841+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:09.941 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:09.941+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:09.966 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:29:09.985 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:09.985 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:09.985 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:09.994 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:10.042 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:10.042+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:10.090 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:10.091 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:10.096 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:10.096 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:10.097 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:10.097 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:10.097 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:10.104 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:10.107 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:10.107 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:10.124 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:10.124 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:10.140 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:10.140 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:10.143 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:10.142+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:10.158 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:10.159 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:10.175 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:10.175 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:10.184 INFO:teuthology.orchestra.run.smithi045.stdout:moved 2 / 82 (2.43902%) 2024-09-17T07:29:10.184 INFO:teuthology.orchestra.run.smithi045.stdout:avg 11.7143 2024-09-17T07:29:10.184 INFO:teuthology.orchestra.run.smithi045.stdout:stddev 3.81993 -> 3.86507 (expected baseline 3.16872) 2024-09-17T07:29:10.184 INFO:teuthology.orchestra.run.smithi045.stdout:min osd.2 with 5 -> 5 pgs (0.426829 -> 0.426829 * mean) 2024-09-17T07:29:10.185 INFO:teuthology.orchestra.run.smithi045.stdout:max osd.7 with 14 -> 15 pgs (1.19512 -> 1.28049 * mean) 2024-09-17T07:29:10.185 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:29:10.185 INFO:teuthology.orchestra.run.smithi045.stdout:oload 110 2024-09-17T07:29:10.185 INFO:teuthology.orchestra.run.smithi045.stdout:max_change 0.05 2024-09-17T07:29:10.185 INFO:teuthology.orchestra.run.smithi045.stdout:max_change_osds 4 2024-09-17T07:29:10.185 INFO:teuthology.orchestra.run.smithi045.stdout:average_utilization 0.0004 2024-09-17T07:29:10.185 INFO:teuthology.orchestra.run.smithi045.stdout:overload_utilization 0.0004 2024-09-17T07:29:10.185 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 weight 1.0000 -> 0.9500 2024-09-17T07:29:10.185 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 weight 1.0000 -> 0.9500 2024-09-17T07:29:10.187 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-17T07:29:10.196 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:10.196 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:10.200 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:10.200 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:10.204 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-17T07:29:10.204 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-17T07:29:10.213 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:10.214 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:10.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:10.246 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:10.245+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:10.344 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:10.344+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:10.354 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:10.354 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:10.364 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:10.445 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:10.445+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:10.511 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:10.511 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:10.523 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:10.546 INFO:teuthology.orchestra.run.smithi045.stdout:moved 0 / 82 (0%) 2024-09-17T07:29:10.546 INFO:teuthology.orchestra.run.smithi045.stdout:avg 11.7143 2024-09-17T07:29:10.546 INFO:teuthology.orchestra.run.smithi045.stdout:stddev 3.86507 -> 3.86507 (expected baseline 3.16872) 2024-09-17T07:29:10.546 INFO:teuthology.orchestra.run.smithi045.stdout:min osd.2 with 5 -> 5 pgs (0.426829 -> 0.426829 * mean) 2024-09-17T07:29:10.546 INFO:teuthology.orchestra.run.smithi045.stdout:max osd.7 with 15 -> 15 pgs (1.28049 -> 1.28049 * mean) 2024-09-17T07:29:10.547 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:29:10.547 INFO:teuthology.orchestra.run.smithi045.stdout:oload 1000 2024-09-17T07:29:10.547 INFO:teuthology.orchestra.run.smithi045.stdout:max_change 0.05 2024-09-17T07:29:10.547 INFO:teuthology.orchestra.run.smithi045.stdout:max_change_osds 4 2024-09-17T07:29:10.547 INFO:teuthology.orchestra.run.smithi045.stdout:average_utilization 0.0004 2024-09-17T07:29:10.547 INFO:teuthology.orchestra.run.smithi045.stdout:overload_utilization 0.0042 2024-09-17T07:29:10.547 INFO:teuthology.orchestra.run.smithi045.stderr:no change 2024-09-17T07:29:10.547 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:10.546+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:10.559 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T07:29:10.559 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-17T07:29:10.630 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:10.630 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:10.639 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:10.681 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:10.680+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:10.748 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:10.748+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:10.790 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:10.790 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:10.799 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:10.849 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:10.849+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:10.947 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:10.947 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:10.950 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:10.950+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:10.955 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:11.051 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:11.051+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:11.103 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:11.103 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:11.111 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:11.153 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:11.152+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:11.185 INFO:teuthology.orchestra.run.smithi045.stdout:moved 4 / 82 (4.87805%) 2024-09-17T07:29:11.185 INFO:teuthology.orchestra.run.smithi045.stdout:avg 11.7143 2024-09-17T07:29:11.185 INFO:teuthology.orchestra.run.smithi045.stdout:stddev 3.86507 -> 3.30738 (expected baseline 3.16872) 2024-09-17T07:29:11.185 INFO:teuthology.orchestra.run.smithi045.stdout:min osd.2 with 5 -> 7 pgs (0.426829 -> 0.597561 * mean) 2024-09-17T07:29:11.186 INFO:teuthology.orchestra.run.smithi045.stdout:max osd.7 with 15 -> 14 pgs (1.28049 -> 1.19512 * mean) 2024-09-17T07:29:11.186 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:29:11.186 INFO:teuthology.orchestra.run.smithi045.stdout:oload 110 2024-09-17T07:29:11.186 INFO:teuthology.orchestra.run.smithi045.stdout:max_change 3 2024-09-17T07:29:11.186 INFO:teuthology.orchestra.run.smithi045.stdout:max_change_osds 4 2024-09-17T07:29:11.186 INFO:teuthology.orchestra.run.smithi045.stdout:average_utilization 0.0004 2024-09-17T07:29:11.186 INFO:teuthology.orchestra.run.smithi045.stdout:overload_utilization 0.0005 2024-09-17T07:29:11.186 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 weight 0.9500 -> 0.7833 2024-09-17T07:29:11.186 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 weight 1.0000 -> 0.8526 2024-09-17T07:29:11.186 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 weight 0.9500 -> 0.8455 2024-09-17T07:29:11.222 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:11.223 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:11.231 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:11.254 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:11.254+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:11.355 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:11.354+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:11.379 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:11.379 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:11.387 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:11.455 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:11.455+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:11.495 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:11.495 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:11.504 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:11.557 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:11.556+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:11.619 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:11.620 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:11.629 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:11.658 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:11.658+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:11.759 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:11.759+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:11.772 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:11.772 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:11.781 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:11.859 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:11.859+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:11.884 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:11.884 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:11.893 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:11.960 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:11.960+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:12.037 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:12.037 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:12.046 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:12.061 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:12.061+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:12.162 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:12.162+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:12.167 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:12.168 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:12.178 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:12.201 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:29:12.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T07:29:12.263 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:12.262+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:12.322 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:12.322 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:12.328 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:12.328 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:12.331 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:12.346 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:12.346 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:12.364 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:12.364+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:12.365 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:12.366 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:12.366 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:12.398 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:12.398 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:12.414 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:12.415 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:12.431 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:12.431 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:12.448 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:12.448 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:12.451 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:12.451 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:12.465 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:12.465+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:12.475 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:12.476 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:12.476 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:12.476 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:12.484 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:12.568 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:12.568+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:12.628 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:12.629 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:12.637 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:12.679 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:12.678+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:12.768 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:12.767+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:12.782 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:12.782 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:12.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:12.782 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:12.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.783 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19118 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1966080~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.783 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905451+0000", 2024-09-17T07:29:12.783 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8757419629999994, 2024-09-17T07:29:12.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046375417000000002, 2024-09-17T07:29:12.783 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.783 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.783 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.783 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.783 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.783 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.783 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19118 2024-09-17T07:29:12.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.784 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905451+0000", 2024-09-17T07:29:12.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905451+0000", 2024-09-17T07:29:12.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905452+0000", 2024-09-17T07:29:12.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:29:12.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905453+0000", 2024-09-17T07:29:12.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.719e-06 2024-09-17T07:29:12.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905459+0000", 2024-09-17T07:29:12.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6999999999999996e-06 2024-09-17T07:29:12.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905464+0000", 2024-09-17T07:29:12.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7310000000000001e-06 2024-09-17T07:29:12.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.951558+0000", 2024-09-17T07:29:12.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046094176000000001 2024-09-17T07:29:12.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.951608+0000", 2024-09-17T07:29:12.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9846999999999999e-05 2024-09-17T07:29:12.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.951827+0000", 2024-09-17T07:29:12.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021888499999999999 2024-09-17T07:29:12.787 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.787 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.787 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.787 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19119 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2195456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.787 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905469+0000", 2024-09-17T07:29:12.787 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8757239160000001, 2024-09-17T07:29:12.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046589024999999999, 2024-09-17T07:29:12.788 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.788 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.788 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.788 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.788 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.788 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.788 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19119 2024-09-17T07:29:12.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.788 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905469+0000", 2024-09-17T07:29:12.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905469+0000", 2024-09-17T07:29:12.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905470+0000", 2024-09-17T07:29:12.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:29:12.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905471+0000", 2024-09-17T07:29:12.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.761e-06 2024-09-17T07:29:12.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905477+0000", 2024-09-17T07:29:12.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7919999999999998e-06 2024-09-17T07:29:12.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905482+0000", 2024-09-17T07:29:12.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8740000000000003e-06 2024-09-17T07:29:12.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.951850+0000", 2024-09-17T07:29:12.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046367617 2024-09-17T07:29:12.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.951894+0000", 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4637e-05 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952058+0000", 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016400900000000001 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19120 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2424832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905488+0000", 2024-09-17T07:29:12.791 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8757054699999998, 2024-09-17T07:29:12.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046863363999999998, 2024-09-17T07:29:12.792 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.792 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.792 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.792 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.792 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.792 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.792 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19120 2024-09-17T07:29:12.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.792 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905488+0000", 2024-09-17T07:29:12.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905488+0000", 2024-09-17T07:29:12.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905488+0000", 2024-09-17T07:29:12.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-17T07:29:12.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905490+0000", 2024-09-17T07:29:12.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:29:12.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905497+0000", 2024-09-17T07:29:12.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0520000000000004e-06 2024-09-17T07:29:12.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905502+0000", 2024-09-17T07:29:12.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:29:12.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952082+0000", 2024-09-17T07:29:12.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046580557000000002 2024-09-17T07:29:12.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952127+0000", 2024-09-17T07:29:12.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5300999999999999e-05 2024-09-17T07:29:12.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952351+0000", 2024-09-17T07:29:12.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022359300000000001 2024-09-17T07:29:12.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.795 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.796 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.796 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19121 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2654208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.796 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905508+0000", 2024-09-17T07:29:12.796 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8756853949999996, 2024-09-17T07:29:12.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047141743999999999, 2024-09-17T07:29:12.796 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.796 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.796 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.796 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19121 2024-09-17T07:29:12.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.797 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905508+0000", 2024-09-17T07:29:12.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905508+0000", 2024-09-17T07:29:12.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905508+0000", 2024-09-17T07:29:12.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2599999999999998e-07 2024-09-17T07:29:12.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905514+0000", 2024-09-17T07:29:12.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:29:12.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905520+0000", 2024-09-17T07:29:12.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5130000000000004e-06 2024-09-17T07:29:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905524+0000", 2024-09-17T07:29:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6e-06 2024-09-17T07:29:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952375+0000", 2024-09-17T07:29:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850476000000002 2024-09-17T07:29:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952418+0000", 2024-09-17T07:29:12.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3205999999999999e-05 2024-09-17T07:29:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952650+0000", 2024-09-17T07:29:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023145500000000001 2024-09-17T07:29:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19122 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2883584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905531+0000", 2024-09-17T07:29:12.800 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8756623280000007, 2024-09-17T07:29:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047364581000000003, 2024-09-17T07:29:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19122 2024-09-17T07:29:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905531+0000", 2024-09-17T07:29:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905531+0000", 2024-09-17T07:29:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905531+0000", 2024-09-17T07:29:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:29:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905533+0000", 2024-09-17T07:29:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8640000000000001e-06 2024-09-17T07:29:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905539+0000", 2024-09-17T07:29:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.857e-06 2024-09-17T07:29:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905543+0000", 2024-09-17T07:29:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4320000000000001e-06 2024-09-17T07:29:12.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952676+0000", 2024-09-17T07:29:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047132276000000001 2024-09-17T07:29:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952723+0000", 2024-09-17T07:29:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7333000000000002e-05 2024-09-17T07:29:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952895+0000", 2024-09-17T07:29:12.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017240300000000001 2024-09-17T07:29:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19123 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905549+0000", 2024-09-17T07:29:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.875643857, 2024-09-17T07:29:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047648507999999999, 2024-09-17T07:29:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19123 2024-09-17T07:29:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905549+0000", 2024-09-17T07:29:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905549+0000", 2024-09-17T07:29:12.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905550+0000", 2024-09-17T07:29:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905552+0000", 2024-09-17T07:29:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8369999999999999e-06 2024-09-17T07:29:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905557+0000", 2024-09-17T07:29:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7200000000000003e-06 2024-09-17T07:29:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905562+0000", 2024-09-17T07:29:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3749999999999996e-06 2024-09-17T07:29:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952920+0000", 2024-09-17T07:29:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.04735872 2024-09-17T07:29:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.952965+0000", 2024-09-17T07:29:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4950000000000002e-05 2024-09-17T07:29:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953198+0000", 2024-09-17T07:29:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000232527 2024-09-17T07:29:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8756256459999996, 2024-09-17T07:29:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:29:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:29:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:29:12.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:29:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:29:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:29:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:29:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:29:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:29:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:29:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:29:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:29:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:29:12.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:29:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:29:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8756069689999997, 2024-09-17T07:29:12.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:29:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:29:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:29:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:29:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:29:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:29:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:29:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:29:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:29:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:29:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:29:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:29:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:29:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:29:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:29:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.818 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8755886159999999, 2024-09-17T07:29:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:29:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:29:12.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:12.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:29:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:29:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:29:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:29:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:29:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:29:12.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:29:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8755690180000002, 2024-09-17T07:29:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:29:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:29:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:29:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:29:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:29:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:29:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:29:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:29:12.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:29:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:29:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:29:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:29:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:29:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:29:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:29:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:29:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:12.827 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8755490629999994, 2024-09-17T07:29:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:29:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:29:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:29:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:29:12.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:29:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:29:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:29:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:29:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:29:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:29:12.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:29:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:29:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:29:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:29:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:29:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.831 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8755304079999995, 2024-09-17T07:29:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:29:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:29:12.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:29:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:29:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:29:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:29:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:29:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:29:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:29:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:29:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:29:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:29:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:29:12.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:29:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:29:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:12.836 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8755119320000002, 2024-09-17T07:29:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:29:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:29:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:29:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:29:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:29:12.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:29:12.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:29:12.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:29:12.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:29:12.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:29:12.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:29:12.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:29:12.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:29:12.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:29:12.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:29:12.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:29:12.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.841 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.841 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.841 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:12.841 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8754923330000004, 2024-09-17T07:29:12.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:29:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:29:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:12.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:29:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:29:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:29:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:29:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:29:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:29:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:29:12.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:29:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:29:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:29:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:29:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:29:12.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:29:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.875473758, 2024-09-17T07:29:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:29:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.846 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:29:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:29:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:29:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:29:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:29:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:29:12.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:29:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:29:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:29:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:29:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:29:12.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8754513310000007, 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.850 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:29:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:29:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:29:12.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:29:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:29:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:29:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:29:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:29:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:29:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:29:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:29:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:29:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:29:12.854 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8754235139999995, 2024-09-17T07:29:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:29:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.855 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:29:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:29:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:29:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:29:12.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:29:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:29:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:29:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:29:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:29:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:29:12.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:29:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:29:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:29:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:29:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8754015699999993, 2024-09-17T07:29:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:29:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:29:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:29:12.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:29:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:29:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:29:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:29:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:29:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:29:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:29:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:29:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:29:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:29:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:29:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:29:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8753804659999993, 2024-09-17T07:29:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:29:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:29:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:12.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:29:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:29:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:29:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:29:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:29:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:29:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:29:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:29:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:29:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:29:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:29:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:29:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 8.8753608630000009, 2024-09-17T07:29:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:29:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:29:12.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:29:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:29:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:29:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:29:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:29:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:29:12.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:29:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:29:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:29:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:29:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:29:12.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:29:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:29:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:29:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:12.873 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:12.875 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:12.875 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:12.874+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:12.975 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:12.975+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:13.018 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:13.018 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:13.018 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:13.018 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:13.027 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:13.069 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-17T07:29:13.111 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:13.111+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:13.170 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:13.170 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:13.171 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:13.171 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:13.171 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:13.177 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:13.176+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:13.179 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:13.278 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:13.277+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:13.331 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:13.332 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:13.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:13.332 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:13.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.332 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:29:13.332 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:13.332 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.995533330999997, 2024-09-17T07:29:13.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:29:13.332 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.332 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.332 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:13.332 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:29:13.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:29:13.333 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:29:13.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.333 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:13.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:13.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:29:13.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:29:13.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:29:13.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:29:13.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:29:13.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:29:13.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:29:13.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:29:13.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:29:13.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:29:13.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:29:13.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:29:13.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:29:13.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:29:13.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:29:13.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:29:13.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:29:13.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:29:13.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:29:13.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:29:13.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:13.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:29:13.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:29:13.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:13.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:29:13.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:29:13.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:13.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:29:13.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:29:13.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:29:13.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:29:13.339 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.339 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.339 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.339 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7378 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7176192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.339 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:13.339 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.406491296999999, 2024-09-17T07:29:13.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073003980999999996, 2024-09-17T07:29:13.339 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.339 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.339 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.340 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.340 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7378 2024-09-17T07:29:13.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.340 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:13.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:13.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:13.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1799999999999999e-07 2024-09-17T07:29:13.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923713+0000", 2024-09-17T07:29:13.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.057e-06 2024-09-17T07:29:13.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923716+0000", 2024-09-17T07:29:13.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0599999999999999e-06 2024-09-17T07:29:13.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923719+0000", 2024-09-17T07:29:13.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8219999999999998e-06 2024-09-17T07:29:13.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996401+0000", 2024-09-17T07:29:13.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072681467999999999 2024-09-17T07:29:13.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996452+0000", 2024-09-17T07:29:13.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.134e-05 2024-09-17T07:29:13.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996716+0000", 2024-09-17T07:29:13.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000264016 2024-09-17T07:29:13.343 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.343 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.343 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.343 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.343 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:13.343 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.406481181, 2024-09-17T07:29:13.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:29:13.344 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.344 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.344 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.344 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:29:13.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.344 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:13.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:13.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:29:13.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:29:13.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:29:13.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:29:13.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:29:13.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:29:13.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:29:13.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:29:13.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:29:13.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:29:13.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:29:13.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:29:13.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:29:13.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:29:13.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.348 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.348 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7398 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6389760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.348 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:13.348 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.401212647000001, 2024-09-17T07:29:13.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072891580999999997, 2024-09-17T07:29:13.348 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.348 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.348 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.349 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7398 2024-09-17T07:29:13.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.349 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:13.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:13.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:13.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:13.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928995+0000", 2024-09-17T07:29:13.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0759999999999996e-06 2024-09-17T07:29:13.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928999+0000", 2024-09-17T07:29:13.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1609999999999996e-06 2024-09-17T07:29:13.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929002+0000", 2024-09-17T07:29:13.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8449999999999999e-06 2024-09-17T07:29:13.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001598+0000", 2024-09-17T07:29:13.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072596318000000007 2024-09-17T07:29:13.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001663+0000", 2024-09-17T07:29:13.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4995999999999999e-05 2024-09-17T07:29:13.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001882+0000", 2024-09-17T07:29:13.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021897299999999999 2024-09-17T07:29:13.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.353 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.353 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.353 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.353 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:13.353 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.401198064999999, 2024-09-17T07:29:13.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:29:13.353 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.353 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.353 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.354 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.354 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:29:13.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.354 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:13.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:13.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:13.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:29:13.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:29:13.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:29:13.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:29:13.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:29:13.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:29:13.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:29:13.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:29:13.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:29:13.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:29:13.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:29:13.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:29:13.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:29:13.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.357 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.358 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.358 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:13.358 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.401167085000001, 2024-09-17T07:29:13.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:29:13.358 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.358 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.358 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.358 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:29:13.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.358 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:13.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:13.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:29:13.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:29:13.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:29:13.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:29:13.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:29:13.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:29:13.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:29:13.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:29:13.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:29:13.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:29:13.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:29:13.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:29:13.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:29:13.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:29:13.362 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.362 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.362 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.362 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.362 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:13.362 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.401137036000002, 2024-09-17T07:29:13.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:29:13.362 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.362 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.362 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.363 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.363 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.363 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.363 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:29:13.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.363 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:13.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:13.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:29:13.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:29:13.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:29:13.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:29:13.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:29:13.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:29:13.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:29:13.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:29:13.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:29:13.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:29:13.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:29:13.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:29:13.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:29:13.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:29:13.366 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.366 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.366 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.366 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.366 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:13.366 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.401108628999999, 2024-09-17T07:29:13.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:29:13.367 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.367 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.367 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.367 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:29:13.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.367 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:13.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:13.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:13.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:13.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:29:13.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:29:13.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:29:13.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:29:13.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:29:13.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:29:13.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:29:13.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:29:13.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:29:13.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:29:13.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:29:13.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:29:13.370 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.371 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.371 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.371 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.371 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:13.371 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.401088808000001, 2024-09-17T07:29:13.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:29:13.371 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.371 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.371 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.371 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:29:13.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.372 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:13.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:13.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:29:13.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:29:13.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:29:13.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:29:13.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:29:13.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:29:13.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:29:13.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:29:13.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:29:13.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:29:13.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:29:13.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:29:13.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:29:13.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:29:13.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.375 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.375 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.375 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:13.376 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.401064556000001, 2024-09-17T07:29:13.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:29:13.377 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.377 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.377 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.377 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:29:13.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.377 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:13.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:13.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:29:13.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:29:13.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:29:13.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:29:13.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:29:13.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:29:13.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:29:13.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:29:13.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:29:13.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:29:13.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:29:13.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:29:13.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:29:13.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:29:13.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.380 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.381 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7418 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7700480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.381 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:13.381 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.395658532999999, 2024-09-17T07:29:13.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072752965000000003, 2024-09-17T07:29:13.381 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.381 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.381 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.381 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7418 2024-09-17T07:29:13.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.381 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3599999999999999e-07 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934547+0000", 2024-09-17T07:29:13.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3350000000000001e-06 2024-09-17T07:29:13.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934554+0000", 2024-09-17T07:29:13.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9859999999999997e-06 2024-09-17T07:29:13.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934557+0000", 2024-09-17T07:29:13.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.818e-06 2024-09-17T07:29:13.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007104+0000", 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072546779000000006 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007147+0000", 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2904000000000001e-05 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007298+0000", 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015080700000000001 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.385 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.385 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:13.385 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.395641968, 2024-09-17T07:29:13.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:29:13.385 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.385 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.385 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.385 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:29:13.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.385 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:13.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:13.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:29:13.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:29:13.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:29:13.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:29:13.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:29:13.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:29:13.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:29:13.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:29:13.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:29:13.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:29:13.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:29:13.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:29:13.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:29:13.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:29:13.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.389 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.389 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.389 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:13.389 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.395595093000001, 2024-09-17T07:29:13.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:29:13.390 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.390 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.390 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.390 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:29:13.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.390 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:13.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:13.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:13.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:13.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:29:13.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:29:13.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:29:13.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:29:13.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:29:13.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:29:13.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:29:13.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:29:13.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:29:13.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:29:13.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:29:13.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:29:13.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.394 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.394 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.394 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.394 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:13.394 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.395580388999999, 2024-09-17T07:29:13.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:29:13.394 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.394 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.394 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.395 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.395 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:29:13.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.395 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:13.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:13.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:13.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:13.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:29:13.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:29:13.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:29:13.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:29:13.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:29:13.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:29:13.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:29:13.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:29:13.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.395557294, 2024-09-17T07:29:13.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:29:13.399 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.399 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.399 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.399 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.399 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.399 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.399 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:29:13.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.399 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:13.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:29:13.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:29:13.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:29:13.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:29:13.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:29:13.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:29:13.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:29:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:29:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:29:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:29:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.402 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.395525919000001, 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:13.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:13.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:29:13.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:29:13.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:29:13.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:29:13.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:29:13.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:29:13.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:29:13.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:29:13.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:29:13.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:29:13.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:29:13.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:29:13.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:29:13.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:29:13.407 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.407 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.407 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.407 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.407 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:13.407 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.395510658999999, 2024-09-17T07:29:13.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:29:13.407 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.407 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.407 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.408 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:29:13.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.408 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:13.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:13.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:13.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:29:13.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:29:13.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:29:13.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:29:13.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:29:13.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:29:13.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:29:13.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:29:13.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:29:13.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:29:13.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:29:13.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:29:13.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:29:13.411 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.411 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.411 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.412 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.412 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:13.412 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.395490589000001, 2024-09-17T07:29:13.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:29:13.412 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.412 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.412 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.412 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.412 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.412 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.412 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:29:13.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.412 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:13.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:13.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:13.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:29:13.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:29:13.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:29:13.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:29:13.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:29:13.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:29:13.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:29:13.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:29:13.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:29:13.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:29:13.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:29:13.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:29:13.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:29:13.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.416 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.416 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.416 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:13.416 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.395478649000001, 2024-09-17T07:29:13.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:29:13.417 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.417 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.417 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.417 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:29:13.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.417 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:13.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:13.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:13.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:13.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:29:13.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:29:13.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:29:13.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:29:13.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:29:13.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:29:13.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:29:13.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:29:13.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:29:13.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:29:13.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:29:13.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:29:13.420 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.420 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.420 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.421 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:13.421 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:13.421 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.395455491, 2024-09-17T07:29:13.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:29:13.421 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:13.421 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:13.421 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:13.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:13.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:13.421 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.421 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:29:13.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.421 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:29:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:29:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:29:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:29:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:29:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:29:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:29:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:29:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:29:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:29:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:29:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:29:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:29:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:29:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:13.425 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:13.426 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:13.426+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:13.427 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:13.492 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-17T07:29:13.496 INFO:teuthology.orchestra.run.smithi045.stdout:1601 punch from 0xa418d4a to 0xa41a30a, (0x15c0 bytes) 2024-09-17T07:29:13.499 INFO:teuthology.orchestra.run.smithi045.stdout:1602 write 0x48eb1e thru 0x494914 (0x5df7 bytes) 2024-09-17T07:29:13.503 INFO:teuthology.orchestra.run.smithi045.stdout:1605 writesame 0x3e055a9 thru 0x3e0674f data_size 0x1(0x11a7 bytes) 2024-09-17T07:29:13.507 INFO:teuthology.orchestra.run.smithi045.stdout:1606 read 0x6d80886 thru 0x6d825e5 (0x1d60 bytes) 2024-09-17T07:29:13.507 INFO:teuthology.orchestra.run.smithi045.stdout:1608 punch from 0x65c01da to 0x65c70ea, (0x6f10 bytes) 2024-09-17T07:29:13.509 INFO:teuthology.orchestra.run.smithi045.stdout:1609 trunc from 0xc9f033c to 0xaac19de 2024-09-17T07:29:13.527 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:13.527+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:13.542 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:13.542 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:13.551 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:13.628 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:13.628+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:13.658 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:13.658 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:13.659 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:13.659 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:13.659 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:13.667 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:13.729 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:13.729+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:13.776 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.642565072, 2024-09-17T07:29:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:29:13.777 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:29:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:29:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:29:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:29:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:29:13.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:29:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:29:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:29:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:29:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:29:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:29:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:29:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:29:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:29:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:29:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:29:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:29:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:29:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:29:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:29:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:29:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:29:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:29:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:29:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3365983679999998, 2024-09-17T07:29:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:29:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:29:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:29:13.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:29:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:29:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:29:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:29:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:29:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:29:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:29:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:29:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:29:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:29:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:29:13.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:29:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:29:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3365762529999996, 2024-09-17T07:29:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:29:13.791 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:29:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:29:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:29:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:29:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:29:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:29:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:29:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:29:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:29:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:29:13.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:29:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:29:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:29:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3365521490000001, 2024-09-17T07:29:13.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:29:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:29:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:29:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:29:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:29:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:29:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:29:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:29:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:29:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:29:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:29:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:29:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:29:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:29:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.799 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.33652806, 2024-09-17T07:29:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:29:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:29:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:29:13.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:29:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:29:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:29:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:29:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:29:13.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:29:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:29:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:29:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:29:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:29:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3365036410000002, 2024-09-17T07:29:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:29:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:29:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.804 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:29:13.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:29:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:29:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:29:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:29:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:29:13.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:29:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:29:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:29:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:29:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:29:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3364819409999997, 2024-09-17T07:29:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:29:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:29:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.808 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:13.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:29:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:29:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:29:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:29:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:29:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:29:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:29:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:29:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:29:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:29:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:29:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3364372089999996, 2024-09-17T07:29:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:29:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:29:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:29:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:29:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:29:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:29:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:29:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:29:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:29:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:29:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:29:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:29:13.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:29:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:29:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3364087429999998, 2024-09-17T07:29:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:29:13.816 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:29:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:29:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:29:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:29:13.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:29:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:29:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:29:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:29:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:29:13.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:29:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:29:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:29:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3362914909999999, 2024-09-17T07:29:13.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:29:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:29:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:29:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:29:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:29:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:29:13.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:29:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:29:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:29:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:29:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:29:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:29:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:29:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:29:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:13.824 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3358653890000003, 2024-09-17T07:29:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:29:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:29:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:29:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:29:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:29:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:29:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:29:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:29:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:29:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:29:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:29:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:29:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:29:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3358457179999998, 2024-09-17T07:29:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:29:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:29:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:29:13.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:29:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:29:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:29:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:29:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:29:13.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:29:13.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:29:13.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:29:13.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:29:13.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:29:13.832 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.832 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.832 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3358228539999999, 2024-09-17T07:29:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:29:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:29:13.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:29:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:29:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:29:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:29:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:29:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:29:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:29:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:29:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:29:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:29:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:29:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.836 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3358015529999996, 2024-09-17T07:29:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:29:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.837 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:29:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:29:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:29:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:29:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:29:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:29:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:29:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:29:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:29:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:29:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:29:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:29:13.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:29:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3357537649999998, 2024-09-17T07:29:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:29:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.841 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:29:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:29:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:29:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:29:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:29:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:29:13.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:29:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:29:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:29:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:29:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:29:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:29:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:29:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3357265429999998, 2024-09-17T07:29:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:29:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.845 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:29:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:29:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:29:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:29:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:29:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:29:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:29:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:29:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:29:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:29:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:29:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:29:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3357061559999996, 2024-09-17T07:29:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:29:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.849 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:29:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:29:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:29:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:29:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:29:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:29:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:29:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:29:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:29:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:29:13.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:29:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:29:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:29:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3356864509999999, 2024-09-17T07:29:13.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:29:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:29:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:29:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:29:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:29:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:29:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:29:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:29:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:29:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:29:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:29:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:29:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:29:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:29:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.857 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.335663373, 2024-09-17T07:29:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:29:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:29:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:29:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:29:13.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:29:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:29:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:29:13.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:29:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:29:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:29:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:29:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:29:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:29:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:29:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3356428109999996, 2024-09-17T07:29:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:29:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:13.862 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:29:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:13.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:29:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:29:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:29:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:13.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:29:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:29:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:29:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:29:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:29:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:29:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:13.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:29:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:29:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:13.866 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:13.866 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:13.868 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:13.868 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:13.867+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:13.969 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:13.968+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:14.016 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:14.016 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:14.017 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:14.017 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:14.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:14.069 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:14.069+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:14.134 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:14.134 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:14.134 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:14.145 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:14.189 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:14.188+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:14.256 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120984511, 2024-09-17T07:29:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:29:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.257 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:29:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:29:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:29:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:29:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:29:14.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.261 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120947613999999, 2024-09-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:29:14.263 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:29:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:14.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:29:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:29:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:29:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:29:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:29:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:29:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:29:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:29:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:29:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:29:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:29:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:29:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:29:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:29:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:29:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:29:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:29:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:29:14.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:29:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:29:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:29:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120921646999999, 2024-09-17T07:29:14.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:29:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:29:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:29:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:29:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:29:14.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:29:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:29:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:29:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:29:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:29:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:29:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:29:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:29:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:29:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:29:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:29:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:29:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:29:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:29:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:29:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:29:14.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:29:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:29:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120897052, 2024-09-17T07:29:14.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:29:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:29:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:14.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:29:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:29:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:29:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:29:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:29:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:29:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:29:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:29:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:29:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:29:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:29:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:29:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:29:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:29:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:29:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:29:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:29:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:29:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:29:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:29:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:29:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120873565, 2024-09-17T07:29:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:29:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:29:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:29:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:29:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:29:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:29:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:29:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:29:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:29:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:29:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:29:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:29:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:29:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:29:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:29:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:29:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:29:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:29:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:29:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:29:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:29:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:29:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120859827, 2024-09-17T07:29:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:29:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:29:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:29:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:29:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:29:14.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:29:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:29:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:29:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:29:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:29:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:29:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:29:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:29:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:29:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:29:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:29:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:29:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:29:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:29:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:29:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:29:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.294 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120829638, 2024-09-17T07:29:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:29:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:29:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:29:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:29:14.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:29:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:29:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:29:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:29:14.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:29:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:29:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:29:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:29:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:29:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:29:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:29:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:29:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:29:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:29:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:29:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:29:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:29:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:29:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.300 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120793132999999, 2024-09-17T07:29:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:29:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:29:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:29:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:29:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:29:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:29:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:29:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:29:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:29:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:29:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:29:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:29:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:29:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:29:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:29:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:29:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:29:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:29:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:29:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:29:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:29:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:29:14.306 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120765374000001, 2024-09-17T07:29:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:29:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.307 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:29:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:29:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:29:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:29:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:29:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:29:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:29:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:29:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:29:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:29:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:29:14.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:29:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:29:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:29:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:29:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:29:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:29:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:29:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:29:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:29:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:29:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.313 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120743214000001, 2024-09-17T07:29:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:29:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.314 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:29:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:29:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:29:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:29:14.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:29:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:29:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:29:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:29:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:29:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:29:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:29:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:29:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:29:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:29:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:29:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:29:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:29:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:29:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:29:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:29:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:29:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120730052999999, 2024-09-17T07:29:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:29:14.320 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:29:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:29:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:29:14.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:29:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:29:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:29:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:29:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:29:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:29:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:29:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:29:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:29:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:29:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:29:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:29:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:29:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:29:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:29:14.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:29:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:29:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:29:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120711464999999, 2024-09-17T07:29:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:29:14.326 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:29:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:29:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:29:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:29:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:29:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:29:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:29:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:29:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:29:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:29:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:29:14.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:29:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:29:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:29:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:29:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:29:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:29:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:29:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:29:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120678653999999, 2024-09-17T07:29:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:29:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:29:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.333 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:29:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:29:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:29:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:29:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:29:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:29:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:29:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:29:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:29:14.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:29:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:29:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:29:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:29:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:29:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:29:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:29:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:29:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:29:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.338 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120659597, 2024-09-17T07:29:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:29:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:29:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:29:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:29:14.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:29:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:29:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:29:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:29:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:29:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:29:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:29:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:29:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:29:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:29:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:29:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:29:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:29:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:29:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:29:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:29:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:29:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:29:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:29:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:29:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120637488, 2024-09-17T07:29:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:29:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:29:14.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:29:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:29:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:29:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:29:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:29:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:29:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:29:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:29:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:29:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:29:14.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:29:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:29:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:29:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:29:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:29:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:29:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:29:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:29:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:29:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:29:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:29:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120611381, 2024-09-17T07:29:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:29:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:29:14.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:29:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:29:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:29:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:29:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:29:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:29:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:29:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:29:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:29:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:29:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:29:14.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:29:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:29:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:29:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:29:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:29:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:29:14.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:29:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:29:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:29:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:29:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120582236000001, 2024-09-17T07:29:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:29:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:29:14.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.358 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:29:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:29:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:29:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:29:14.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:29:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:29:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:29:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:29:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.360 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:29:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:29:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:29:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:29:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:29:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:29:14.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:29:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:29:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:29:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:29:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:29:14.362 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:29:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120550432999998, 2024-09-17T07:29:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:29:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.363 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:29:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.364 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:29:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:29:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:29:14.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:29:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:29:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:29:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:29:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:29:14.366 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:29:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:29:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:29:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:29:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:29:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.367 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:29:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:29:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:29:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:29:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:29:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:29:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:29:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:29:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120523486, 2024-09-17T07:29:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:29:14.369 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:29:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:29:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:29:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:29:14.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:29:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:29:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:29:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.372 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:29:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:29:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:29:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:29:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:29:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:29:14.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:29:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:29:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:29:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:29:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:29:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:29:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:29:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:29:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:14.375 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 16.120482537000001, 2024-09-17T07:29:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:29:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:29:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.376 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:29:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:29:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:29:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:29:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:29:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:29:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:29:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:29:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:29:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:29:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:29:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:29:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:29:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:29:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:29:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:29:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:29:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:29:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:29:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:29:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:29:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:29:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:14.381 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:14.381 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:14.383 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:14.384 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:14.383+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:14.452 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:29:14.484 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:14.484+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:14.532 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:14.533 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:14.533 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:14.542 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:14.585 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:14.584+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:14.612 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:14.612 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:14.645 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:14.645 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:14.661 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:14.661 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:14.662 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:14.675 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:14.675 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:14.679 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:14.679 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:14.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:14.685 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:14.685+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:14.696 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:14.697 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:14.714 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:14.715 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:14.731 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:14.732 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:14.735 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:14.735 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:14.787 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:14.786+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:14.792 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:14.792 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:14.801 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:14.888 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:14.888+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:14.915 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:14.915 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:14.924 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:14.989 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:14.988+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:15.028 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:15.028 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:15.037 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:1610 read 0x1feddd3 thru 0x1ff66b7 (0x88e5 bytes) 2024-09-17T07:29:15.080 INFO:teuthology.orchestra.run.smithi045.stdout:1611 write 0xa9bfbc1 thru 0xa9ccb37 (0xcf77 bytes) 2024-09-17T07:29:15.083 INFO:teuthology.orchestra.run.smithi045.stdout:1612 write 0x44ad8ef thru 0x44b3ff3 (0x6705 bytes) 2024-09-17T07:29:15.086 INFO:teuthology.orchestra.run.smithi045.stdout:1613 writesame 0x7bedf3c thru 0x7bf47cf data_size 0x45(0x6894 bytes) 2024-09-17T07:29:15.088 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T07:29:15.089 INFO:teuthology.orchestra.run.smithi045.stdout:1614 read 0x445d14c thru 0x4461401 (0x42b6 bytes) 2024-09-17T07:29:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:1615 trunc from 0xaac19de to 0x4ccd9ba 2024-09-17T07:29:15.121 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:15.121+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:1617 read 0x1121635 thru 0x112be9d (0xa869 bytes) 2024-09-17T07:29:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:1618 punch from 0x15792d4 to 0x1584512, (0xb23e bytes) 2024-09-17T07:29:15.124 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:15.139 INFO:teuthology.orchestra.run.smithi045.stdout:1621 write 0xbf962ec thru 0xbf9f55b (0x9270 bytes) 2024-09-17T07:29:15.143 INFO:teuthology.orchestra.run.smithi045.stdout:1622 read 0x3aaf1bc thru 0x3abbef6 (0xcd3b bytes) 2024-09-17T07:29:15.143 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:15.143 INFO:teuthology.orchestra.run.smithi045.stdout:1625 writesame 0xf43ee8 thru 0xf50aef data_size 0x4(0xcc08 bytes) 2024-09-17T07:29:15.151 INFO:teuthology.orchestra.run.smithi045.stdout:1626 read 0x305f38f thru 0x3062788 (0x33fa bytes) 2024-09-17T07:29:15.151 INFO:teuthology.orchestra.run.smithi045.stdout:1627 read 0x4841566 thru 0x48467ef (0x528a bytes) 2024-09-17T07:29:15.151 INFO:teuthology.orchestra.run.smithi045.stdout:1628 write 0x6d26f5e thru 0x6d364a2 (0xf545 bytes) 2024-09-17T07:29:15.162 INFO:teuthology.orchestra.run.smithi045.stdout:1629 write 0xe29bfd6 thru 0xe29cd27 (0xd52 bytes) 2024-09-17T07:29:15.166 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:15.166 INFO:teuthology.orchestra.run.smithi045.stdout:1631 write 0xa8fa685 thru 0xa9014c7 (0x6e43 bytes) 2024-09-17T07:29:15.169 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:15.169 INFO:teuthology.orchestra.run.smithi045.stdout:1636 writesame 0x2d7c064 thru 0x2d82b56 data_size 0x13(0x6af3 bytes) 2024-09-17T07:29:15.174 INFO:teuthology.orchestra.run.smithi045.stdout:1638 read 0xb99d5c thru 0xb9b99b (0x1c40 bytes) 2024-09-17T07:29:15.174 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:15.174 INFO:teuthology.orchestra.run.smithi045.stdout:1641 write 0x2f38f87 thru 0x2f3b23e (0x22b8 bytes) 2024-09-17T07:29:15.177 INFO:teuthology.orchestra.run.smithi045.stdout:1642 punch from 0xe0e52a to 0xe1d643, (0xf119 bytes) 2024-09-17T07:29:15.177 INFO:teuthology.orchestra.run.smithi045.stdout:1643 read 0xbff8329 thru 0xbffdd91 (0x5a69 bytes) 2024-09-17T07:29:15.177 INFO:teuthology.orchestra.run.smithi045.stdout:1644 writesame 0x9033dc7 thru 0x9039208 data_size 0xf(0x5442 bytes) 2024-09-17T07:29:15.180 INFO:teuthology.orchestra.run.smithi045.stdout:1645 write 0x6be3712 thru 0x6be8f3d (0x582c bytes) 2024-09-17T07:29:15.183 INFO:teuthology.orchestra.run.smithi045.stdout:1646 read 0x489797f thru 0x48981bc (0x83e bytes) 2024-09-17T07:29:15.184 INFO:teuthology.orchestra.run.smithi045.stdout:1647 write 0x1d6af8f thru 0x1d755d6 (0xa648 bytes) 2024-09-17T07:29:15.187 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:15.187 INFO:teuthology.orchestra.run.smithi045.stdout:1651 read 0x2e47cef thru 0x2e521b6 (0xa4c8 bytes) 2024-09-17T07:29:15.187 INFO:teuthology.orchestra.run.smithi045.stdout:1652 write 0x978d5b5 thru 0x979a753 (0xd19f bytes) 2024-09-17T07:29:15.189 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:15.189 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:15.189 INFO:teuthology.orchestra.run.smithi045.stdout:1653 read 0x836ece9 thru 0x8376628 (0x7940 bytes) 2024-09-17T07:29:15.189 INFO:teuthology.orchestra.run.smithi045.stdout:1654 write 0x127bfdd thru 0x127dc88 (0x1cac bytes) 2024-09-17T07:29:15.190 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:15.190+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:15.193 INFO:teuthology.orchestra.run.smithi045.stdout:1655 writesame 0x5748823 thru 0x5751ec4 data_size 0x3(0x96a2 bytes) 2024-09-17T07:29:15.198 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:15.199 INFO:teuthology.orchestra.run.smithi045.stdout:1656 write 0xc2e2be8 thru 0xc2e6e33 (0x424c bytes) 2024-09-17T07:29:15.203 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:15.203 INFO:teuthology.orchestra.run.smithi045.stdout:1658 write 0x7ffb970 thru 0x80037de (0x7e6f bytes) 2024-09-17T07:29:15.206 INFO:teuthology.orchestra.run.smithi045.stdout:1659 writesame 0x926371b thru 0x9265902 data_size 0x23(0x21e8 bytes) 2024-09-17T07:29:15.209 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:15.209 INFO:teuthology.orchestra.run.smithi045.stdout:1662 write 0x1239701 thru 0x123e2ee (0x4bee bytes) 2024-09-17T07:29:15.210 INFO:teuthology.orchestra.run.smithi045.stdout:1663 read 0x1317ac9 thru 0x131d3ec (0x5924 bytes) 2024-09-17T07:29:15.211 INFO:teuthology.orchestra.run.smithi045.stdout:1664 read 0x14d837c thru 0x14e6a0b (0xe690 bytes) 2024-09-17T07:29:15.211 INFO:teuthology.orchestra.run.smithi045.stdout:1665 punch from 0xbd7fe11 to 0xbd82146, (0x2335 bytes) 2024-09-17T07:29:15.211 INFO:teuthology.orchestra.run.smithi045.stdout:1666 write 0x1a521f1 thru 0x1a5ad44 (0x8b54 bytes) 2024-09-17T07:29:15.214 INFO:teuthology.orchestra.run.smithi045.stdout:1671 read 0x8dde94e thru 0x8de5915 (0x6fc8 bytes) 2024-09-17T07:29:15.214 INFO:teuthology.orchestra.run.smithi045.stdout:1673 write 0x23dfc48 thru 0x23ea659 (0xaa12 bytes) 2024-09-17T07:29:15.217 INFO:teuthology.orchestra.run.smithi045.stdout:1674 read 0x6cebe9a thru 0x6cf23ec (0x6553 bytes) 2024-09-17T07:29:15.217 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:15.217 INFO:teuthology.orchestra.run.smithi045.stdout:1677 write 0x1b60321 thru 0x1b6ebe6 (0xe8c6 bytes) 2024-09-17T07:29:15.220 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:15.220 INFO:teuthology.orchestra.run.smithi045.stdout:1679 read 0xaf93f3e thru 0xaf9ec44 (0xad07 bytes) 2024-09-17T07:29:15.220 INFO:teuthology.orchestra.run.smithi045.stdout:1681 write 0xb7a39f7 thru 0xb7a540b (0x1a15 bytes) 2024-09-17T07:29:15.221 INFO:teuthology.orchestra.run.smithi045.stdout:1684 write 0x2407838 thru 0x240bb56 (0x431f bytes) 2024-09-17T07:29:15.225 INFO:teuthology.orchestra.run.smithi045.stdout:1685 punch from 0xda22332 to 0xda245cc, (0x229a bytes) 2024-09-17T07:29:15.225 INFO:teuthology.orchestra.run.smithi045.stdout:1686 write 0xc072a40 thru 0xc07ad75 (0x8336 bytes) 2024-09-17T07:29:15.228 INFO:teuthology.orchestra.run.smithi045.stdout:1687 punch from 0x4fc8cc5 to 0x4fd3388, (0xa6c3 bytes) 2024-09-17T07:29:15.228 INFO:teuthology.orchestra.run.smithi045.stdout:1689 punch from 0x47e10a1 to 0x47ea8b3, (0x9812 bytes) 2024-09-17T07:29:15.228 INFO:teuthology.orchestra.run.smithi045.stdout:1690 trunc from 0xe29cd28 to 0xe0afaba 2024-09-17T07:29:15.238 INFO:teuthology.orchestra.run.smithi045.stdout:1691 write 0x7c4ff85 thru 0x7c5d9f8 (0xda74 bytes) 2024-09-17T07:29:15.239 INFO:teuthology.orchestra.run.smithi045.stdout:1692 write 0x7e53ea7 thru 0x7e63e93 (0xffed bytes) 2024-09-17T07:29:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:1693 read 0x1bae349 thru 0x1bb0979 (0x2631 bytes) 2024-09-17T07:29:15.242 INFO:teuthology.orchestra.run.smithi045.stdout:1694 writesame 0x8e8cf70 thru 0x8e8ee56 data_size 0x9(0x1ee7 bytes) 2024-09-17T07:29:15.243 INFO:teuthology.orchestra.run.smithi045.stdout:1697 read 0x9ae7b67 thru 0x9ae94be (0x1958 bytes) 2024-09-17T07:29:15.256 INFO:teuthology.orchestra.run.smithi045.stdout:1698 trunc from 0xe0afaba to 0x2ae3410 2024-09-17T07:29:15.289 INFO:teuthology.orchestra.run.smithi045.stdout:1699 write 0x7d87fa8 thru 0x7d95324 (0xd37d bytes) 2024-09-17T07:29:15.291 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:15.291+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:15.304 INFO:teuthology.orchestra.run.smithi045.stdout:1700 write 0xd969012 thru 0xd969df4 (0xde3 bytes) 2024-09-17T07:29:15.318 INFO:teuthology.orchestra.run.smithi045.stdout:1701 trunc from 0xd969df5 to 0x3b431ee 2024-09-17T07:29:15.329 INFO:teuthology.orchestra.run.smithi045.stdout:1702 read 0xe4cdc7 thru 0xe5cb29 (0xfd63 bytes) 2024-09-17T07:29:15.329 INFO:teuthology.orchestra.run.smithi045.stdout:1703 punch from 0x58cd10 to 0x5967ce, (0x9abe bytes) 2024-09-17T07:29:15.329 INFO:teuthology.orchestra.run.smithi045.stdout:1704 clone 16 order 20 su 65536 sc 8 2024-09-17T07:29:15.349 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:15.349 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:15.358 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:15.392 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:15.392+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:15.461 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:15.461 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:15.470 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:15.514 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:15.514+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:15.574 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:15.574 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:15.582 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:15.624 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:15.623+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:15.696 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:15.695+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:15.727 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:15.727 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:15.736 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:15.798 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:15.797+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:15.879 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:15.879 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:15.888 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:15.931 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:15.930+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:15.990 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:15.990 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:15.999 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:16.000 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:15.998+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:16.100 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:16.099+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:16.143 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:16.143 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:16.152 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:16.201 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:16.201+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:16.204 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-17T07:29:16.204 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:29:16.205 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-17T07:29:16.205 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T07:29:16.205 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T07:29:16.205 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-17T07:29:16.206 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-17T07:29:16.276 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone15 from 0x3b431ee (overlap 0x2ae3410) to 0x893d9c 2024-09-17T07:29:16.302 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:16.302+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:16.309 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:16.310 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:16.321 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:16.355 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:16.355 INFO:teuthology.orchestra.run.smithi192.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T07:29:16.355 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:16.367 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-17T07:29:16.403 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:16.402+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:16.431 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:16.431 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:16.440 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:16.504 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:16.503+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:16.550 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:16.551 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:16.560 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:16.605 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:16.604+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:16.613 INFO:teuthology.orchestra.run.smithi045.stdout:1705 writesame 0xda1da09 thru 0xda1f544 data_size 0x2a(0x1b3c bytes) 2024-09-17T07:29:16.620 INFO:teuthology.orchestra.run.smithi045.stdout:1706 trunc from 0xda1f545 to 0x78a5f4c 2024-09-17T07:29:16.625 INFO:teuthology.orchestra.run.smithi045.stdout:1708 punch from 0x18fc58d to 0x190c0c0, (0xfb33 bytes) 2024-09-17T07:29:16.631 INFO:teuthology.orchestra.run.smithi045.stdout:1709 trunc from 0x78a5f4c to 0xc776840 2024-09-17T07:29:16.639 INFO:teuthology.orchestra.run.smithi045.stdout:1710 trunc from 0xc776840 to 0x990946b 2024-09-17T07:29:16.655 INFO:teuthology.orchestra.run.smithi045.stdout:1711 write 0xe65c7ed thru 0xe6666e0 (0x9ef4 bytes) 2024-09-17T07:29:16.662 INFO:teuthology.orchestra.run.smithi045.stdout:1712 read 0x51dae8c thru 0x51e2918 (0x7a8d bytes) 2024-09-17T07:29:16.662 INFO:teuthology.orchestra.run.smithi045.stdout:1713 writesame 0xb25fccc thru 0xb269259 data_size 0x1b(0x958e bytes) 2024-09-17T07:29:16.669 INFO:teuthology.orchestra.run.smithi045.stdout:1716 read 0xcf705ba thru 0xcf752be (0x4d05 bytes) 2024-09-17T07:29:16.669 INFO:teuthology.orchestra.run.smithi045.stdout:1717 punch from 0x771974b to 0x77263a6, (0xcc5b bytes) 2024-09-17T07:29:16.670 INFO:teuthology.orchestra.run.smithi045.stdout:1718 punch from 0x8d943e0 to 0x8d96d62, (0x2982 bytes) 2024-09-17T07:29:16.670 INFO:teuthology.orchestra.run.smithi045.stdout:1719 writesame 0xac4763e thru 0xac5755b data_size 0x46(0xff1e bytes) 2024-09-17T07:29:16.676 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:16.676 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:16.676 INFO:teuthology.orchestra.run.smithi045.stdout:1720 punch from 0x9f60036 to 0x9f633eb, (0x33b5 bytes) 2024-09-17T07:29:16.677 INFO:teuthology.orchestra.run.smithi045.stdout:1721 write 0x7a73e02 thru 0x7a81f37 (0xe136 bytes) 2024-09-17T07:29:16.681 INFO:teuthology.orchestra.run.smithi045.stdout:1722 write 0x6c24337 thru 0x6c2ac7b (0x6945 bytes) 2024-09-17T07:29:16.684 INFO:teuthology.orchestra.run.smithi045.stdout:1724 read 0xa87071 thru 0xa9287a (0xb80a bytes) 2024-09-17T07:29:16.684 INFO:teuthology.orchestra.run.smithi045.stdout:1726 read 0x64eb3a thru 0x65952d (0xa9f4 bytes) 2024-09-17T07:29:16.684 INFO:teuthology.orchestra.run.smithi045.stdout:1729 write 0x1ee5ddb thru 0x1ee87c9 (0x29ef bytes) 2024-09-17T07:29:16.686 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:16.702 INFO:teuthology.orchestra.run.smithi045.stdout:1732 trunc from 0xe6666e1 to 0x3e86643 2024-09-17T07:29:16.706 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:16.704+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:16.720 INFO:teuthology.orchestra.run.smithi045.stdout:1733 flatten 2024-09-17T07:29:16.736 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:29:16.762 INFO:teuthology.orchestra.run.smithi045.stdout:1735 trunc from 0x3e86643 to 0x185a138 2024-09-17T07:29:16.793 INFO:teuthology.orchestra.run.smithi045.stdout:1737 trunc from 0x185a138 to 0xaafaba8 2024-09-17T07:29:16.798 INFO:teuthology.orchestra.run.smithi045.stdout:1739 write 0x71ed6f3 thru 0x71ed897 (0x1a5 bytes) 2024-09-17T07:29:16.801 INFO:teuthology.orchestra.run.smithi045.stdout:1740 write 0x8c81325 thru 0x8c8535d (0x4039 bytes) 2024-09-17T07:29:16.807 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:16.805+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:16.812 INFO:teuthology.orchestra.run.smithi045.stdout:1741 write 0xd01cc15 thru 0xd02958c (0xc978 bytes) 2024-09-17T07:29:16.817 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:16.817 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:16.818 INFO:teuthology.orchestra.run.smithi045.stdout:1742 write 0x3013fe0 thru 0x301f2af (0xb2d0 bytes) 2024-09-17T07:29:16.821 INFO:teuthology.orchestra.run.smithi045.stdout:1743 writesame 0xa94d451 thru 0xa94ec4b data_size 0x1(0x17fb bytes) 2024-09-17T07:29:16.826 INFO:teuthology.orchestra.run.smithi045.stdout:1745 write 0x53164dd thru 0x5321462 (0xaf86 bytes) 2024-09-17T07:29:16.827 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:16.831 INFO:teuthology.orchestra.run.smithi045.stdout:1746 punch from 0x962fbac to 0x963aa7d, (0xaed1 bytes) 2024-09-17T07:29:16.831 INFO:teuthology.orchestra.run.smithi045.stdout:1747 read 0xdd5471 thru 0xde04ef (0xb07f bytes) 2024-09-17T07:29:16.832 INFO:teuthology.orchestra.run.smithi045.stdout:1748 write 0x92fa618 thru 0x9306f5d (0xc946 bytes) 2024-09-17T07:29:16.835 INFO:teuthology.orchestra.run.smithi045.stdout:1749 writesame 0x3483de1 thru 0x3493682 data_size 0x86(0xf8a2 bytes) 2024-09-17T07:29:16.838 INFO:teuthology.orchestra.run.smithi045.stdout:1750 writesame 0xc6904fe thru 0xc6938ba data_size 0x5(0x33bd bytes) 2024-09-17T07:29:16.841 INFO:teuthology.orchestra.run.smithi045.stdout:1751 write 0x19f576a thru 0x1a00b29 (0xb3c0 bytes) 2024-09-17T07:29:16.852 INFO:teuthology.orchestra.run.smithi045.stdout:1752 writesame 0xea0d71c thru 0xea111ff data_size 0x2(0x3ae4 bytes) 2024-09-17T07:29:16.858 INFO:teuthology.orchestra.run.smithi045.stdout:1755 write 0xed678ab thru 0xed6d650 (0x5da6 bytes) 2024-09-17T07:29:16.861 INFO:teuthology.orchestra.run.smithi045.stdout:1756 punch from 0x6d1b5b0 to 0x6d280db, (0xcb2b bytes) 2024-09-17T07:29:16.861 INFO:teuthology.orchestra.run.smithi045.stdout:1757 write 0xa2d0fdd thru 0xa2d5a4a (0x4a6e bytes) 2024-09-17T07:29:16.863 INFO:teuthology.orchestra.run.smithi045.stdout:1759 read 0xdb39e44 thru 0xdb3f761 (0x591e bytes) 2024-09-17T07:29:16.863 INFO:teuthology.orchestra.run.smithi045.stdout:1760 writesame 0x32073a6 thru 0x320a9be data_size 0x1(0x3619 bytes) 2024-09-17T07:29:16.869 INFO:teuthology.orchestra.run.smithi045.stdout:1761 punch from 0xb54e710 to 0xb55711f, (0x8a0f bytes) 2024-09-17T07:29:16.883 INFO:teuthology.orchestra.run.smithi045.stdout:1762 trunc from 0xed6d651 to 0x324fecb 2024-09-17T07:29:16.886 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:16.886 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:16.897 INFO:teuthology.orchestra.run.smithi045.stdout:1763 punch from 0x2c908ba to 0x2c90e8d, (0x5d3 bytes) 2024-09-17T07:29:16.901 INFO:teuthology.orchestra.run.smithi045.stdout:1764 write 0x3332e8d thru 0x333fb3d (0xccb1 bytes) 2024-09-17T07:29:16.905 INFO:teuthology.orchestra.run.smithi045.stdout:1765 writesame 0x12536a7 thru 0x12571c0 data_size 0xa(0x3b1a bytes) 2024-09-17T07:29:16.905 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:16.906 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:16.907 INFO:teuthology.orchestra.run.smithi045.stdout:1766 read 0x1162576 thru 0x1170dbb (0xe846 bytes) 2024-09-17T07:29:16.908 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:16.906+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:16.908 INFO:teuthology.orchestra.run.smithi045.stdout:1767 punch from 0x30f844f to 0x30fb40d, (0x2fbe bytes) 2024-09-17T07:29:16.918 INFO:teuthology.orchestra.run.smithi045.stdout:1768 trunc from 0x333fb3e to 0xa6679ba 2024-09-17T07:29:16.925 INFO:teuthology.orchestra.run.smithi045.stdout:1769 write 0x68ddad6 thru 0x68e39e1 (0x5f0c bytes) 2024-09-17T07:29:16.930 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:16.930 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:16.930 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:16.934 INFO:teuthology.orchestra.run.smithi045.stdout:1770 write 0xacf0149 thru 0xacf3c39 (0x3af1 bytes) 2024-09-17T07:29:16.937 INFO:teuthology.orchestra.run.smithi045.stdout:1771 write 0x77efd94 thru 0x77fd890 (0xdafd bytes) 2024-09-17T07:29:16.941 INFO:teuthology.orchestra.run.smithi045.stdout:1772 punch from 0x4a78230 to 0x4a83abd, (0xb88d bytes) 2024-09-17T07:29:16.941 INFO:teuthology.orchestra.run.smithi045.stdout:1774 read 0x4aa640f thru 0x4aa975c (0x334e bytes) 2024-09-17T07:29:16.941 INFO:teuthology.orchestra.run.smithi045.stdout:1776 clone 17 order 21 su 16384 sc 13 2024-09-17T07:29:16.948 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:16.948 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:16.951 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:16.951 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:16.951 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:16.951 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:16.961 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:16.965 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:16.966 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:16.982 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:16.982 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:17.000 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:17.000 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:17.003 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:17.003 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:17.045 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:17.044+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:17.072 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:17.072 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:17.073 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:17.073 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:17.073 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:17.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:17.123 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:17.122+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:17.210 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:17.209+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:17.233 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098803041, 2024-09-17T07:29:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:29:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.234 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:29:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:29:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:29:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:29:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:29:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:29:17.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:29:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:29:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:29:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:29:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:29:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:29:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:29:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:29:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:29:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:29:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:29:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:29:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:29:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:29:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:29:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:29:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:29:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.239 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098766143999999, 2024-09-17T07:29:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:29:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.240 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:29:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:29:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:29:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:29:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:29:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:29:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:29:17.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:29:17.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.244 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:29:17.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:29:17.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.244 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:29:17.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:29:17.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.245 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.245 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:29:17.245 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:29:17.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.245 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.245 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:29:17.245 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:29:17.245 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.245 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.245 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.246 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.246 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:17.246 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098740177, 2024-09-17T07:29:17.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:29:17.246 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.246 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.246 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.246 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.246 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.246 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.246 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:29:17.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.246 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:17.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:17.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:29:17.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:17.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.248 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.248 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:29:17.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:29:17.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.248 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.248 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:29:17.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:29:17.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.248 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.248 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:29:17.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:29:17.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.249 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.249 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:29:17.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:29:17.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.249 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.249 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:29:17.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:29:17.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:29:17.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:29:17.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:29:17.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:29:17.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:29:17.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:29:17.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:29:17.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:29:17.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:29:17.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:29:17.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.251 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.252 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.252 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:17.252 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098715582000001, 2024-09-17T07:29:17.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:29:17.252 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.252 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.252 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.252 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.252 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.252 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:29:17.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.252 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:17.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:17.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:17.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:29:17.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:29:17.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:29:17.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:29:17.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:29:17.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:29:17.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:29:17.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:29:17.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:29:17.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:29:17.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:29:17.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:29:17.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:29:17.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.257 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:29:17.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:29:17.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.261 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.261 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.261 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.261 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:17.261 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098692095000001, 2024-09-17T07:29:17.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:29:17.261 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.261 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.262 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.262 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:29:17.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.262 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:17.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:17.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:29:17.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:17.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:29:17.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:29:17.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:29:17.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:29:17.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:29:17.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:29:17.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:29:17.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:29:17.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:17.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:29:17.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:29:17.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:29:17.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:29:17.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:29:17.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:29:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:29:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:29:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:29:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:29:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:29:17.266 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.267 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.267 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.267 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.267 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:17.267 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098678357000001, 2024-09-17T07:29:17.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:29:17.267 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.267 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.267 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.267 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.267 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.267 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:29:17.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.268 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:17.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:17.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:17.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:29:17.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:29:17.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:29:17.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:29:17.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:29:17.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:29:17.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:29:17.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:17.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:29:17.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:17.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:29:17.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:29:17.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:29:17.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:29:17.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:29:17.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:29:17.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:29:17.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:29:17.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:29:17.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:29:17.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:29:17.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.273 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.274 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.274 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:17.274 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098648168, 2024-09-17T07:29:17.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:29:17.274 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.274 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.274 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.274 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.274 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:29:17.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.274 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:29:17.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:29:17.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:17.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:29:17.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:29:17.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:29:17.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:29:17.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:29:17.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:29:17.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:29:17.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:29:17.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:29:17.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:29:17.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:17.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:29:17.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:29:17.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:29:17.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:29:17.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:29:17.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:29:17.279 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.279 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.280 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.280 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.280 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:17.280 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098611663, 2024-09-17T07:29:17.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:29:17.280 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.280 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.280 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.280 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.280 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.281 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:29:17.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.281 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:17.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:17.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:17.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:29:17.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:29:17.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:29:17.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:29:17.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:29:17.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:29:17.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:29:17.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:29:17.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:29:17.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:17.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:29:17.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:29:17.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:29:17.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:29:17.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:29:17.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:29:17.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:29:17.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:29:17.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:29:17.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:29:17.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:29:17.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.285 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.286 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.286 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:17.286 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098583904000002, 2024-09-17T07:29:17.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:29:17.286 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.286 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.286 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.286 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.286 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.286 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:29:17.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.287 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:17.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:17.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:29:17.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:29:17.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:29:17.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:29:17.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:29:17.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:29:17.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:29:17.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:29:17.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:29:17.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:17.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:29:17.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:29:17.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:29:17.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:29:17.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:29:17.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:29:17.291 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.291 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.291 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.291 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.291 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:17.292 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098561744000001, 2024-09-17T07:29:17.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:29:17.292 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.292 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.292 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.292 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.292 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.292 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:29:17.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.292 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:17.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:17.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:17.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:29:17.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:29:17.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:29:17.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:29:17.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:17.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:29:17.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:29:17.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:29:17.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:29:17.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:29:17.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:29:17.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:29:17.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:29:17.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:29:17.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:29:17.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:29:17.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:29:17.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:29:17.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:29:17.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:29:17.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:29:17.297 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.297 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.297 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.297 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.297 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:17.298 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098548582999999, 2024-09-17T07:29:17.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:29:17.298 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.298 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.298 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.298 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.298 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.298 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.298 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:29:17.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.298 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:29:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:29:17.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:29:17.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:29:17.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:29:17.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:29:17.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:29:17.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:29:17.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:29:17.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:17.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:29:17.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:29:17.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:29:17.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:29:17.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:29:17.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:29:17.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:29:17.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:29:17.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:29:17.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:29:17.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:29:17.303 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.303 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.303 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.303 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.303 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:17.303 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098529995, 2024-09-17T07:29:17.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:29:17.303 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.304 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.304 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.304 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.304 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:29:17.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.304 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:17.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:17.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:29:17.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:29:17.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:29:17.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:29:17.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:17.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:29:17.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:17.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:29:17.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:17.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:29:17.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:17.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:29:17.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:29:17.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:29:17.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:29:17.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:29:17.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:29:17.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:29:17.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:29:17.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:29:17.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:29:17.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:29:17.309 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.309 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.309 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.309 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.309 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:17.309 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098497183999999, 2024-09-17T07:29:17.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:29:17.309 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.310 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.310 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.310 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.310 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:29:17.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.310 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:17.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:17.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:17.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:29:17.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:29:17.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:29:17.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:29:17.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:17.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:29:17.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:29:17.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:29:17.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:29:17.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:17.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:17.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:29:17.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:29:17.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:29:17.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:29:17.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:29:17.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:29:17.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:29:17.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:29:17.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:29:17.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:29:17.315 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.315 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.315 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.315 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.315 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:17.315 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098478127, 2024-09-17T07:29:17.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:29:17.315 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.315 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.315 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.316 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.316 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.316 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:29:17.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.316 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:17.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:17.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:29:17.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:29:17.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:29:17.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:29:17.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:29:17.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:29:17.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:29:17.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:29:17.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:29:17.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:29:17.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:29:17.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:29:17.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:29:17.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:29:17.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:29:17.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:29:17.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:29:17.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:29:17.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:29:17.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:29:17.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:29:17.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:29:17.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.320 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.321 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.321 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:17.321 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098456018, 2024-09-17T07:29:17.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:29:17.321 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.321 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.321 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.321 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.321 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.322 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:29:17.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.322 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:17.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:17.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:29:17.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:29:17.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:17.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:29:17.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:29:17.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:29:17.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:29:17.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:29:17.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:29:17.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:29:17.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:29:17.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:29:17.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:29:17.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:29:17.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:29:17.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:29:17.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:29:17.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:29:17.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:29:17.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:29:17.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:29:17.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:29:17.326 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.326 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.326 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.327 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.327 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:17.327 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098429911, 2024-09-17T07:29:17.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:29:17.327 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.327 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.327 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.327 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.327 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:29:17.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.327 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:17.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:17.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:29:17.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:29:17.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:17.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:29:17.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:29:17.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:29:17.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:29:17.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:29:17.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:29:17.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:29:17.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:29:17.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:29:17.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:29:17.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:29:17.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:29:17.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:29:17.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:29:17.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:29:17.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:29:17.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:29:17.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:29:17.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:29:17.332 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.332 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.333 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.333 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.333 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:17.333 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098400766000001, 2024-09-17T07:29:17.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:29:17.333 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.333 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.333 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.333 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.333 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:29:17.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.334 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:17.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:17.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:17.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:29:17.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:29:17.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:17.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:29:17.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:29:17.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:29:17.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:29:17.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:29:17.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:29:17.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:29:17.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:29:17.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:29:17.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:29:17.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:29:17.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:29:17.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:29:17.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:29:17.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:29:17.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:29:17.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:29:17.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:29:17.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.338 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.339 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.339 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:17.339 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098368962999999, 2024-09-17T07:29:17.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:29:17.339 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.339 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.339 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.339 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:29:17.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.340 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:17.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:17.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:29:17.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:29:17.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:17.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:29:17.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:29:17.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:29:17.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:29:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:29:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:29:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:29:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:29:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:29:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:29:17.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:29:17.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:29:17.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:29:17.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:29:17.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:29:17.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:29:17.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:29:17.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:29:17.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:29:17.344 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.344 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.344 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.344 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.345 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:17.345 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098342016, 2024-09-17T07:29:17.345 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:29:17.345 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.345 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.345 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.345 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.345 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.345 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.345 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:29:17.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.345 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.345 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:17.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:17.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:29:17.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:29:17.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.347 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:29:17.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:29:17.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.347 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.347 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:17.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:29:17.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.347 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.347 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:17.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:29:17.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:29:17.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:29:17.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:29:17.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:29:17.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:29:17.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:29:17.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:29:17.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:29:17.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:29:17.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:29:17.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:29:17.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:29:17.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:29:17.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:29:17.350 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.350 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.350 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.350 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:17.350 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:17.350 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.098301067000001, 2024-09-17T07:29:17.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:29:17.351 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.351 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.351 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:17.351 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:17.351 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:17.351 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:29:17.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.351 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.351 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.351 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:17.351 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:17.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:29:17.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:29:17.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:29:17.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:29:17.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.353 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.353 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:29:17.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:29:17.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.353 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.353 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:29:17.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:29:17.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.353 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.353 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:29:17.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:29:17.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:17.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:29:17.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:29:17.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:29:17.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:29:17.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:17.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:29:17.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:29:17.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.355 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:29:17.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:29:17.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.355 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:29:17.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:29:17.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.355 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:29:17.356 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:29:17.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.356 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.356 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.356 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:17.357 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:17.358 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:17.357+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:17.392 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-17T07:29:17.403 INFO:teuthology.orchestra.run.smithi045.stdout:1779 punch from 0x2426287 to 0x24282af, (0x2028 bytes) 2024-09-17T07:29:17.406 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:17.406 INFO:teuthology.orchestra.run.smithi045.stdout:1782 punch from 0x1b1008a to 0x1b11583, (0x14f9 bytes) 2024-09-17T07:29:17.406 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:17.407 INFO:teuthology.orchestra.run.smithi045.stdout:1785 writesame 0x775b819 thru 0x7766228 data_size 0x18(0xaa10 bytes) 2024-09-17T07:29:17.416 INFO:teuthology.orchestra.run.smithi045.stdout:1786 write 0x45ee432 thru 0x45f8aec (0xa6bb bytes) 2024-09-17T07:29:17.428 INFO:teuthology.orchestra.run.smithi045.stdout:1787 write 0x5268907 thru 0x527730a (0xea04 bytes) 2024-09-17T07:29:17.438 INFO:teuthology.orchestra.run.smithi045.stdout:1788 punch from 0xa365fc7 to 0xa37064b, (0xa684 bytes) 2024-09-17T07:29:17.438 INFO:teuthology.orchestra.run.smithi045.stdout:1789 read 0xcc2a1 thru 0xd9093 (0xcdf3 bytes) 2024-09-17T07:29:17.438 INFO:teuthology.orchestra.run.smithi045.stdout:1790 write 0xe1b3d1 thru 0xe2707a (0xbcaa bytes) 2024-09-17T07:29:17.458 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:17.457+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:17.466 INFO:teuthology.orchestra.run.smithi045.stdout:1791 write 0xe321567 thru 0xe32926e (0x7d08 bytes) 2024-09-17T07:29:17.470 INFO:teuthology.orchestra.run.smithi045.stdout:1793 read 0xbd70716 thru 0xbd7de28 (0xd713 bytes) 2024-09-17T07:29:17.471 INFO:teuthology.orchestra.run.smithi045.stdout:1794 write 0x321d007 thru 0x322100b (0x4005 bytes) 2024-09-17T07:29:17.476 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:17.477 INFO:teuthology.orchestra.run.smithi045.stdout:1798 read 0xde73515 thru 0xde77a21 (0x450d bytes) 2024-09-17T07:29:17.477 INFO:teuthology.orchestra.run.smithi045.stdout:1799 punch from 0x66c92b4 to 0x66d352e, (0xa27a bytes) 2024-09-17T07:29:17.477 INFO:teuthology.orchestra.run.smithi045.stdout:1800 read 0x56f770d thru 0x57059db (0xe2cf bytes) 2024-09-17T07:29:17.484 INFO:teuthology.orchestra.run.smithi045.stdout:1803 trunc from 0xe32926f to 0x7b11c96 2024-09-17T07:29:17.514 INFO:teuthology.orchestra.run.smithi045.stdout:1804 trunc from 0x7b11c96 to 0x2cb5160 2024-09-17T07:29:17.519 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:17.520 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:17.520 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:17.520 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:17.532 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:17.559 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:17.558+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:17.563 INFO:teuthology.orchestra.run.smithi045.stdout:1806 trunc from 0x2cb5160 to 0xdf68313 2024-09-17T07:29:17.569 INFO:teuthology.orchestra.run.smithi045.stdout:1807 punch from 0x34bad3b to 0x34c29fa, (0x7cbf bytes) 2024-09-17T07:29:17.569 INFO:teuthology.orchestra.run.smithi045.stdout:1808 writesame 0x5866898 thru 0x58754a9 data_size 0x29(0xec12 bytes) 2024-09-17T07:29:17.575 INFO:teuthology.orchestra.run.smithi045.stdout:1809 read 0x1a6ad58 thru 0x1a70ceb (0x5f94 bytes) 2024-09-17T07:29:17.580 INFO:teuthology.orchestra.run.smithi045.stdout:1810 trunc from 0xdf68313 to 0x9db327f 2024-09-17T07:29:17.585 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:17.588 INFO:teuthology.orchestra.run.smithi045.stdout:1813 trunc from 0x9db327f to 0x7da57fd 2024-09-17T07:29:17.593 INFO:teuthology.orchestra.run.smithi045.stdout:1814 write 0x49ed3f6 thru 0x49f92ac (0xbeb7 bytes) 2024-09-17T07:29:17.610 INFO:teuthology.orchestra.run.smithi045.stdout:1817 write 0xd08e7d1 thru 0xd093cd7 (0x5507 bytes) 2024-09-17T07:29:17.614 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:17.614 INFO:teuthology.orchestra.run.smithi045.stdout:1819 read 0x5fbebf8 thru 0x5fce5f9 (0xfa02 bytes) 2024-09-17T07:29:17.614 INFO:teuthology.orchestra.run.smithi045.stdout:1820 read 0x2c806ad thru 0x2c86ee2 (0x6836 bytes) 2024-09-17T07:29:17.615 INFO:teuthology.orchestra.run.smithi045.stdout:1821 read 0x7b8228d thru 0x7b90492 (0xe206 bytes) 2024-09-17T07:29:17.620 INFO:teuthology.orchestra.run.smithi045.stdout:1824 write 0xd649c9b thru 0xd6565ba (0xc920 bytes) 2024-09-17T07:29:17.626 INFO:teuthology.orchestra.run.smithi045.stdout:1826 punch from 0xba837bf to 0xba93258, (0xfa99 bytes) 2024-09-17T07:29:17.626 INFO:teuthology.orchestra.run.smithi045.stdout:1827 write 0xb8ddf98 thru 0xb8edf4e (0xffb7 bytes) 2024-09-17T07:29:17.631 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:17.631 INFO:teuthology.orchestra.run.smithi045.stdout:1829 punch from 0x6339aea to 0x633b483, (0x1999 bytes) 2024-09-17T07:29:17.631 INFO:teuthology.orchestra.run.smithi045.stdout:1830 punch from 0xd4b544 to 0xd4f75d, (0x4219 bytes) 2024-09-17T07:29:17.631 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:17.631 INFO:teuthology.orchestra.run.smithi045.stdout:1833 read 0xb0d7fec thru 0xb0da829 (0x283e bytes) 2024-09-17T07:29:17.631 INFO:teuthology.orchestra.run.smithi045.stdout:1834 write 0xd61d9af thru 0xd61f66b (0x1cbd bytes) 2024-09-17T07:29:17.631 INFO:teuthology.orchestra.run.smithi045.stdout:1835 write 0xaab47fe thru 0xaab5330 (0xb33 bytes) 2024-09-17T07:29:17.642 INFO:teuthology.orchestra.run.smithi045.stdout:1836 trunc from 0xd6565bb to 0x63511f4 2024-09-17T07:29:17.655 INFO:teuthology.orchestra.run.smithi045.stdout:1837 write 0x5094c5b thru 0x509a3c0 (0x5766 bytes) 2024-09-17T07:29:17.660 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:17.659+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:17.661 INFO:teuthology.orchestra.run.smithi045.stdout:1838 punch from 0x2072b3b to 0x2079a49, (0x6f0e bytes) 2024-09-17T07:29:17.661 INFO:teuthology.orchestra.run.smithi045.stdout:1839 punch from 0x5a37746 to 0x5a3e1ba, (0x6a74 bytes) 2024-09-17T07:29:17.661 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:17.661 INFO:teuthology.orchestra.run.smithi045.stdout:1841 read 0xe5da0a thru 0xe6ad2e (0xd325 bytes) 2024-09-17T07:29:17.661 INFO:teuthology.orchestra.run.smithi045.stdout:1842 write 0x2248520 thru 0x22561f6 (0xdcd7 bytes) 2024-09-17T07:29:17.663 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:17.671 INFO:teuthology.orchestra.run.smithi045.stdout:1845 write 0x93890dc thru 0x93979e9 (0xe90e bytes) 2024-09-17T07:29:17.681 INFO:teuthology.orchestra.run.smithi045.stdout:1847 write 0x3b99097 thru 0x3ba4bc8 (0xbb32 bytes) 2024-09-17T07:29:17.682 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:17.682 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:17.682 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:17.682 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:17.682 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:17.690 INFO:teuthology.orchestra.run.smithi045.stdout:1848 read 0x5ebe152 thru 0x5ec9e83 (0xbd32 bytes) 2024-09-17T07:29:17.691 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:17.701 INFO:teuthology.orchestra.run.smithi045.stdout:1849 write 0xd6a0969 thru 0xd6a40aa (0x3742 bytes) 2024-09-17T07:29:17.705 INFO:teuthology.orchestra.run.smithi045.stdout:1851 read 0x5977d7f thru 0x597a3f7 (0x2679 bytes) 2024-09-17T07:29:17.705 INFO:teuthology.orchestra.run.smithi045.stdout:1852 read 0x95340f0 thru 0x9534e73 (0xd84 bytes) 2024-09-17T07:29:17.706 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:17.706 INFO:teuthology.orchestra.run.smithi045.stdout:1855 write 0x793f158 thru 0x7940dc3 (0x1c6c bytes) 2024-09-17T07:29:17.710 INFO:teuthology.orchestra.run.smithi045.stdout:1856 read 0x6b70d11 thru 0x6b74e79 (0x4169 bytes) 2024-09-17T07:29:17.710 INFO:teuthology.orchestra.run.smithi045.stdout:1857 read 0x30377bb thru 0x30382ab (0xaf1 bytes) 2024-09-17T07:29:17.717 INFO:teuthology.orchestra.run.smithi045.stdout:1859 trunc from 0xd6a40ab to 0x770be76 2024-09-17T07:29:17.733 INFO:teuthology.orchestra.run.smithi045.stdout:1860 write 0x7dd2079 thru 0x7dd52dc (0x3264 bytes) 2024-09-17T07:29:17.736 INFO:teuthology.orchestra.run.smithi045.stdout:1862 writesame 0x7bf3b06 thru 0x7c035cd data_size 0x5(0xfac8 bytes) 2024-09-17T07:29:17.744 INFO:teuthology.orchestra.run.smithi045.stdout:1864 write 0x1179ff thru 0x11c1dc (0x47de bytes) 2024-09-17T07:29:17.753 INFO:teuthology.orchestra.run.smithi045.stdout:1865 read 0x59d368 thru 0x5ac223 (0xeebc bytes) 2024-09-17T07:29:17.753 INFO:teuthology.orchestra.run.smithi045.stdout:1867 punch from 0xf295f1 to 0xf30c76, (0x7685 bytes) 2024-09-17T07:29:17.758 INFO:teuthology.orchestra.run.smithi045.stdout:1869 trunc from 0x7dd52dd to 0xc879d9a 2024-09-17T07:29:17.761 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:17.761+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:17.763 INFO:teuthology.orchestra.run.smithi045.stdout:1871 read 0x7b3727e thru 0x7b3903d (0x1dc0 bytes) 2024-09-17T07:29:17.763 INFO:teuthology.orchestra.run.smithi045.stdout:1872 punch from 0x7641704 to 0x764da64, (0xc360 bytes) 2024-09-17T07:29:17.763 INFO:teuthology.orchestra.run.smithi045.stdout:1873 write 0x94ec49e thru 0x94fae74 (0xe9d7 bytes) 2024-09-17T07:29:17.770 INFO:teuthology.orchestra.run.smithi045.stdout:1874 write 0x1d0ece2 thru 0x1d0f7a7 (0xac6 bytes) 2024-09-17T07:29:17.771 INFO:teuthology.orchestra.run.smithi045.stdout:1875 write 0x9533434 thru 0x953cb15 (0x96e2 bytes) 2024-09-17T07:29:17.777 INFO:teuthology.orchestra.run.smithi045.stdout:1877 writesame 0x2990c93 thru 0x29925aa data_size 0xb(0x1918 bytes) 2024-09-17T07:29:17.778 INFO:teuthology.orchestra.run.smithi045.stdout:1878 read 0x8bc81db thru 0x8bd081f (0x8645 bytes) 2024-09-17T07:29:17.779 INFO:teuthology.orchestra.run.smithi045.stdout:1879 write 0x30097b6 thru 0x300d5ad (0x3df8 bytes) 2024-09-17T07:29:17.781 INFO:teuthology.orchestra.run.smithi045.stdout:1880 write 0x7a0a314 thru 0x7a16328 (0xc015 bytes) 2024-09-17T07:29:17.787 INFO:teuthology.orchestra.run.smithi045.stdout:1881 trunc from 0xc879d9a to 0xaeea665 2024-09-17T07:29:17.792 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:17.792 INFO:teuthology.orchestra.run.smithi045.stdout:1883 write 0x594dec5 thru 0x594faa8 (0x1be4 bytes) 2024-09-17T07:29:17.796 INFO:teuthology.orchestra.run.smithi045.stdout:1884 read 0xae4c44b thru 0xae51d58 (0x590e bytes) 2024-09-17T07:29:17.796 INFO:teuthology.orchestra.run.smithi045.stdout:1885 write 0x4c124f0 thru 0x4c1d51d (0xb02e bytes) 2024-09-17T07:29:17.802 INFO:teuthology.orchestra.run.smithi045.stdout:1887 read 0x854ffdb thru 0x8553bca (0x3bf0 bytes) 2024-09-17T07:29:17.803 INFO:teuthology.orchestra.run.smithi045.stdout:1888 write 0x79061af thru 0x79109b3 (0xa805 bytes) 2024-09-17T07:29:17.806 INFO:teuthology.orchestra.run.smithi045.stdout:1889 punch from 0x368a0db to 0x368a712, (0x637 bytes) 2024-09-17T07:29:17.806 INFO:teuthology.orchestra.run.smithi045.stdout:1890 write 0x5648d7f thru 0x5652baa (0x9e2c bytes) 2024-09-17T07:29:17.812 INFO:teuthology.orchestra.run.smithi045.stdout:1893 writesame 0xfcbcc thru 0x106023 data_size 0x65(0x9458 bytes) 2024-09-17T07:29:17.822 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:17.822 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:17.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:17.822 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:17.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.822 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:4848 3.9 e33/23 3:97363921:::rbd_data.1183cff49e3a.0000000000000004:head v 33'102, mlcod=32'100)", 2024-09-17T07:29:17.822 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:17.823 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 23.581027918, 2024-09-17T07:29:17.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023715110000000002, 2024-09-17T07:29:17.823 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.823 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.823 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:17.823 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:17.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:17.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239497+0000", 2024-09-17T07:29:17.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.53e-06 2024-09-17T07:29:17.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240142+0000", 2024-09-17T07:29:17.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00064481300000000005 2024-09-17T07:29:17.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240160+0000", 2024-09-17T07:29:17.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8349999999999999e-05 2024-09-17T07:29:17.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240169+0000", 2024-09-17T07:29:17.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7690000000000002e-06 2024-09-17T07:29:17.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240246+0000", 2024-09-17T07:29:17.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6277000000000002e-05 2024-09-17T07:29:17.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240270+0000", 2024-09-17T07:29:17.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4575000000000001e-05 2024-09-17T07:29:17.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:17.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.241813+0000", 2024-09-17T07:29:17.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015432860000000001 2024-09-17T07:29:17.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.241867+0000", 2024-09-17T07:29:17.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3910999999999999e-05 2024-09-17T07:29:17.827 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.827 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.827 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.827 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:5460 3.9 e33/23 3:90773955:::rbd_data.1183cff49e3a.000000000000000a:head v 33'106, mlcod=33'104)", 2024-09-17T07:29:17.827 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:17.827 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 23.293931332, 2024-09-17T07:29:17.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020825850000000001, 2024-09-17T07:29:17.827 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:17.828 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:17.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:17.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526593+0000", 2024-09-17T07:29:17.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.54e-07 2024-09-17T07:29:17.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527050+0000", 2024-09-17T07:29:17.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000456522 2024-09-17T07:29:17.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527061+0000", 2024-09-17T07:29:17.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1722000000000001e-05 2024-09-17T07:29:17.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527067+0000", 2024-09-17T07:29:17.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0599999999999998e-06 2024-09-17T07:29:17.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527112+0000", 2024-09-17T07:29:17.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999000000000001e-05 2024-09-17T07:29:17.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527128+0000", 2024-09-17T07:29:17.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6920999999999999e-05 2024-09-17T07:29:17.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:17.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528636+0000", 2024-09-17T07:29:17.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001507836 2024-09-17T07:29:17.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528675+0000", 2024-09-17T07:29:17.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8671000000000003e-05 2024-09-17T07:29:17.832 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.832 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.832 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.832 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:6005 3.0 e33/23 3:037cd38f:::rbd_data.1183cff49e3a.0000000000000012:head v 33'25, mlcod=32'23)", 2024-09-17T07:29:17.833 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:17.833 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 23.195898197999998, 2024-09-17T07:29:17.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019123899999999999, 2024-09-17T07:29:17.833 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:17.833 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:17.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:17.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624627+0000", 2024-09-17T07:29:17.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5600000000000001e-06 2024-09-17T07:29:17.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624777+0000", 2024-09-17T07:29:17.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015030800000000001 2024-09-17T07:29:17.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624792+0000", 2024-09-17T07:29:17.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5089000000000001e-05 2024-09-17T07:29:17.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624799+0000", 2024-09-17T07:29:17.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7939999999999997e-06 2024-09-17T07:29:17.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624855+0000", 2024-09-17T07:29:17.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5782999999999998e-05 2024-09-17T07:29:17.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624871+0000", 2024-09-17T07:29:17.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6192e-05 2024-09-17T07:29:17.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:17.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.626507+0000", 2024-09-17T07:29:17.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016354500000000001 2024-09-17T07:29:17.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.626538+0000", 2024-09-17T07:29:17.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1214000000000003e-05 2024-09-17T07:29:17.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.837 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.837 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:6683 3.9 e33/23 3:97ddc208:::rbd_data.1183cff49e3a.000000000000001e:head v 33'110, mlcod=33'108)", 2024-09-17T07:29:17.838 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.764442+0000", 2024-09-17T07:29:17.838 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 23.056081796000001, 2024-09-17T07:29:17.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017329470000000001, 2024-09-17T07:29:17.838 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.838 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:17.838 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.764442+0000", 2024-09-17T07:29:17.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.764442+0000", 2024-09-17T07:29:17.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.764443+0000", 2024-09-17T07:29:17.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:29:17.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765126+0000", 2024-09-17T07:29:17.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00068291100000000002 2024-09-17T07:29:17.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765146+0000", 2024-09-17T07:29:17.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9406999999999999e-05 2024-09-17T07:29:17.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765156+0000", 2024-09-17T07:29:17.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0037000000000001e-05 2024-09-17T07:29:17.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765209+0000", 2024-09-17T07:29:17.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3732e-05 2024-09-17T07:29:17.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.765226+0000", 2024-09-17T07:29:17.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6787e-05 2024-09-17T07:29:17.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:17.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.766139+0000", 2024-09-17T07:29:17.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00091277899999999996 2024-09-17T07:29:17.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.766175+0000", 2024-09-17T07:29:17.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5942999999999998e-05 2024-09-17T07:29:17.842 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.842 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.842 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.842 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:7670 3.4 e33/23 3:20e02957:::rbd_data.1183cff49e3a.000000000000002d:head v 33'30, mlcod=32'28)", 2024-09-17T07:29:17.842 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.955123+0000", 2024-09-17T07:29:17.842 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.865400335, 2024-09-17T07:29:17.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0018080839999999999, 2024-09-17T07:29:17.842 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.842 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:17.842 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955123+0000", 2024-09-17T07:29:17.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955123+0000", 2024-09-17T07:29:17.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955125+0000", 2024-09-17T07:29:17.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3060000000000001e-06 2024-09-17T07:29:17.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955285+0000", 2024-09-17T07:29:17.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00016064700000000001 2024-09-17T07:29:17.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955303+0000", 2024-09-17T07:29:17.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7465999999999998e-05 2024-09-17T07:29:17.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955311+0000", 2024-09-17T07:29:17.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4780000000000008e-06 2024-09-17T07:29:17.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955373+0000", 2024-09-17T07:29:17.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1977000000000006e-05 2024-09-17T07:29:17.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.955388+0000", 2024-09-17T07:29:17.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4260000000000001e-05 2024-09-17T07:29:17.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:17.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.956893+0000", 2024-09-17T07:29:17.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001505309 2024-09-17T07:29:17.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.956931+0000", 2024-09-17T07:29:17.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8640999999999998e-05 2024-09-17T07:29:17.846 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.846 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.846 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.847 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8595 3.4 e33/23 3:237b2a73:::rbd_data.1183cff49e3a.0000000000000049:head v 33'37, mlcod=33'35)", 2024-09-17T07:29:17.847 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:17.847 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.745881238999999, 2024-09-17T07:29:17.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020228939999999999, 2024-09-17T07:29:17.847 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.847 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.847 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:17.847 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:17.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:17.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074643+0000", 2024-09-17T07:29:17.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3099999999999997e-07 2024-09-17T07:29:17.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074834+0000", 2024-09-17T07:29:17.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000191137 2024-09-17T07:29:17.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074850+0000", 2024-09-17T07:29:17.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6101999999999999e-05 2024-09-17T07:29:17.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074859+0000", 2024-09-17T07:29:17.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2169999999999994e-06 2024-09-17T07:29:17.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074919+0000", 2024-09-17T07:29:17.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.003e-05 2024-09-17T07:29:17.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074944+0000", 2024-09-17T07:29:17.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5761e-05 2024-09-17T07:29:17.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:17.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076622+0000", 2024-09-17T07:29:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001677977 2024-09-17T07:29:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076665+0000", 2024-09-17T07:29:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2938999999999997e-05 2024-09-17T07:29:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17069 3.0 e41/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 41'110, mlcod=40'108)", 2024-09-17T07:29:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.080771515, 2024-09-17T07:29:17.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011408908000000001, 2024-09-17T07:29:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739753+0000", 2024-09-17T07:29:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:29:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743532+0000", 2024-09-17T07:29:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037794510000000001 2024-09-17T07:29:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743557+0000", 2024-09-17T07:29:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5253e-05 2024-09-17T07:29:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743569+0000", 2024-09-17T07:29:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1161999999999999e-05 2024-09-17T07:29:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743650+0000", 2024-09-17T07:29:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1087999999999993e-05 2024-09-17T07:29:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743674+0000", 2024-09-17T07:29:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4562e-05 2024-09-17T07:29:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751095+0000", 2024-09-17T07:29:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074211140000000004 2024-09-17T07:29:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751161+0000", 2024-09-17T07:29:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5707e-05 2024-09-17T07:29:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.855 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17240 3.0 e42/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 42'113, mlcod=41'110)", 2024-09-17T07:29:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.595113031, 2024-09-17T07:29:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064392440000000002, 2024-09-17T07:29:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:29:17.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225418+0000", 2024-09-17T07:29:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8360000000000003e-06 2024-09-17T07:29:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225426+0000", 2024-09-17T07:29:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2670000000000006e-06 2024-09-17T07:29:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225434+0000", 2024-09-17T07:29:17.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0560000000000005e-06 2024-09-17T07:29:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225514+0000", 2024-09-17T07:29:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9979000000000001e-05 2024-09-17T07:29:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225535+0000", 2024-09-17T07:29:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0965e-05 2024-09-17T07:29:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231797+0000", 2024-09-17T07:29:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062621129999999997 2024-09-17T07:29:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231850+0000", 2024-09-17T07:29:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2611999999999997e-05 2024-09-17T07:29:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20054 3.9 e44/23 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:head v 44'131, mlcod=44'129)", 2024-09-17T07:29:17.860 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 12.552961915000001, 2024-09-17T07:29:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028510570000000002, 2024-09-17T07:29:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:17.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267563+0000", 2024-09-17T07:29:17.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.584e-06 2024-09-17T07:29:17.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268269+0000", 2024-09-17T07:29:17.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00070567399999999999 2024-09-17T07:29:17.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268287+0000", 2024-09-17T07:29:17.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.84e-05 2024-09-17T07:29:17.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268297+0000", 2024-09-17T07:29:17.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9059999999999997e-06 2024-09-17T07:29:17.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268368+0000", 2024-09-17T07:29:17.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0562999999999995e-05 2024-09-17T07:29:17.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268393+0000", 2024-09-17T07:29:17.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4958999999999999e-05 2024-09-17T07:29:17.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:17.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270330+0000", 2024-09-17T07:29:17.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001936644 2024-09-17T07:29:17.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270413+0000", 2024-09-17T07:29:17.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3326999999999997e-05 2024-09-17T07:29:17.865 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.865 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.865 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.865 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20055 3.9 e44/23 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:head v 44'133, mlcod=44'129)", 2024-09-17T07:29:17.865 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.268886+0000", 2024-09-17T07:29:17.865 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 12.551637844, 2024-09-17T07:29:17.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019232920000000001, 2024-09-17T07:29:17.865 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.865 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.865 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:17.865 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268886+0000", 2024-09-17T07:29:17.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268886+0000", 2024-09-17T07:29:17.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268887+0000", 2024-09-17T07:29:17.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.85e-07 2024-09-17T07:29:17.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268905+0000", 2024-09-17T07:29:17.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8263000000000001e-05 2024-09-17T07:29:17.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268916+0000", 2024-09-17T07:29:17.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0876000000000001e-05 2024-09-17T07:29:17.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268922+0000", 2024-09-17T07:29:17.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7579999999999998e-06 2024-09-17T07:29:17.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.269201+0000", 2024-09-17T07:29:17.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027979200000000001 2024-09-17T07:29:17.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.269224+0000", 2024-09-17T07:29:17.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2838999999999999e-05 2024-09-17T07:29:17.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:17.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270774+0000", 2024-09-17T07:29:17.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015495629999999999 2024-09-17T07:29:17.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270809+0000", 2024-09-17T07:29:17.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5416000000000001e-05 2024-09-17T07:29:17.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.869 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.870 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20163 3.9 e44/23 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:head v 44'134, mlcod=44'131)", 2024-09-17T07:29:17.870 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.294194+0000", 2024-09-17T07:29:17.870 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 12.526329493, 2024-09-17T07:29:17.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001874824, 2024-09-17T07:29:17.870 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.870 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:17.870 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294194+0000", 2024-09-17T07:29:17.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294194+0000", 2024-09-17T07:29:17.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294196+0000", 2024-09-17T07:29:17.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3599999999999999e-06 2024-09-17T07:29:17.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294663+0000", 2024-09-17T07:29:17.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00046696200000000002 2024-09-17T07:29:17.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294683+0000", 2024-09-17T07:29:17.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0829999999999999e-05 2024-09-17T07:29:17.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294697+0000", 2024-09-17T07:29:17.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3524e-05 2024-09-17T07:29:17.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294758+0000", 2024-09-17T07:29:17.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0618999999999997e-05 2024-09-17T07:29:17.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294776+0000", 2024-09-17T07:29:17.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8431e-05 2024-09-17T07:29:17.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:17.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.296042+0000", 2024-09-17T07:29:17.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012656329999999999 2024-09-17T07:29:17.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.296069+0000", 2024-09-17T07:29:17.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7464999999999999e-05 2024-09-17T07:29:17.874 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.874 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.874 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.874 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:26232 3.9 e47/23 3:94b30449:::rbd_data.11834cd21cdd.000000000000000c:head v 47'146, mlcod=45'143)", 2024-09-17T07:29:17.874 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.272078+0000", 2024-09-17T07:29:17.874 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.5484458500000002, 2024-09-17T07:29:17.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001742335, 2024-09-17T07:29:17.874 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.874 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.875 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:17.875 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.272078+0000", 2024-09-17T07:29:17.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.272078+0000", 2024-09-17T07:29:17.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.272078+0000", 2024-09-17T07:29:17.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3000000000000002e-07 2024-09-17T07:29:17.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.272084+0000", 2024-09-17T07:29:17.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5199999999999997e-06 2024-09-17T07:29:17.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.272095+0000", 2024-09-17T07:29:17.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1787000000000001e-05 2024-09-17T07:29:17.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.272104+0000", 2024-09-17T07:29:17.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.867e-06 2024-09-17T07:29:17.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.272166+0000", 2024-09-17T07:29:17.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1618999999999994e-05 2024-09-17T07:29:17.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.272187+0000", 2024-09-17T07:29:17.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0829e-05 2024-09-17T07:29:17.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:17.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.273785+0000", 2024-09-17T07:29:17.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015978780000000001 2024-09-17T07:29:17.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.273820+0000", 2024-09-17T07:29:17.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5505e-05 2024-09-17T07:29:17.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.878 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.879 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.1f 51/50 [PushOp(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head, version: 40'83, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head@40'83, size: 19583394, copy_subset: [(0, 19583394)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:29:17.879 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:13.339735+0000", 2024-09-17T07:29:17.879 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.4807891460000002, 2024-09-17T07:29:17.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0018873760000000001, 2024-09-17T07:29:17.879 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.879 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.879 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.339735+0000", 2024-09-17T07:29:17.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.339735+0000", 2024-09-17T07:29:17.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.339736+0000", 2024-09-17T07:29:17.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.392e-06 2024-09-17T07:29:17.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340637+0000", 2024-09-17T07:29:17.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00090133599999999998 2024-09-17T07:29:17.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340668+0000", 2024-09-17T07:29:17.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0488e-05 2024-09-17T07:29:17.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340683+0000", 2024-09-17T07:29:17.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5315000000000002e-05 2024-09-17T07:29:17.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340777+0000", 2024-09-17T07:29:17.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3456999999999996e-05 2024-09-17T07:29:17.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340786+0000", 2024-09-17T07:29:17.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2220000000000008e-06 2024-09-17T07:29:17.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.341622+0000", 2024-09-17T07:29:17.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083616599999999995 2024-09-17T07:29:17.882 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.883 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.883 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.883 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.1f 51/50 [PushOp(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head, version: 40'83, data_included: [1310720~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head@40'83, size: 19583394, copy_subset: [(0, 19583394)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2359296, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:29:17.883 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:13.343605+0000", 2024-09-17T07:29:17.883 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.4769185189999998, 2024-09-17T07:29:17.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001842457, 2024-09-17T07:29:17.883 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.883 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.883 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.883 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.343605+0000", 2024-09-17T07:29:17.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.343605+0000", 2024-09-17T07:29:17.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.343606+0000", 2024-09-17T07:29:17.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1030000000000001e-06 2024-09-17T07:29:17.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.344579+0000", 2024-09-17T07:29:17.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097232999999999998 2024-09-17T07:29:17.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.344610+0000", 2024-09-17T07:29:17.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1272000000000001e-05 2024-09-17T07:29:17.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.344625+0000", 2024-09-17T07:29:17.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4728e-05 2024-09-17T07:29:17.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.344714+0000", 2024-09-17T07:29:17.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9134000000000004e-05 2024-09-17T07:29:17.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.344726+0000", 2024-09-17T07:29:17.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2176e-05 2024-09-17T07:29:17.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.345448+0000", 2024-09-17T07:29:17.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072171400000000004 2024-09-17T07:29:17.887 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.887 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.887 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.887 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.1f 51/50 [PushOp(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head, version: 40'83, data_included: [2359296~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head@40'83, size: 19583394, copy_subset: [(0, 19583394)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3407872, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2359296, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:29:17.887 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:13.346978+0000", 2024-09-17T07:29:17.887 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.4735459769999997, 2024-09-17T07:29:17.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001738826, 2024-09-17T07:29:17.887 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.888 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.888 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.346978+0000", 2024-09-17T07:29:17.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.346978+0000", 2024-09-17T07:29:17.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.346979+0000", 2024-09-17T07:29:17.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.172e-06 2024-09-17T07:29:17.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.347954+0000", 2024-09-17T07:29:17.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097514699999999999 2024-09-17T07:29:17.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.347982+0000", 2024-09-17T07:29:17.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7535999999999999e-05 2024-09-17T07:29:17.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.347995+0000", 2024-09-17T07:29:17.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3691e-05 2024-09-17T07:29:17.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.348084+0000", 2024-09-17T07:29:17.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8819000000000005e-05 2024-09-17T07:29:17.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.348095+0000", 2024-09-17T07:29:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0679000000000001e-05 2024-09-17T07:29:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.348717+0000", 2024-09-17T07:29:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00062178199999999998 2024-09-17T07:29:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.1f 51/50 [PushOp(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head, version: 40'83, data_included: [3407872~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head@40'83, size: 19583394, copy_subset: [(0, 19583394)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4456448, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3407872, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:29:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:13.350296+0000", 2024-09-17T07:29:17.891 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.4702282010000003, 2024-09-17T07:29:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00191949, 2024-09-17T07:29:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.350296+0000", 2024-09-17T07:29:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.350296+0000", 2024-09-17T07:29:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.350297+0000", 2024-09-17T07:29:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4440000000000001e-06 2024-09-17T07:29:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351304+0000", 2024-09-17T07:29:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001007319 2024-09-17T07:29:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351331+0000", 2024-09-17T07:29:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6701000000000001e-05 2024-09-17T07:29:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351346+0000", 2024-09-17T07:29:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4911e-05 2024-09-17T07:29:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351425+0000", 2024-09-17T07:29:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9328999999999999e-05 2024-09-17T07:29:17.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351436+0000", 2024-09-17T07:29:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0467e-05 2024-09-17T07:29:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.352215+0000", 2024-09-17T07:29:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000779319 2024-09-17T07:29:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.1f 51/50 [PushOp(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head, version: 40'83, data_included: [4456448~524288,5242880~524288], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head@40'83, size: 19583394, copy_subset: [(0, 19583394)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5767168, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4456448, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:29:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:13.353684+0000", 2024-09-17T07:29:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.4668398790000001, 2024-09-17T07:29:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017904640000000001, 2024-09-17T07:29:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.353684+0000", 2024-09-17T07:29:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.353684+0000", 2024-09-17T07:29:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.353685+0000", 2024-09-17T07:29:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0839999999999999e-06 2024-09-17T07:29:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.354695+0000", 2024-09-17T07:29:17.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010101489999999999 2024-09-17T07:29:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.354726+0000", 2024-09-17T07:29:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0497e-05 2024-09-17T07:29:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.354740+0000", 2024-09-17T07:29:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4260999999999999e-05 2024-09-17T07:29:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.354811+0000", 2024-09-17T07:29:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1491999999999999e-05 2024-09-17T07:29:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.354822+0000", 2024-09-17T07:29:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0835999999999999e-05 2024-09-17T07:29:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.355474+0000", 2024-09-17T07:29:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00065214500000000003 2024-09-17T07:29:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.899 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.1f 51/50 [PushOp(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head, version: 40'83, data_included: [5767168~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head@40'83, size: 19583394, copy_subset: [(0, 19583394)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 6815744, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5767168, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:29:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:13.356876+0000", 2024-09-17T07:29:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.4636479189999996, 2024-09-17T07:29:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017289790000000001, 2024-09-17T07:29:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.356876+0000", 2024-09-17T07:29:17.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.356876+0000", 2024-09-17T07:29:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.356877+0000", 2024-09-17T07:29:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6500000000000008e-07 2024-09-17T07:29:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.357845+0000", 2024-09-17T07:29:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00096788600000000005 2024-09-17T07:29:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.357867+0000", 2024-09-17T07:29:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2348e-05 2024-09-17T07:29:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.357879+0000", 2024-09-17T07:29:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2490000000000001e-05 2024-09-17T07:29:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.357957+0000", 2024-09-17T07:29:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7835000000000003e-05 2024-09-17T07:29:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.357967+0000", 2024-09-17T07:29:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8740000000000007e-06 2024-09-17T07:29:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.358605+0000", 2024-09-17T07:29:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063758100000000002 2024-09-17T07:29:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.903 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:32543 3.0 e55/23 3:03bfd97d:::rbd_data.11839a97b753.0000000000000007:head v 55'150, mlcod=54'148)", 2024-09-17T07:29:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.36588677800000002, 2024-09-17T07:29:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031813480000000001, 2024-09-17T07:29:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5199999999999998e-07 2024-09-17T07:29:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455083+0000", 2024-09-17T07:29:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044629199999999999 2024-09-17T07:29:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455098+0000", 2024-09-17T07:29:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4183e-05 2024-09-17T07:29:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455106+0000", 2024-09-17T07:29:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:29:17.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455173+0000", 2024-09-17T07:29:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6346000000000005e-05 2024-09-17T07:29:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455200+0000", 2024-09-17T07:29:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7319000000000001e-05 2024-09-17T07:29:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457774+0000", 2024-09-17T07:29:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025742650000000001 2024-09-17T07:29:17.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457818+0000", 2024-09-17T07:29:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3891999999999999e-05 2024-09-17T07:29:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:32827 3.9 e55/23 3:94cde91d:::rbd_data.11839a97b753.0000000000000037:head v 55'164, mlcod=54'162)", 2024-09-17T07:29:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.500414+0000", 2024-09-17T07:29:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.320109641, 2024-09-17T07:29:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017425209999999999, 2024-09-17T07:29:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:17.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.500414+0000", 2024-09-17T07:29:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.500414+0000", 2024-09-17T07:29:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.500415+0000", 2024-09-17T07:29:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1689999999999999e-06 2024-09-17T07:29:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.500610+0000", 2024-09-17T07:29:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000194463 2024-09-17T07:29:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.500629+0000", 2024-09-17T07:29:17.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9704e-05 2024-09-17T07:29:17.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:17.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.500640+0000", 2024-09-17T07:29:17.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0356e-05 2024-09-17T07:29:17.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:17.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.500707+0000", 2024-09-17T07:29:17.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7173000000000002e-05 2024-09-17T07:29:17.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:17.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.500725+0000", 2024-09-17T07:29:17.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7779999999999999e-05 2024-09-17T07:29:17.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.912 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:17.912 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.502124+0000", 2024-09-17T07:29:17.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001399174 2024-09-17T07:29:17.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:17.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:17.912 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:17.912 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.502157+0000", 2024-09-17T07:29:17.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2702e-05 2024-09-17T07:29:17.912 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.912 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.912 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.913 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:17.913 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:17.913 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:17.914 INFO:teuthology.orchestra.run.smithi045.stdout:1895 write 0x61679e thru 0x625502 (0xed65 bytes) 2024-09-17T07:29:17.915 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:17.916 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:17.916 INFO:teuthology.orchestra.run.smithi045.stdout:1898 trunc from 0xaeea665 to 0xc82b6c2 2024-09-17T07:29:17.916 INFO:teuthology.orchestra.run.smithi045.stdout:1900 write 0x7f18d72 thru 0x7f1ee9a (0x6129 bytes) 2024-09-17T07:29:17.916 INFO:teuthology.orchestra.run.smithi045.stdout:1901 writesame 0x4eaa289 thru 0x4eb93b2 data_size 0x1(0xf12a bytes) 2024-09-17T07:29:17.916 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:17.916 INFO:teuthology.orchestra.run.smithi045.stdout:1904 trunc from 0xc82b6c2 to 0x6d96761 2024-09-17T07:29:17.916 INFO:teuthology.orchestra.run.smithi045.stdout:1905 trunc from 0x6d96761 to 0x130adf1 2024-09-17T07:29:17.916 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:17.913+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:17.939 INFO:teuthology.orchestra.run.smithi045.stdout:1907 trunc from 0x130adf1 to 0xa855c9b 2024-09-17T07:29:17.951 INFO:teuthology.orchestra.run.smithi045.stdout:1908 trunc from 0xa855c9b to 0x638046d 2024-09-17T07:29:17.966 INFO:teuthology.orchestra.run.smithi045.stdout:1909 write 0xa63f81e thru 0xa64da7f (0xe262 bytes) 2024-09-17T07:29:17.973 INFO:teuthology.orchestra.run.smithi045.stdout:1910 write 0x804052f thru 0x8042135 (0x1c07 bytes) 2024-09-17T07:29:17.976 INFO:teuthology.orchestra.run.smithi045.stdout:1912 write 0x13f1ebf thru 0x13f514d (0x328f bytes) 2024-09-17T07:29:17.981 INFO:teuthology.orchestra.run.smithi045.stdout:1913 trunc from 0xa64da80 to 0xda21c5b 2024-09-17T07:29:17.986 INFO:teuthology.orchestra.run.smithi045.stdout:1916 write 0x637a5b1 thru 0x6381c31 (0x7681 bytes) 2024-09-17T07:29:17.991 INFO:teuthology.orchestra.run.smithi045.stdout:1917 writesame 0x3c5540e thru 0x3c61a19 data_size 0x4e(0xc60c bytes) 2024-09-17T07:29:18.005 INFO:teuthology.orchestra.run.smithi045.stdout:1918 trunc from 0xda21c5b to 0x52ecb71 2024-09-17T07:29:18.014 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:18.013+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:18.018 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:18.018 INFO:teuthology.orchestra.run.smithi045.stdout:1921 read 0x44cc4d1 thru 0x44cfb5b (0x368b bytes) 2024-09-17T07:29:18.019 INFO:teuthology.orchestra.run.smithi045.stdout:1922 punch from 0x14db861 to 0x14e0057, (0x47f6 bytes) 2024-09-17T07:29:18.031 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:18.031 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:18.034 INFO:teuthology.orchestra.run.smithi045.stdout:1923 write 0xe37f495 thru 0xe389d7f (0xa8eb bytes) 2024-09-17T07:29:18.041 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:18.045 INFO:teuthology.orchestra.run.smithi045.stdout:1924 trunc from 0xe389d80 to 0xa20dd3c 2024-09-17T07:29:18.055 INFO:teuthology.orchestra.run.smithi045.stdout:1926 read 0x4a1f00f thru 0x4a261c5 (0x71b7 bytes) 2024-09-17T07:29:18.056 INFO:teuthology.orchestra.run.smithi045.stdout:1927 write 0x2d4e72a thru 0x2d5dbe3 (0xf4ba bytes) 2024-09-17T07:29:18.066 INFO:teuthology.orchestra.run.smithi045.stdout:1928 trunc from 0xa20dd3c to 0xdbb3b2f 2024-09-17T07:29:18.084 INFO:teuthology.orchestra.run.smithi045.stdout:1929 trunc from 0xdbb3b2f to 0x329aa9b 2024-09-17T07:29:18.111 INFO:teuthology.orchestra.run.smithi045.stdout:1930 write 0xad8f813 thru 0xad9727f (0x7a6d bytes) 2024-09-17T07:29:18.119 INFO:teuthology.orchestra.run.smithi045.stdout:1931 trunc from 0xad97280 to 0xd96002f 2024-09-17T07:29:18.125 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:18.124+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:18.138 INFO:teuthology.orchestra.run.smithi045.stdout:1932 trunc from 0xd96002f to 0x23dffd1 2024-09-17T07:29:18.150 INFO:teuthology.orchestra.run.smithi045.stdout:1935 writesame 0x11241fb thru 0x112d2a1 data_size 0x1(0x90a7 bytes) 2024-09-17T07:29:18.163 INFO:teuthology.orchestra.run.smithi045.stdout:1938 write 0x2ad450d thru 0x2ad67bd (0x22b1 bytes) 2024-09-17T07:29:18.166 INFO:teuthology.orchestra.run.smithi045.stdout:1940 read 0x28b0909 thru 0x28b2768 (0x1e60 bytes) 2024-09-17T07:29:18.167 INFO:teuthology.orchestra.run.smithi045.stdout:1944 read 0xcb70d5 thru 0xcc38c4 (0xc7f0 bytes) 2024-09-17T07:29:18.167 INFO:teuthology.orchestra.run.smithi045.stdout:1945 clone 18 order 25 su 16777216 sc 7 2024-09-17T07:29:18.191 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:18.191 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:18.191 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:18.192 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:18.192 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:18.200 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:18.216 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:18.216+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:18.227 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T07:29:18.283 INFO:teuthology.orchestra.run.smithi045.stdout:flattening image image_client.0-clone17 2024-09-17T07:29:18.317 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:18.316+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:18.338 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-17T07:29:18.350 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:18.350 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:18.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:18.350 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:18.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.350 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:6683 3.9 3:97ddc208:::rbd_data.1183cff49e3a.000000000000001e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:18.350 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.763526+0000", 2024-09-17T07:29:18.350 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.585073366, 2024-09-17T07:29:18.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032433677000000001, 2024-09-17T07:29:18.350 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.350 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.351 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:18.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.351 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6683 2024-09-17T07:29:18.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.351 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.763526+0000", 2024-09-17T07:29:18.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.763526+0000", 2024-09-17T07:29:18.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.763527+0000", 2024-09-17T07:29:18.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.78e-07 2024-09-17T07:29:18.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764048+0000", 2024-09-17T07:29:18.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00052137799999999997 2024-09-17T07:29:18.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764060+0000", 2024-09-17T07:29:18.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2014000000000001e-05 2024-09-17T07:29:18.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764072+0000", 2024-09-17T07:29:18.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1762e-05 2024-09-17T07:29:18.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764130+0000", 2024-09-17T07:29:18.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7759e-05 2024-09-17T07:29:18.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764194+0000", 2024-09-17T07:29:18.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4189999999999994e-05 2024-09-17T07:29:18.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-17T07:29:18.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764332+0000", 2024-09-17T07:29:18.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013821300000000001 2024-09-17T07:29:18.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:18.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795822+0000", 2024-09-17T07:29:18.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.031489847000000001 2024-09-17T07:29:18.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:18.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795858+0000", 2024-09-17T07:29:18.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6452999999999998e-05 2024-09-17T07:29:18.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:18.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795917+0000", 2024-09-17T07:29:18.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8851999999999998e-05 2024-09-17T07:29:18.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795959+0000", 2024-09-17T07:29:18.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2231000000000003e-05 2024-09-17T07:29:18.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.356 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.357 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:6684 3.9 3:97ddc208:::rbd_data.1183cff49e3a.000000000000001e:head [delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:18.357 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.764091+0000", 2024-09-17T07:29:18.357 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.584508183000001, 2024-09-17T07:29:18.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.032300207999999997, 2024-09-17T07:29:18.357 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.357 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.357 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:18.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.357 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6684 2024-09-17T07:29:18.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.357 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764091+0000", 2024-09-17T07:29:18.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764091+0000", 2024-09-17T07:29:18.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764092+0000", 2024-09-17T07:29:18.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8700000000000005e-07 2024-09-17T07:29:18.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764094+0000", 2024-09-17T07:29:18.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9189999999999999e-06 2024-09-17T07:29:18.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764102+0000", 2024-09-17T07:29:18.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2710000000000003e-06 2024-09-17T07:29:18.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764118+0000", 2024-09-17T07:29:18.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6231000000000001e-05 2024-09-17T07:29:18.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.764995+0000", 2024-09-17T07:29:18.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00087753699999999996 2024-09-17T07:29:18.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.765094+0000", 2024-09-17T07:29:18.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8027000000000002e-05 2024-09-17T07:29:18.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-17T07:29:18.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794998+0000", 2024-09-17T07:29:18.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029904612000000001 2024-09-17T07:29:18.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:18.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796267+0000", 2024-09-17T07:29:18.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001268911 2024-09-17T07:29:18.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:18.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796328+0000", 2024-09-17T07:29:18.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0594000000000003e-05 2024-09-17T07:29:18.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:18.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796356+0000", 2024-09-17T07:29:18.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8787999999999999e-05 2024-09-17T07:29:18.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.796391+0000", 2024-09-17T07:29:18.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4631000000000001e-05 2024-09-17T07:29:18.362 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.363 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.363 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.363 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4483.0:6683 3.9 e33/23 ondisk, result = 0)", 2024-09-17T07:29:18.363 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:29:18.363 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.582360387000001, 2024-09-17T07:29:18.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.029728505999999998, 2024-09-17T07:29:18.363 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.363 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.363 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.363 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:29:18.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:29:18.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766239+0000", 2024-09-17T07:29:18.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9400000000000004e-07 2024-09-17T07:29:18.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.766243+0000", 2024-09-17T07:29:18.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9419999999999997e-06 2024-09-17T07:29:18.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794812+0000", 2024-09-17T07:29:18.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.028568324999999999 2024-09-17T07:29:18.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794822+0000", 2024-09-17T07:29:18.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0786e-05 2024-09-17T07:29:18.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795847+0000", 2024-09-17T07:29:18.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010248919999999999 2024-09-17T07:29:18.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795854+0000", 2024-09-17T07:29:18.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4919999999999998e-06 2024-09-17T07:29:18.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795967+0000", 2024-09-17T07:29:18.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000113275 2024-09-17T07:29:18.367 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.367 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.367 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.367 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:6685 3.14 3:2a16c27c:::rbd_object_map.1183cff49e3a.0000000000000007:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:18.367 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.770642+0000", 2024-09-17T07:29:18.367 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.577957485999999, 2024-09-17T07:29:18.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.027608490999999999, 2024-09-17T07:29:18.367 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.367 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.368 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:18.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.368 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6685 2024-09-17T07:29:18.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.368 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770642+0000", 2024-09-17T07:29:18.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770642+0000", 2024-09-17T07:29:18.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770643+0000", 2024-09-17T07:29:18.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.824e-06 2024-09-17T07:29:18.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.770651+0000", 2024-09-17T07:29:18.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9680000000000006e-06 2024-09-17T07:29:18.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794795+0000", 2024-09-17T07:29:18.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024143286 2024-09-17T07:29:18.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794811+0000", 2024-09-17T07:29:18.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6112000000000001e-05 2024-09-17T07:29:18.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.794961+0000", 2024-09-17T07:29:18.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014987 2024-09-17T07:29:18.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795048+0000", 2024-09-17T07:29:18.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7740999999999999e-05 2024-09-17T07:29:18.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:18.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.795317+0000", 2024-09-17T07:29:18.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026868500000000001 2024-09-17T07:29:18.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:18.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.797274+0000", 2024-09-17T07:29:18.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019571660000000002 2024-09-17T07:29:18.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:18.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.798166+0000", 2024-09-17T07:29:18.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00089202400000000005 2024-09-17T07:29:18.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:18.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.798212+0000", 2024-09-17T07:29:18.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.596e-05 2024-09-17T07:29:18.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.798250+0000", 2024-09-17T07:29:18.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7855000000000003e-05 2024-09-17T07:29:18.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.373 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.373 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27151 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:18.373 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405408+0000", 2024-09-17T07:29:18.373 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9431913850000004, 2024-09-17T07:29:18.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023090975, 2024-09-17T07:29:18.374 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.374 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.374 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.374 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27151 2024-09-17T07:29:18.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.374 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405408+0000", 2024-09-17T07:29:18.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405408+0000", 2024-09-17T07:29:18.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405408+0000", 2024-09-17T07:29:18.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2e-07 2024-09-17T07:29:18.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405411+0000", 2024-09-17T07:29:18.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5490000000000001e-06 2024-09-17T07:29:18.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405417+0000", 2024-09-17T07:29:18.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7379999999999997e-06 2024-09-17T07:29:18.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405427+0000", 2024-09-17T07:29:18.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4409999999999999e-06 2024-09-17T07:29:18.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428213+0000", 2024-09-17T07:29:18.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.022786130000000002 2024-09-17T07:29:18.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428286+0000", 2024-09-17T07:29:18.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3261000000000002e-05 2024-09-17T07:29:18.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428499+0000", 2024-09-17T07:29:18.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021233600000000001 2024-09-17T07:29:18.378 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.378 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.378 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.378 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27152 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 106496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:18.378 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405434+0000", 2024-09-17T07:29:18.378 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9431645310000007, 2024-09-17T07:29:18.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023382574, 2024-09-17T07:29:18.378 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.378 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.378 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.379 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.379 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.379 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27152 2024-09-17T07:29:18.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.379 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405434+0000", 2024-09-17T07:29:18.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405434+0000", 2024-09-17T07:29:18.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405435+0000", 2024-09-17T07:29:18.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7899999999999999e-07 2024-09-17T07:29:18.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405437+0000", 2024-09-17T07:29:18.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.108e-06 2024-09-17T07:29:18.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405444+0000", 2024-09-17T07:29:18.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4829999999999996e-06 2024-09-17T07:29:18.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405448+0000", 2024-09-17T07:29:18.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4669999999999998e-06 2024-09-17T07:29:18.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428527+0000", 2024-09-17T07:29:18.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023079183 2024-09-17T07:29:18.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428601+0000", 2024-09-17T07:29:18.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3385000000000003e-05 2024-09-17T07:29:18.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428817+0000", 2024-09-17T07:29:18.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000216469 2024-09-17T07:29:18.382 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.382 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.382 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.383 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27153 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:18.383 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405455+0000", 2024-09-17T07:29:18.383 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9431443829999999, 2024-09-17T07:29:18.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023682372, 2024-09-17T07:29:18.383 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.383 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.383 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.383 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27153 2024-09-17T07:29:18.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.384 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405455+0000", 2024-09-17T07:29:18.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405455+0000", 2024-09-17T07:29:18.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405455+0000", 2024-09-17T07:29:18.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-09-17T07:29:18.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405457+0000", 2024-09-17T07:29:18.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1040000000000001e-06 2024-09-17T07:29:18.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405464+0000", 2024-09-17T07:29:18.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7050000000000002e-06 2024-09-17T07:29:18.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405469+0000", 2024-09-17T07:29:18.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9019999999999998e-06 2024-09-17T07:29:18.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428847+0000", 2024-09-17T07:29:18.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023378304999999999 2024-09-17T07:29:18.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.428920+0000", 2024-09-17T07:29:18.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2559999999999996e-05 2024-09-17T07:29:18.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429137+0000", 2024-09-17T07:29:18.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217386 2024-09-17T07:29:18.387 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.387 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.387 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.387 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27154 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 139264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:18.387 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405475+0000", 2024-09-17T07:29:18.388 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9431239760000008, 2024-09-17T07:29:18.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023995657, 2024-09-17T07:29:18.388 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.388 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.388 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.388 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.388 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.388 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.388 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27154 2024-09-17T07:29:18.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.388 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405475+0000", 2024-09-17T07:29:18.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405475+0000", 2024-09-17T07:29:18.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405475+0000", 2024-09-17T07:29:18.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8000000000000001e-07 2024-09-17T07:29:18.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405477+0000", 2024-09-17T07:29:18.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.883e-06 2024-09-17T07:29:18.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405484+0000", 2024-09-17T07:29:18.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2990000000000002e-06 2024-09-17T07:29:18.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405488+0000", 2024-09-17T07:29:18.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6659999999999999e-06 2024-09-17T07:29:18.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429166+0000", 2024-09-17T07:29:18.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023677404999999999 2024-09-17T07:29:18.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429238+0000", 2024-09-17T07:29:18.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2570999999999993e-05 2024-09-17T07:29:18.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429471+0000", 2024-09-17T07:29:18.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023245300000000001 2024-09-17T07:29:18.391 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.391 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.392 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.392 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27155 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 155648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:18.392 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405494+0000", 2024-09-17T07:29:18.392 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9431047649999993, 2024-09-17T07:29:18.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024292818000000001, 2024-09-17T07:29:18.392 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.392 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.392 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.392 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.392 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.392 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.393 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27155 2024-09-17T07:29:18.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.393 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405494+0000", 2024-09-17T07:29:18.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405494+0000", 2024-09-17T07:29:18.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405495+0000", 2024-09-17T07:29:18.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:18.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405497+0000", 2024-09-17T07:29:18.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8870000000000001e-06 2024-09-17T07:29:18.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405503+0000", 2024-09-17T07:29:18.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4849999999999997e-06 2024-09-17T07:29:18.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405508+0000", 2024-09-17T07:29:18.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1290000000000004e-06 2024-09-17T07:29:18.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429499+0000", 2024-09-17T07:29:18.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023990592000000002 2024-09-17T07:29:18.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429571+0000", 2024-09-17T07:29:18.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2199000000000004e-05 2024-09-17T07:29:18.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429787+0000", 2024-09-17T07:29:18.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000216126 2024-09-17T07:29:18.396 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.396 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.396 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.396 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27156 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:18.396 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405515+0000", 2024-09-17T07:29:18.397 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9430843670000009, 2024-09-17T07:29:18.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024640863999999998, 2024-09-17T07:29:18.397 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.397 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.397 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.397 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.397 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.397 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.397 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27156 2024-09-17T07:29:18.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.397 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405515+0000", 2024-09-17T07:29:18.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405515+0000", 2024-09-17T07:29:18.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405515+0000", 2024-09-17T07:29:18.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0999999999999999e-07 2024-09-17T07:29:18.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405524+0000", 2024-09-17T07:29:18.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6619999999999994e-06 2024-09-17T07:29:18.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405531+0000", 2024-09-17T07:29:18.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8669999999999996e-06 2024-09-17T07:29:18.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405535+0000", 2024-09-17T07:29:18.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6539999999999998e-06 2024-09-17T07:29:18.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429814+0000", 2024-09-17T07:29:18.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024279207000000001 2024-09-17T07:29:18.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.429886+0000", 2024-09-17T07:29:18.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1729000000000004e-05 2024-09-17T07:29:18.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430156+0000", 2024-09-17T07:29:18.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269335 2024-09-17T07:29:18.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.401 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.401 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27158 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:18.401 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405542+0000", 2024-09-17T07:29:18.401 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9430570580000008, 2024-09-17T07:29:18.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024918612999999999, 2024-09-17T07:29:18.401 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.401 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.401 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.401 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.402 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27158 2024-09-17T07:29:18.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.402 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405542+0000", 2024-09-17T07:29:18.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405542+0000", 2024-09-17T07:29:18.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405542+0000", 2024-09-17T07:29:18.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3799999999999998e-07 2024-09-17T07:29:18.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405544+0000", 2024-09-17T07:29:18.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.812e-06 2024-09-17T07:29:18.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405551+0000", 2024-09-17T07:29:18.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5989999999999998e-06 2024-09-17T07:29:18.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405556+0000", 2024-09-17T07:29:18.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8060000000000002e-06 2024-09-17T07:29:18.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430183+0000", 2024-09-17T07:29:18.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024627860000000001 2024-09-17T07:29:18.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430258+0000", 2024-09-17T07:29:18.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4516000000000003e-05 2024-09-17T07:29:18.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430461+0000", 2024-09-17T07:29:18.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020258200000000001 2024-09-17T07:29:18.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.405 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.405 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27160 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:18.406 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405562+0000", 2024-09-17T07:29:18.406 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9430372859999991, 2024-09-17T07:29:18.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025242179999999999, 2024-09-17T07:29:18.406 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.406 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.406 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.406 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27160 2024-09-17T07:29:18.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.406 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405562+0000", 2024-09-17T07:29:18.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405562+0000", 2024-09-17T07:29:18.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405562+0000", 2024-09-17T07:29:18.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6199999999999998e-07 2024-09-17T07:29:18.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405564+0000", 2024-09-17T07:29:18.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.765e-06 2024-09-17T07:29:18.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405570+0000", 2024-09-17T07:29:18.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3439999999999997e-06 2024-09-17T07:29:18.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405575+0000", 2024-09-17T07:29:18.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7639999999999996e-06 2024-09-17T07:29:18.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430505+0000", 2024-09-17T07:29:18.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024930101999999999 2024-09-17T07:29:18.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430584+0000", 2024-09-17T07:29:18.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8479999999999994e-05 2024-09-17T07:29:18.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430804+0000", 2024-09-17T07:29:18.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000220263 2024-09-17T07:29:18.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.410 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.410 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27162 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:18.410 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405581+0000", 2024-09-17T07:29:18.410 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9430177850000003, 2024-09-17T07:29:18.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025527253999999999, 2024-09-17T07:29:18.411 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.411 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.411 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.411 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27162 2024-09-17T07:29:18.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.411 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405581+0000", 2024-09-17T07:29:18.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405581+0000", 2024-09-17T07:29:18.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405582+0000", 2024-09-17T07:29:18.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-07 2024-09-17T07:29:18.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405584+0000", 2024-09-17T07:29:18.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:29:18.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405590+0000", 2024-09-17T07:29:18.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3820000000000001e-06 2024-09-17T07:29:18.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405595+0000", 2024-09-17T07:29:18.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:29:18.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430832+0000", 2024-09-17T07:29:18.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025237654000000002 2024-09-17T07:29:18.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.430905+0000", 2024-09-17T07:29:18.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2234e-05 2024-09-17T07:29:18.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.431108+0000", 2024-09-17T07:29:18.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020389200000000001 2024-09-17T07:29:18.414 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.414 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.415 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.415 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27165 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:18.415 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405601+0000", 2024-09-17T07:29:18.415 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9429982779999992, 2024-09-17T07:29:18.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025836962000000002, 2024-09-17T07:29:18.415 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.415 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.415 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.416 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.416 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27165 2024-09-17T07:29:18.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.416 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405601+0000", 2024-09-17T07:29:18.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405601+0000", 2024-09-17T07:29:18.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405601+0000", 2024-09-17T07:29:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:29:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405603+0000", 2024-09-17T07:29:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.658e-06 2024-09-17T07:29:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405609+0000", 2024-09-17T07:29:18.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.1689999999999997e-06 2024-09-17T07:29:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405614+0000", 2024-09-17T07:29:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8590000000000004e-06 2024-09-17T07:29:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.431136+0000", 2024-09-17T07:29:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025522382999999999 2024-09-17T07:29:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.431207+0000", 2024-09-17T07:29:18.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1076999999999997e-05 2024-09-17T07:29:18.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.431438+0000", 2024-09-17T07:29:18.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023040499999999999 2024-09-17T07:29:18.419 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.419 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.419 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.419 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27167 3.9 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:e [read 253952~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:18.419 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.405620+0000", 2024-09-17T07:29:18.419 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9429790160000007, 2024-09-17T07:29:18.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.026157786999999998, 2024-09-17T07:29:18.420 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.420 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.420 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.420 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.420 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.420 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.420 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27167 2024-09-17T07:29:18.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.420 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405620+0000", 2024-09-17T07:29:18.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405620+0000", 2024-09-17T07:29:18.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405620+0000", 2024-09-17T07:29:18.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:29:18.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405629+0000", 2024-09-17T07:29:18.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0760000000000003e-06 2024-09-17T07:29:18.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405635+0000", 2024-09-17T07:29:18.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5919999999999997e-06 2024-09-17T07:29:18.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.405640+0000", 2024-09-17T07:29:18.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8470000000000003e-06 2024-09-17T07:29:18.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.431468+0000", 2024-09-17T07:29:18.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.025828002999999999 2024-09-17T07:29:18.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.431556+0000", 2024-09-17T07:29:18.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8213000000000004e-05 2024-09-17T07:29:18.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.431778+0000", 2024-09-17T07:29:18.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022160600000000001 2024-09-17T07:29:18.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.424 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.424 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.424 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27628 3.9 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:e [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:18.424 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.412497+0000", 2024-09-17T07:29:18.424 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9361024829999991, 2024-09-17T07:29:18.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023335651999999998, 2024-09-17T07:29:18.424 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.424 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.424 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.425 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.425 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27628 2024-09-17T07:29:18.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.425 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412497+0000", 2024-09-17T07:29:18.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412497+0000", 2024-09-17T07:29:18.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412497+0000", 2024-09-17T07:29:18.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3000000000000001e-07 2024-09-17T07:29:18.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412499+0000", 2024-09-17T07:29:18.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7230000000000001e-06 2024-09-17T07:29:18.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412505+0000", 2024-09-17T07:29:18.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3969999999999999e-06 2024-09-17T07:29:18.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412510+0000", 2024-09-17T07:29:18.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9429999999999999e-06 2024-09-17T07:29:18.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.435538+0000", 2024-09-17T07:29:18.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023028090000000001 2024-09-17T07:29:18.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.435606+0000", 2024-09-17T07:29:18.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8303e-05 2024-09-17T07:29:18.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.435832+0000", 2024-09-17T07:29:18.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022576599999999999 2024-09-17T07:29:18.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.429 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.429 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.429 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27629 3.9 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:e [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:18.429 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.412516+0000", 2024-09-17T07:29:18.429 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9360834530000002, 2024-09-17T07:29:18.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023701032, 2024-09-17T07:29:18.429 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.429 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.429 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.430 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27629 2024-09-17T07:29:18.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.430 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412516+0000", 2024-09-17T07:29:18.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412516+0000", 2024-09-17T07:29:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412516+0000", 2024-09-17T07:29:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9400000000000001e-07 2024-09-17T07:29:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412518+0000", 2024-09-17T07:29:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6589999999999999e-06 2024-09-17T07:29:18.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412524+0000", 2024-09-17T07:29:18.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3960000000000003e-06 2024-09-17T07:29:18.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412529+0000", 2024-09-17T07:29:18.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9609999999999996e-06 2024-09-17T07:29:18.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.435858+0000", 2024-09-17T07:29:18.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.02332934 2024-09-17T07:29:18.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.435929+0000", 2024-09-17T07:29:18.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0838000000000002e-05 2024-09-17T07:29:18.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.436217+0000", 2024-09-17T07:29:18.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028744399999999998 2024-09-17T07:29:18.433 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.433 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.434 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.434 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27630 3.9 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:e [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:18.434 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.412535+0000", 2024-09-17T07:29:18.434 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9360639539999998, 2024-09-17T07:29:18.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024043742999999999, 2024-09-17T07:29:18.434 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.434 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.434 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.434 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27630 2024-09-17T07:29:18.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.435 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412535+0000", 2024-09-17T07:29:18.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412535+0000", 2024-09-17T07:29:18.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412536+0000", 2024-09-17T07:29:18.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3300000000000003e-07 2024-09-17T07:29:18.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412537+0000", 2024-09-17T07:29:18.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.877e-06 2024-09-17T07:29:18.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412544+0000", 2024-09-17T07:29:18.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3110000000000002e-06 2024-09-17T07:29:18.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412551+0000", 2024-09-17T07:29:18.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4420000000000001e-06 2024-09-17T07:29:18.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.436243+0000", 2024-09-17T07:29:18.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.023692273 2024-09-17T07:29:18.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.436313+0000", 2024-09-17T07:29:18.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9388999999999996e-05 2024-09-17T07:29:18.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.436579+0000", 2024-09-17T07:29:18.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026601800000000003 2024-09-17T07:29:18.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.438 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.439 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.439 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:27631 3.9 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:e [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:18.439 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:08.412557+0000", 2024-09-17T07:29:18.439 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.9360415500000006, 2024-09-17T07:29:18.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.024539926, 2024-09-17T07:29:18.439 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.439 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.439 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.440 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 27631 2024-09-17T07:29:18.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.440 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412557+0000", 2024-09-17T07:29:18.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412557+0000", 2024-09-17T07:29:18.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.412558+0000", 2024-09-17T07:29:18.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4300000000000003e-07 2024-09-17T07:29:18.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.418230+0000", 2024-09-17T07:29:18.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0056723199999999998 2024-09-17T07:29:18.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.419206+0000", 2024-09-17T07:29:18.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00097553099999999997 2024-09-17T07:29:18.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.419213+0000", 2024-09-17T07:29:18.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:29:18.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.436606+0000", 2024-09-17T07:29:18.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.017392364 2024-09-17T07:29:18.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.436675+0000", 2024-09-17T07:29:18.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9086999999999998e-05 2024-09-17T07:29:18.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:08.437097+0000", 2024-09-17T07:29:18.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00042249799999999999 2024-09-17T07:29:18.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.443 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.443 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:30525 3.1b 3:def7c8f0:::rbd_data.118381ffbbab.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T07:29:18.443 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:18.444 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.8263003180000004, 2024-09-17T07:29:18.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55515850300000003, 2024-09-17T07:29:18.444 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.444 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.444 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:18.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.444 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30525 2024-09-17T07:29:18.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.444 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:18.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:18.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522300+0000", 2024-09-17T07:29:18.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:29:18.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522305+0000", 2024-09-17T07:29:18.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-17T07:29:18.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522318+0000", 2024-09-17T07:29:18.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3030999999999999e-05 2024-09-17T07:29:18.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522326+0000", 2024-09-17T07:29:18.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4200000000000001e-06 2024-09-17T07:29:18.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522385+0000", 2024-09-17T07:29:18.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9005000000000002e-05 2024-09-17T07:29:18.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:29:18.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522393+0000", 2024-09-17T07:29:18.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-06 2024-09-17T07:29:18.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075347+0000", 2024-09-17T07:29:18.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55295417800000002 2024-09-17T07:29:18.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075502+0000", 2024-09-17T07:29:18.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154272 2024-09-17T07:29:18.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:18.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075630+0000", 2024-09-17T07:29:18.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012816300000000001 2024-09-17T07:29:18.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:18.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077221+0000", 2024-09-17T07:29:18.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015912929999999999 2024-09-17T07:29:18.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:18.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077353+0000", 2024-09-17T07:29:18.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013215699999999999 2024-09-17T07:29:18.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:18.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077413+0000", 2024-09-17T07:29:18.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0034999999999997e-05 2024-09-17T07:29:18.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077457+0000", 2024-09-17T07:29:18.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4017999999999998e-05 2024-09-17T07:29:18.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.450 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.450 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.450 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:18.452 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:18.452 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:18.450+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:18.532 INFO:teuthology.orchestra.run.smithi045.stdout:1946 read 0xc47c32 thru 0xc5078e (0x8b5d bytes) 2024-09-17T07:29:18.551 INFO:teuthology.orchestra.run.smithi045.stdout:1947 write 0xbcf03c6 thru 0xbcf4d28 (0x4963 bytes) 2024-09-17T07:29:18.552 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:18.551+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:18.553 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:18.554 INFO:teuthology.orchestra.run.smithi045.stdout:1951 write 0x5f119ae thru 0x5f17838 (0x5e8b bytes) 2024-09-17T07:29:18.556 INFO:teuthology.orchestra.run.smithi045.stdout:1953 read 0x33ba94f thru 0x33c4bea (0xa29c bytes) 2024-09-17T07:29:18.556 INFO:teuthology.orchestra.run.smithi045.stdout:1954 read 0xa8ebdfe thru 0xa8f202a (0x622d bytes) 2024-09-17T07:29:18.556 INFO:teuthology.orchestra.run.smithi045.stdout:1955 writesame 0xabf243b thru 0xac021ac data_size 0x1(0xfd72 bytes) 2024-09-17T07:29:18.588 INFO:teuthology.orchestra.run.smithi045.stdout:1956 read 0x7344535 thru 0x7346de5 (0x28b1 bytes) 2024-09-17T07:29:18.588 INFO:teuthology.orchestra.run.smithi045.stdout:1957 clone 19 order 19 su 4096 sc 4 2024-09-17T07:29:18.600 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:18.600 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:18.600 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:18.609 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:18.653 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:18.652+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:18.712 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:18.712 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:18.712 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:18.712 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:18.712 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:18.721 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:18.753 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:18.753+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19985 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5275648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.571951993999999, 2024-09-17T07:29:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071819998999999995, 2024-09-17T07:29:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19985 2024-09-17T07:29:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252493+0000", 2024-09-17T07:29:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6300000000000004e-07 2024-09-17T07:29:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252496+0000", 2024-09-17T07:29:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-17T07:29:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252499+0000", 2024-09-17T07:29:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:29:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324036+0000", 2024-09-17T07:29:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071536587999999998 2024-09-17T07:29:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324118+0000", 2024-09-17T07:29:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2336999999999995e-05 2024-09-17T07:29:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324312+0000", 2024-09-17T07:29:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000193709 2024-09-17T07:29:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19986 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5505024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.571940842, 2024-09-17T07:29:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072083761999999996, 2024-09-17T07:29:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19986 2024-09-17T07:29:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:29:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252508+0000", 2024-09-17T07:29:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:29:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252511+0000", 2024-09-17T07:29:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.145e-06 2024-09-17T07:29:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252514+0000", 2024-09-17T07:29:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5569999999999998e-06 2024-09-17T07:29:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324326+0000", 2024-09-17T07:29:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071812681000000003 2024-09-17T07:29:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324372+0000", 2024-09-17T07:29:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5893000000000003e-05 2024-09-17T07:29:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324587+0000", 2024-09-17T07:29:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214608 2024-09-17T07:29:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19987 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5734400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.571927224, 2024-09-17T07:29:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072276676999999998, 2024-09-17T07:29:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19987 2024-09-17T07:29:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252518+0000", 2024-09-17T07:29:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-17T07:29:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252522+0000", 2024-09-17T07:29:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2440000000000004e-06 2024-09-17T07:29:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252525+0000", 2024-09-17T07:29:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5919999999999999e-06 2024-09-17T07:29:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324621+0000", 2024-09-17T07:29:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072096442999999996 2024-09-17T07:29:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324699+0000", 2024-09-17T07:29:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8061999999999999e-05 2024-09-17T07:29:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324793+0000", 2024-09-17T07:29:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4207000000000001e-05 2024-09-17T07:29:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19988 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.571916252999999, 2024-09-17T07:29:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072585798000000007, 2024-09-17T07:29:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19988 2024-09-17T07:29:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252529+0000", 2024-09-17T07:29:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:29:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252532+0000", 2024-09-17T07:29:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9579999999999999e-06 2024-09-17T07:29:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252534+0000", 2024-09-17T07:29:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5730000000000002e-06 2024-09-17T07:29:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324825+0000", 2024-09-17T07:29:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072290757999999997 2024-09-17T07:29:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324874+0000", 2024-09-17T07:29:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9360000000000002e-05 2024-09-17T07:29:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325113+0000", 2024-09-17T07:29:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000239025 2024-09-17T07:29:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19989 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.571906631999999, 2024-09-17T07:29:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072963494000000004, 2024-09-17T07:29:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19989 2024-09-17T07:29:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:29:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:29:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:29:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2900000000000002e-07 2024-09-17T07:29:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252542+0000", 2024-09-17T07:29:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8600000000000003e-06 2024-09-17T07:29:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252545+0000", 2024-09-17T07:29:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.605e-06 2024-09-17T07:29:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325144+0000", 2024-09-17T07:29:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072599460000000005 2024-09-17T07:29:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325223+0000", 2024-09-17T07:29:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8473999999999994e-05 2024-09-17T07:29:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325501+0000", 2024-09-17T07:29:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000278063 2024-09-17T07:29:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19990 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6422528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.571896161, 2024-09-17T07:29:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073241127000000003, 2024-09-17T07:29:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19990 2024-09-17T07:29:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252552+0000", 2024-09-17T07:29:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4719999999999997e-06 2024-09-17T07:29:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252557+0000", 2024-09-17T07:29:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3039999999999998e-06 2024-09-17T07:29:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252560+0000", 2024-09-17T07:29:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8830000000000002e-06 2024-09-17T07:29:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325529+0000", 2024-09-17T07:29:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072968880999999999 2024-09-17T07:29:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325574+0000", 2024-09-17T07:29:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466000000000001e-05 2024-09-17T07:29:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325789+0000", 2024-09-17T07:29:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021490999999999999 2024-09-17T07:29:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19991 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6651904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.571876859, 2024-09-17T07:29:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073720552999999994, 2024-09-17T07:29:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19991 2024-09-17T07:29:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4699999999999998e-07 2024-09-17T07:29:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252568+0000", 2024-09-17T07:29:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5300000000000002e-07 2024-09-17T07:29:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252572+0000", 2024-09-17T07:29:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9160000000000003e-06 2024-09-17T07:29:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252575+0000", 2024-09-17T07:29:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:29:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325820+0000", 2024-09-17T07:29:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073245597999999995 2024-09-17T07:29:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325901+0000", 2024-09-17T07:29:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0913000000000003e-05 2024-09-17T07:29:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326288+0000", 2024-09-17T07:29:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038646900000000002 2024-09-17T07:29:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19992 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6881280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.571866505999999, 2024-09-17T07:29:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074054683999999996, 2024-09-17T07:29:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19992 2024-09-17T07:29:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252578+0000", 2024-09-17T07:29:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:29:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252579+0000", 2024-09-17T07:29:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2500000000000004e-07 2024-09-17T07:29:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252582+0000", 2024-09-17T07:29:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-17T07:29:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252585+0000", 2024-09-17T07:29:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.542e-06 2024-09-17T07:29:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326315+0000", 2024-09-17T07:29:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073729921000000004 2024-09-17T07:29:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326359+0000", 2024-09-17T07:29:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4095999999999999e-05 2024-09-17T07:29:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326632+0000", 2024-09-17T07:29:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027314499999999997 2024-09-17T07:29:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19993 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7110656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.571856201999999, 2024-09-17T07:29:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074425142999999999, 2024-09-17T07:29:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19993 2024-09-17T07:29:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:29:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252589+0000", 2024-09-17T07:29:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.33e-07 2024-09-17T07:29:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252593+0000", 2024-09-17T07:29:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:29:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252595+0000", 2024-09-17T07:29:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.621e-06 2024-09-17T07:29:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326666+0000", 2024-09-17T07:29:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074070787999999999 2024-09-17T07:29:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326752+0000", 2024-09-17T07:29:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5748000000000006e-05 2024-09-17T07:29:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327013+0000", 2024-09-17T07:29:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026098400000000002 2024-09-17T07:29:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19994 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7340032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.571845743000001, 2024-09-17T07:29:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074765920999999999, 2024-09-17T07:29:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19994 2024-09-17T07:29:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:29:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252599+0000", 2024-09-17T07:29:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2099999999999995e-07 2024-09-17T07:29:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252603+0000", 2024-09-17T07:29:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:29:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252606+0000", 2024-09-17T07:29:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-09-17T07:29:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327047+0000", 2024-09-17T07:29:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074441050999999994 2024-09-17T07:29:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327121+0000", 2024-09-17T07:29:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4283999999999996e-05 2024-09-17T07:29:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327364+0000", 2024-09-17T07:29:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000243155 2024-09-17T07:29:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19995 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7569408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.571835046, 2024-09-17T07:29:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075172694999999998, 2024-09-17T07:29:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19995 2024-09-17T07:29:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252610+0000", 2024-09-17T07:29:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6799999999999999e-07 2024-09-17T07:29:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252614+0000", 2024-09-17T07:29:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000002e-06 2024-09-17T07:29:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252616+0000", 2024-09-17T07:29:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.419e-06 2024-09-17T07:29:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327407+0000", 2024-09-17T07:29:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074790617000000004 2024-09-17T07:29:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327498+0000", 2024-09-17T07:29:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0758000000000001e-05 2024-09-17T07:29:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327782+0000", 2024-09-17T07:29:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028395599999999998 2024-09-17T07:29:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7798784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.571824729999999, 2024-09-17T07:29:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075605832999999997, 2024-09-17T07:29:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:29:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:29:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:29:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:29:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:29:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:29:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:29:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327844+0000", 2024-09-17T07:29:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075217709999999993 2024-09-17T07:29:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327927+0000", 2024-09-17T07:29:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2628999999999998e-05 2024-09-17T07:29:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328225+0000", 2024-09-17T07:29:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029800600000000002 2024-09-17T07:29:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8028160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.571814377000001, 2024-09-17T07:29:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076061237000000004, 2024-09-17T07:29:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:29:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:29:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:29:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:29:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:29:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:29:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:29:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328268+0000", 2024-09-17T07:29:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075626323999999995 2024-09-17T07:29:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328352+0000", 2024-09-17T07:29:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3578999999999994e-05 2024-09-17T07:29:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328691+0000", 2024-09-17T07:29:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033915099999999999 2024-09-17T07:29:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8257536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.571798926, 2024-09-17T07:29:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076441903000000005, 2024-09-17T07:29:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:29:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:29:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:29:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:29:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:29:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:29:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:29:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:29:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328747+0000", 2024-09-17T07:29:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076094192000000005 2024-09-17T07:29:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328824+0000", 2024-09-17T07:29:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7096999999999997e-05 2024-09-17T07:29:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.329087+0000", 2024-09-17T07:29:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026249999999999998 2024-09-17T07:29:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20029 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.561713938, 2024-09-17T07:29:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071783696999999994, 2024-09-17T07:29:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20029 2024-09-17T07:29:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262735+0000", 2024-09-17T07:29:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.955e-06 2024-09-17T07:29:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262737+0000", 2024-09-17T07:29:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0090000000000002e-06 2024-09-17T07:29:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262743+0000", 2024-09-17T07:29:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2890000000000003e-06 2024-09-17T07:29:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262748+0000", 2024-09-17T07:29:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334163+0000", 2024-09-17T07:29:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071415336999999995 2024-09-17T07:29:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334243+0000", 2024-09-17T07:29:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9548000000000005e-05 2024-09-17T07:29:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334514+0000", 2024-09-17T07:29:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027086800000000001 2024-09-17T07:29:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7356416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.561682767000001, 2024-09-17T07:29:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072172616999999994, 2024-09-17T07:29:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:29:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:29:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:29:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:29:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:29:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:29:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:29:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:29:18.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:29:18.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334558+0000", 2024-09-17T07:29:18.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071780790999999997 2024-09-17T07:29:18.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334647+0000", 2024-09-17T07:29:18.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9448000000000001e-05 2024-09-17T07:29:18.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334934+0000", 2024-09-17T07:29:18.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028654599999999999 2024-09-17T07:29:18.899 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.899 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.899 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.899 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7585792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.899 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:18.899 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.561661055, 2024-09-17T07:29:18.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072606135000000002, 2024-09-17T07:29:18.899 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.900 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.900 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.900 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.900 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.900 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.900 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:29:18.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.900 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:18.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:18.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:18.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:29:18.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:29:18.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:29:18.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:29:18.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:29:18.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:29:18.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:29:18.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335116+0000", 2024-09-17T07:29:18.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072323897999999998 2024-09-17T07:29:18.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335166+0000", 2024-09-17T07:29:18.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9882000000000001e-05 2024-09-17T07:29:18.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335389+0000", 2024-09-17T07:29:18.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022334300000000001 2024-09-17T07:29:18.904 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.904 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.904 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.904 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7815168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.904 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:18.904 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.561648856, 2024-09-17T07:29:18.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073051230999999994, 2024-09-17T07:29:18.904 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.904 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.904 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.904 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.905 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:29:18.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.905 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:18.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:18.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:18.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:18.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:29:18.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:29:18.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:29:18.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:29:18.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:29:18.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:29:18.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335431+0000", 2024-09-17T07:29:18.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072627852000000007 2024-09-17T07:29:18.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335525+0000", 2024-09-17T07:29:18.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3708000000000005e-05 2024-09-17T07:29:18.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335846+0000", 2024-09-17T07:29:18.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032182000000000001 2024-09-17T07:29:18.908 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.908 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.908 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.908 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.908 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:18.909 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.561637998, 2024-09-17T07:29:18.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073284199999999994, 2024-09-17T07:29:18.909 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.909 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.909 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.909 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:29:18.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.909 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:29:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:29:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:29:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:29:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:29:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:29:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:29:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335863+0000", 2024-09-17T07:29:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073048921000000003 2024-09-17T07:29:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335904+0000", 2024-09-17T07:29:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1545000000000003e-05 2024-09-17T07:29:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336090+0000", 2024-09-17T07:29:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185787 2024-09-17T07:29:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.913 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.913 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.913 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8273920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:18.913 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:18.913 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.561623738, 2024-09-17T07:29:18.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073670478999999997, 2024-09-17T07:29:18.913 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:18.913 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:18.913 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:18.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:18.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:18.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:18.913 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:29:18.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.914 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:18.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:18.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:18.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:18.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:18.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:18.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:18.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:18.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:29:18.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:18.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:29:18.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:29:18.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:18.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:29:18.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:29:18.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:29:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:29:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336126+0000", 2024-09-17T07:29:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073297254000000006 2024-09-17T07:29:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336211+0000", 2024-09-17T07:29:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4728000000000006e-05 2024-09-17T07:29:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336491+0000", 2024-09-17T07:29:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027979399999999998 2024-09-17T07:29:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:18.919 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:18.919 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:18.918+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:19.005 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:29:19.022 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:19.023 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:19.031 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:19.049 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:19.049+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:19.121 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:19.120+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:19.134 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:19.134 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:19.134 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:19.136 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:19.136 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:19.143 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:19.153 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:19.153 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:19.169 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:19.169 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:19.169 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:19.186 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:19.186 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:19.203 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:19.203 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:19.220 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:19.220 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:19.221 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:19.220+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:19.227 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-17T07:29:19.247 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:19.247 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:19.250 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:19.250 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:19.251 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:19.251 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:19.260 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:19.280 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone18 from 0xbcf4d29 (overlap 0x2ad67be) to 0x156b3df 2024-09-17T07:29:19.321 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:19.321+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:19.354 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-17T07:29:19.376 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:19.376 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:19.385 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:19.410 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:19.419 INFO:teuthology.orchestra.run.smithi045.stdout:1959 trunc from 0xbcf4d29 to 0x36de152 2024-09-17T07:29:19.428 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:19.427+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:19.490 INFO:teuthology.orchestra.run.smithi045.stdout:1962 writesame 0x274fd32 thru 0x2751105 data_size 0x1b(0x13d4 bytes) 2024-09-17T07:29:19.505 INFO:teuthology.orchestra.run.smithi045.stdout:1963 writesame 0x1664264 thru 0x166d695 data_size 0x3(0x9432 bytes) 2024-09-17T07:29:19.523 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:19.524 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:19.524 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:19.522+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:19.536 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:19.580 INFO:teuthology.orchestra.run.smithi045.stdout:1965 write 0xea0a0e6 thru 0xea0dfdf (0x3efa bytes) 2024-09-17T07:29:19.586 INFO:teuthology.orchestra.run.smithi045.stdout:1966 write 0xd795931 thru 0xd796dff (0x14cf bytes) 2024-09-17T07:29:19.589 INFO:teuthology.orchestra.run.smithi045.stdout:1967 writesame 0x91e3e24 thru 0x91ef9cb data_size 0x14(0xbba8 bytes) 2024-09-17T07:29:19.594 INFO:teuthology.orchestra.run.smithi045.stdout:1968 write 0x4266005 thru 0x426952d (0x3529 bytes) 2024-09-17T07:29:19.599 INFO:teuthology.orchestra.run.smithi045.stdout:1969 writesame 0x958e461 thru 0x9591867 data_size 0x1(0x3407 bytes) 2024-09-17T07:29:19.605 INFO:teuthology.orchestra.run.smithi045.stdout:1970 read 0x76b29f8 thru 0x76b3cf2 (0x12fb bytes) 2024-09-17T07:29:19.606 INFO:teuthology.orchestra.run.smithi045.stdout:1971 punch from 0x44dbd4c to 0x44dc8f6, (0xbaa bytes) 2024-09-17T07:29:19.606 INFO:teuthology.orchestra.run.smithi045.stdout:1972 clone 20 order 25 su 8388608 sc 8 2024-09-17T07:29:19.625 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:19.623+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:19.682 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:19.682 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:19.690 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:19.731 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:19.730+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:19.798 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:19.798 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:19.807 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:19.828 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:19.827+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:19.913 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:19.913 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:19.922 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:19.965 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:19.965+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:20.029 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:20.030 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:20.030 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:20.029+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:20.038 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:20.130 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:20.130+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:20.186 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:20.186 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:20.195 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:20.235 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:20.235+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:20.296 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone19 from 0xea0dfe0 (overlap 0x36de152) to 0x0 2024-09-17T07:29:20.332 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:20.331+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:20.355 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:20.355 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:20.365 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:20.433 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:20.432+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:20.498 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:20.499 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:20.509 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:20.533 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:20.532+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:20.635 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:20.633+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:20.645 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:20.646 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:20.655 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:20.736 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:20.735+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:20.787 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:20.788 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:20.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:20.837 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:20.835+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:20.915 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:20.916 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:20.925 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:20.937 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:20.937+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:21.030 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:21.031 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:21.038 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:21.037+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:21.039 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:21.139 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:21.137+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:21.166 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:21.167 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:21.177 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:21.246 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T07:29:21.252 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:29:21.262 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:21.261+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:21.340 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:21.339+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:21.364 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:21.364 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:21.366 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-17T07:29:21.366 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:29:21.366 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-17T07:29:21.366 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-17T07:29:21.377 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:21.448 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:21.448 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:21.461 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:21.460+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:21.468 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:21.468 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:21.490 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:21.491 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:21.491 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:21.509 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:21.510 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:21.532 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:21.532 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:21.541 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:21.541+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:21.551 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:21.551 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:21.553 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:21.553 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:21.553 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:21.553 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:21.554 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:bench [] [] [] [] 2024-09-17T07:29:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs debug_inject_read_zeros 2024-09-17T07:29:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs files list 2024-09-17T07:29:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs stats 2024-09-17T07:29:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:bluestore allocator dump block 2024-09-17T07:29:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: invalid command 2024-09-17T07:29:21.558 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:21.558 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T07:29:21.559 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-17T07:29:21.565 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:21.571 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:21.571 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:21.574 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:21.575 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:21.642 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:21.642+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:21.731 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:21.731 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:21.731 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:bench [] [] [] [] 2024-09-17T07:29:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs debug_inject_read_zeros 2024-09-17T07:29:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs files list 2024-09-17T07:29:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs stats 2024-09-17T07:29:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:bluestore allocator dump block 2024-09-17T07:29:21.732 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: invalid command 2024-09-17T07:29:21.734 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:21.735 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T07:29:21.735 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-17T07:29:21.744 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:21.744 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:21.753 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:21.776 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:21.776+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:21.844 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:21.844+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:21.887 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:21.887 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:21.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:21.887 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:21.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.887 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19985 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5275648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.888 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:21.888 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.633373857999999, 2024-09-17T07:29:21.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071819998999999995, 2024-09-17T07:29:21.888 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.888 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.888 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.888 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.888 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.888 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.888 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19985 2024-09-17T07:29:21.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.888 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:21.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:21.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:21.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:21.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252493+0000", 2024-09-17T07:29:21.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6300000000000004e-07 2024-09-17T07:29:21.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252496+0000", 2024-09-17T07:29:21.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-17T07:29:21.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252499+0000", 2024-09-17T07:29:21.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:29:21.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324036+0000", 2024-09-17T07:29:21.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071536587999999998 2024-09-17T07:29:21.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324118+0000", 2024-09-17T07:29:21.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2336999999999995e-05 2024-09-17T07:29:21.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324312+0000", 2024-09-17T07:29:21.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000193709 2024-09-17T07:29:21.892 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.892 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.892 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.892 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19986 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5505024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.892 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:21.892 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.633362706, 2024-09-17T07:29:21.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072083761999999996, 2024-09-17T07:29:21.892 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.892 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.892 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.892 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.892 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.893 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.893 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19986 2024-09-17T07:29:21.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.893 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:21.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252508+0000", 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252511+0000", 2024-09-17T07:29:21.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.145e-06 2024-09-17T07:29:21.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252514+0000", 2024-09-17T07:29:21.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5569999999999998e-06 2024-09-17T07:29:21.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324326+0000", 2024-09-17T07:29:21.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071812681000000003 2024-09-17T07:29:21.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324372+0000", 2024-09-17T07:29:21.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5893000000000003e-05 2024-09-17T07:29:21.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324587+0000", 2024-09-17T07:29:21.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214608 2024-09-17T07:29:21.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.896 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.897 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19987 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5734400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.897 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:21.897 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.633349087999999, 2024-09-17T07:29:21.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072276676999999998, 2024-09-17T07:29:21.897 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.897 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.897 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.897 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19987 2024-09-17T07:29:21.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.897 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:21.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:21.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:21.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:21.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252518+0000", 2024-09-17T07:29:21.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-17T07:29:21.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252522+0000", 2024-09-17T07:29:21.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2440000000000004e-06 2024-09-17T07:29:21.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252525+0000", 2024-09-17T07:29:21.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5919999999999999e-06 2024-09-17T07:29:21.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324621+0000", 2024-09-17T07:29:21.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072096442999999996 2024-09-17T07:29:21.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324699+0000", 2024-09-17T07:29:21.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8061999999999999e-05 2024-09-17T07:29:21.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324793+0000", 2024-09-17T07:29:21.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4207000000000001e-05 2024-09-17T07:29:21.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.901 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.901 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19988 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.901 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:21.901 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.633338117000001, 2024-09-17T07:29:21.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072585798000000007, 2024-09-17T07:29:21.901 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.901 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19988 2024-09-17T07:29:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252529+0000", 2024-09-17T07:29:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:29:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252532+0000", 2024-09-17T07:29:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9579999999999999e-06 2024-09-17T07:29:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252534+0000", 2024-09-17T07:29:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5730000000000002e-06 2024-09-17T07:29:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324825+0000", 2024-09-17T07:29:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072290757999999997 2024-09-17T07:29:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324874+0000", 2024-09-17T07:29:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9360000000000002e-05 2024-09-17T07:29:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325113+0000", 2024-09-17T07:29:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000239025 2024-09-17T07:29:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.906 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19989 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.906 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:21.906 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.633328496000001, 2024-09-17T07:29:21.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072963494000000004, 2024-09-17T07:29:21.906 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.906 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.906 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.906 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.906 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19989 2024-09-17T07:29:21.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.906 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:21.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:21.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:29:21.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:29:21.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:29:21.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2900000000000002e-07 2024-09-17T07:29:21.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252542+0000", 2024-09-17T07:29:21.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8600000000000003e-06 2024-09-17T07:29:21.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252545+0000", 2024-09-17T07:29:21.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.605e-06 2024-09-17T07:29:21.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325144+0000", 2024-09-17T07:29:21.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072599460000000005 2024-09-17T07:29:21.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325223+0000", 2024-09-17T07:29:21.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8473999999999994e-05 2024-09-17T07:29:21.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325501+0000", 2024-09-17T07:29:21.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000278063 2024-09-17T07:29:21.910 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.910 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.910 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.910 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19990 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6422528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.910 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:21.910 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.633318025000001, 2024-09-17T07:29:21.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073241127000000003, 2024-09-17T07:29:21.910 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.910 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.910 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.911 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.911 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19990 2024-09-17T07:29:21.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.911 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:21.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:21.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:21.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:21.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252552+0000", 2024-09-17T07:29:21.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4719999999999997e-06 2024-09-17T07:29:21.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252557+0000", 2024-09-17T07:29:21.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3039999999999998e-06 2024-09-17T07:29:21.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252560+0000", 2024-09-17T07:29:21.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8830000000000002e-06 2024-09-17T07:29:21.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325529+0000", 2024-09-17T07:29:21.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072968880999999999 2024-09-17T07:29:21.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325574+0000", 2024-09-17T07:29:21.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466000000000001e-05 2024-09-17T07:29:21.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325789+0000", 2024-09-17T07:29:21.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021490999999999999 2024-09-17T07:29:21.915 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.915 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.915 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.915 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19991 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6651904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.915 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:21.915 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.633298722999999, 2024-09-17T07:29:21.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073720552999999994, 2024-09-17T07:29:21.916 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.916 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.916 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.916 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.916 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.916 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.916 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19991 2024-09-17T07:29:21.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.916 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:21.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:21.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:21.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4699999999999998e-07 2024-09-17T07:29:21.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252568+0000", 2024-09-17T07:29:21.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5300000000000002e-07 2024-09-17T07:29:21.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252572+0000", 2024-09-17T07:29:21.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9160000000000003e-06 2024-09-17T07:29:21.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252575+0000", 2024-09-17T07:29:21.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:29:21.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325820+0000", 2024-09-17T07:29:21.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073245597999999995 2024-09-17T07:29:21.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325901+0000", 2024-09-17T07:29:21.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0913000000000003e-05 2024-09-17T07:29:21.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326288+0000", 2024-09-17T07:29:21.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038646900000000002 2024-09-17T07:29:21.919 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.919 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.919 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.920 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19992 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6881280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.920 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:21.920 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.633288369999999, 2024-09-17T07:29:21.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074054683999999996, 2024-09-17T07:29:21.920 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.920 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.920 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.920 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.920 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.920 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.920 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19992 2024-09-17T07:29:21.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.920 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:21.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:21.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252578+0000", 2024-09-17T07:29:21.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:29:21.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252579+0000", 2024-09-17T07:29:21.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2500000000000004e-07 2024-09-17T07:29:21.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252582+0000", 2024-09-17T07:29:21.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-17T07:29:21.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252585+0000", 2024-09-17T07:29:21.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.542e-06 2024-09-17T07:29:21.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326315+0000", 2024-09-17T07:29:21.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073729921000000004 2024-09-17T07:29:21.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326359+0000", 2024-09-17T07:29:21.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4095999999999999e-05 2024-09-17T07:29:21.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326632+0000", 2024-09-17T07:29:21.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027314499999999997 2024-09-17T07:29:21.924 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.924 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.924 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.924 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19993 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7110656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.924 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:21.924 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.633278065999999, 2024-09-17T07:29:21.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074425142999999999, 2024-09-17T07:29:21.924 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.924 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.925 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.925 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19993 2024-09-17T07:29:21.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.925 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:21.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:21.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:21.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:29:21.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252589+0000", 2024-09-17T07:29:21.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.33e-07 2024-09-17T07:29:21.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252593+0000", 2024-09-17T07:29:21.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:29:21.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252595+0000", 2024-09-17T07:29:21.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.621e-06 2024-09-17T07:29:21.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326666+0000", 2024-09-17T07:29:21.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074070787999999999 2024-09-17T07:29:21.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326752+0000", 2024-09-17T07:29:21.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5748000000000006e-05 2024-09-17T07:29:21.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327013+0000", 2024-09-17T07:29:21.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026098400000000002 2024-09-17T07:29:21.928 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.928 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.928 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.929 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19994 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7340032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.929 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:21.929 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.633267607000001, 2024-09-17T07:29:21.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074765920999999999, 2024-09-17T07:29:21.929 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.929 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.929 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.929 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19994 2024-09-17T07:29:21.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.929 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:21.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:21.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:21.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:29:21.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252599+0000", 2024-09-17T07:29:21.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2099999999999995e-07 2024-09-17T07:29:21.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252603+0000", 2024-09-17T07:29:21.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:29:21.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252606+0000", 2024-09-17T07:29:21.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-09-17T07:29:21.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327047+0000", 2024-09-17T07:29:21.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074441050999999994 2024-09-17T07:29:21.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327121+0000", 2024-09-17T07:29:21.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4283999999999996e-05 2024-09-17T07:29:21.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327364+0000", 2024-09-17T07:29:21.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000243155 2024-09-17T07:29:21.933 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.933 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.933 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.933 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19995 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7569408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.933 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:21.933 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.63325691, 2024-09-17T07:29:21.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075172694999999998, 2024-09-17T07:29:21.933 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.933 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.933 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.934 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.934 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.934 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.934 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19995 2024-09-17T07:29:21.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.934 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:21.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:21.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:21.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:21.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252610+0000", 2024-09-17T07:29:21.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6799999999999999e-07 2024-09-17T07:29:21.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252614+0000", 2024-09-17T07:29:21.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000002e-06 2024-09-17T07:29:21.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252616+0000", 2024-09-17T07:29:21.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.419e-06 2024-09-17T07:29:21.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327407+0000", 2024-09-17T07:29:21.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074790617000000004 2024-09-17T07:29:21.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327498+0000", 2024-09-17T07:29:21.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0758000000000001e-05 2024-09-17T07:29:21.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327782+0000", 2024-09-17T07:29:21.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028395599999999998 2024-09-17T07:29:21.937 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.937 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.937 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.937 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7798784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.938 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:21.938 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.633246593999999, 2024-09-17T07:29:21.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075605832999999997, 2024-09-17T07:29:21.938 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.938 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.938 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.938 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.938 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.938 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.938 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:29:21.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.938 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:21.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:21.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:21.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:21.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:29:21.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:29:21.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:29:21.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:29:21.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:29:21.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:29:21.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327844+0000", 2024-09-17T07:29:21.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075217709999999993 2024-09-17T07:29:21.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327927+0000", 2024-09-17T07:29:21.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2628999999999998e-05 2024-09-17T07:29:21.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328225+0000", 2024-09-17T07:29:21.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029800600000000002 2024-09-17T07:29:21.942 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.942 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.942 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.942 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8028160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.943 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:21.943 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.633236240999999, 2024-09-17T07:29:21.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076061237000000004, 2024-09-17T07:29:21.943 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.943 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.943 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.943 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.943 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.943 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.943 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:29:21.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.943 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:21.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:21.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:21.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:21.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:29:21.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:29:21.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:29:21.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:29:21.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:29:21.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:29:21.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328268+0000", 2024-09-17T07:29:21.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075626323999999995 2024-09-17T07:29:21.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328352+0000", 2024-09-17T07:29:21.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3578999999999994e-05 2024-09-17T07:29:21.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328691+0000", 2024-09-17T07:29:21.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033915099999999999 2024-09-17T07:29:21.947 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.947 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.947 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.947 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8257536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.947 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:21.947 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.633220789999999, 2024-09-17T07:29:21.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076441903000000005, 2024-09-17T07:29:21.947 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.947 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.947 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.947 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.948 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.948 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.948 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:29:21.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.948 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:21.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:21.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:21.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:29:21.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:29:21.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:29:21.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:29:21.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:29:21.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:29:21.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:29:21.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328747+0000", 2024-09-17T07:29:21.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076094192000000005 2024-09-17T07:29:21.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328824+0000", 2024-09-17T07:29:21.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7096999999999997e-05 2024-09-17T07:29:21.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.329087+0000", 2024-09-17T07:29:21.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026249999999999998 2024-09-17T07:29:21.951 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20029 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.623135802, 2024-09-17T07:29:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071783696999999994, 2024-09-17T07:29:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20029 2024-09-17T07:29:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262735+0000", 2024-09-17T07:29:21.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.955e-06 2024-09-17T07:29:21.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262737+0000", 2024-09-17T07:29:21.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0090000000000002e-06 2024-09-17T07:29:21.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262743+0000", 2024-09-17T07:29:21.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2890000000000003e-06 2024-09-17T07:29:21.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262748+0000", 2024-09-17T07:29:21.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:21.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334163+0000", 2024-09-17T07:29:21.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071415336999999995 2024-09-17T07:29:21.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334243+0000", 2024-09-17T07:29:21.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9548000000000005e-05 2024-09-17T07:29:21.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334514+0000", 2024-09-17T07:29:21.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027086800000000001 2024-09-17T07:29:21.956 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.956 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.956 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.956 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7356416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.956 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:21.956 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.623104631, 2024-09-17T07:29:21.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072172616999999994, 2024-09-17T07:29:21.957 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.957 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.957 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.957 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:29:21.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.957 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:21.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:21.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:29:21.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:29:21.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:29:21.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:29:21.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:29:21.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:29:21.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:29:21.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:29:21.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334558+0000", 2024-09-17T07:29:21.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071780790999999997 2024-09-17T07:29:21.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334647+0000", 2024-09-17T07:29:21.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9448000000000001e-05 2024-09-17T07:29:21.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334934+0000", 2024-09-17T07:29:21.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028654599999999999 2024-09-17T07:29:21.960 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.961 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.961 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.961 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7585792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.961 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:21.961 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.623082919000002, 2024-09-17T07:29:21.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072606135000000002, 2024-09-17T07:29:21.961 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.961 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.961 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.962 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.962 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.962 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:29:21.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.962 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:21.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:21.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:21.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:29:21.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:29:21.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:29:21.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:29:21.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:29:21.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:29:21.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:29:21.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335116+0000", 2024-09-17T07:29:21.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072323897999999998 2024-09-17T07:29:21.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335166+0000", 2024-09-17T07:29:21.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9882000000000001e-05 2024-09-17T07:29:21.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335389+0000", 2024-09-17T07:29:21.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022334300000000001 2024-09-17T07:29:21.965 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.965 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.965 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.965 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7815168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.623070720000001, 2024-09-17T07:29:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073051230999999994, 2024-09-17T07:29:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:29:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:21.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:21.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:21.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:21.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:29:21.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:29:21.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:29:21.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:29:21.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:29:21.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:29:21.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335431+0000", 2024-09-17T07:29:21.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072627852000000007 2024-09-17T07:29:21.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335525+0000", 2024-09-17T07:29:21.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3708000000000005e-05 2024-09-17T07:29:21.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335846+0000", 2024-09-17T07:29:21.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032182000000000001 2024-09-17T07:29:21.970 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.970 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.970 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.970 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.970 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:21.970 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.623059862000002, 2024-09-17T07:29:21.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073284199999999994, 2024-09-17T07:29:21.970 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.970 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.970 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.971 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.971 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:29:21.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.971 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:21.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:21.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:21.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:29:21.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:29:21.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:29:21.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:29:21.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:29:21.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:29:21.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:29:21.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335863+0000", 2024-09-17T07:29:21.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073048921000000003 2024-09-17T07:29:21.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335904+0000", 2024-09-17T07:29:21.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1545000000000003e-05 2024-09-17T07:29:21.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336090+0000", 2024-09-17T07:29:21.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185787 2024-09-17T07:29:21.974 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.974 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.974 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.974 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8273920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:21.974 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:21.975 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.623045602000001, 2024-09-17T07:29:21.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073670478999999997, 2024-09-17T07:29:21.975 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:21.975 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:21.975 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:21.975 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:21.975 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:21.975 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:21.975 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:29:21.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.975 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:21.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:21.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:21.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:21.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:21.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:21.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:21.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:21.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:29:21.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:21.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:29:21.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:29:21.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:21.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:29:21.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:29:21.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:21.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:29:21.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:29:21.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:21.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336126+0000", 2024-09-17T07:29:21.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073297254000000006 2024-09-17T07:29:21.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:21.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336211+0000", 2024-09-17T07:29:21.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4728000000000006e-05 2024-09-17T07:29:21.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:21.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:21.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:21.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336491+0000", 2024-09-17T07:29:21.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027979399999999998 2024-09-17T07:29:21.978 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.979 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.979 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.979 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:21.979 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:21.979 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:21.981 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:21.981 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:21.981 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:21.978+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:bench [] [] [] [] 2024-09-17T07:29:21.982 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs debug_inject_read_zeros 2024-09-17T07:29:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs files list 2024-09-17T07:29:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs stats 2024-09-17T07:29:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:bluestore allocator dump block 2024-09-17T07:29:21.983 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: invalid command 2024-09-17T07:29:21.983 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T07:29:21.983 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-17T07:29:22.015 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-17T07:29:22.046 INFO:teuthology.orchestra.run.smithi045.stdout:1973 punch from 0xc5d90ab to 0xc5e5f1e, (0xce73 bytes) 2024-09-17T07:29:22.050 INFO:teuthology.orchestra.run.smithi045.stdout:1974 read 0x441681c thru 0x441aa61 (0x4246 bytes) 2024-09-17T07:29:22.050 INFO:teuthology.orchestra.run.smithi045.stdout:1975 write 0x9cbebfc thru 0x9cc7b4c (0x8f51 bytes) 2024-09-17T07:29:22.081 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:22.080+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:22.129 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:22.129 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:22.129 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:22.129 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:22.138 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:22.181 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:22.180+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:22.234 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-17T07:29:22.264 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:22.265 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:22.275 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:22.317 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:22.315+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:22.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-17T07:29:22.327+0000 7f0ebf7fe640 0 --2- 172.21.15.45:0/2708553816 >> v2:172.21.15.45:3300/0 conn(0x7f0ec0143400 0x7f0ec014e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-17T07:29:22.331 INFO:teuthology.orchestra.run.smithi045.stdout:1976 write 0x6d317d4 thru 0x6d32fea (0x1817 bytes) 2024-09-17T07:29:22.337 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:22.342 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:22.342 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T07:29:22.343 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-17T07:29:22.373 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:22.373 INFO:teuthology.orchestra.run.smithi045.stdout:1978 writesame 0xf11129 thru 0xf12ea9 data_size 0xd(0x1d81 bytes) 2024-09-17T07:29:22.383 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:22.382+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:22.429 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:22.429 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:22.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:22.429 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:22.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.429 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:22.429 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:22.429 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.521824699, 2024-09-17T07:29:22.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:29:22.429 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.429 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.429 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.430 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:29:22.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.430 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:22.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:22.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:22.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:29:22.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:29:22.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:29:22.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:29:22.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:29:22.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:29:22.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:29:22.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:29:22.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:29:22.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:29:22.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:29:22.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:29:22.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:29:22.434 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.435 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.435 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.435 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:22.435 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:22.435 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.521806022, 2024-09-17T07:29:22.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:29:22.435 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.435 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.435 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.436 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:29:22.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.436 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:22.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:22.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:29:22.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:22.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:29:22.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:29:22.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:29:22.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:29:22.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:29:22.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:29:22.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:29:22.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:29:22.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:29:22.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:29:22.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:29:22.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:29:22.439 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.439 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.439 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.439 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:22.439 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:22.439 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.521787668999998, 2024-09-17T07:29:22.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:29:22.440 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.440 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.440 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.440 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:29:22.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.440 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:22.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:22.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:22.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:29:22.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:29:22.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:29:22.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:29:22.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:29:22.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:29:22.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:29:22.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:29:22.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:29:22.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:29:22.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:29:22.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:29:22.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:29:22.444 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.444 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.444 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.444 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:22.444 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:22.444 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.521768071, 2024-09-17T07:29:22.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:29:22.444 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.444 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.444 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.445 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:29:22.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.445 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:22.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:22.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:29:22.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:29:22.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:29:22.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:29:22.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:29:22.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:29:22.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:29:22.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:29:22.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:29:22.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:29:22.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:29:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:29:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:29:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:29:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.521748116000001, 2024-09-17T07:29:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:29:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:29:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:29:22.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:29:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:29:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:29:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:29:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:29:22.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:29:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:29:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:29:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:29:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:29:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:29:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:29:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.521729461, 2024-09-17T07:29:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:29:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.453 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:29:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:29:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:29:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:29:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:29:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:29:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:29:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:29:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:29:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:29:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:29:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:29:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:29:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:29:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:22.457 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.521710984999999, 2024-09-17T07:29:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:29:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:29:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:29:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:29:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:29:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:29:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:29:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:29:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:29:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:29:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:29:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:29:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:29:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:29:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:29:22.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:29:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.521691386000001, 2024-09-17T07:29:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:29:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:29:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:29:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:29:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:29:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:29:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:29:22.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:29:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:29:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:29:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:29:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:29:22.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:29:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:29:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:29:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.521672810999998, 2024-09-17T07:29:22.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:29:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:29:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:22.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:29:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:29:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:29:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:29:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:29:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:29:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:29:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:29:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:29:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:29:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:29:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:29:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:29:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.521650384000001, 2024-09-17T07:29:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:29:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:29:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.471 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:22.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:29:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:29:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:29:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:29:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:29:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:29:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:29:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:29:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:29:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:29:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:29:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:29:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.521622567000001, 2024-09-17T07:29:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:29:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.475 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:29:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:29:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:29:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:29:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:29:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:29:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:29:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:29:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:29:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:29:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:29:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:29:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:29:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:29:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.521600623000001, 2024-09-17T07:29:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:29:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:29:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:29:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:29:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:29:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:29:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:29:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:29:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:29:22.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:29:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:29:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:29:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:29:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:29:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:29:22.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.521579518999999, 2024-09-17T07:29:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:29:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:29:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:22.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:29:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:29:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:29:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:29:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:29:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:29:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:29:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:29:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:29:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:29:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:29:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:29:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.488 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.521559916000001, 2024-09-17T07:29:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:29:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:29:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:29:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:29:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:29:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:29:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:29:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:29:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:29:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:29:22.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:29:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:29:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:29:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:29:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:29:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:29:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.56861880300000001, 2024-09-17T07:29:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:29:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:29:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:29:22.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:29:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:29:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:29:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:29:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:29:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:29:22.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:29:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:29:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:29:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.56860514299999998, 2024-09-17T07:29:22.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:29:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:29:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:22.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:29:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:29:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:29:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:29:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:29:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:29:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:29:22.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:29:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:29:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:29:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:29:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:29:22.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:29:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.56859036100000004, 2024-09-17T07:29:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:29:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:29:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:29:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:29:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:29:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:29:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:29:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:29:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:29:22.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:29:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:29:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:29:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:29:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:29:22.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:29:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.56857946999999998, 2024-09-17T07:29:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:29:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.507 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:29:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:29:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:29:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:29:22.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:29:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:29:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:29:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:29:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:29:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:29:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:29:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:29:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:29:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:22.511 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.56856991000000001, 2024-09-17T07:29:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:29:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:29:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:29:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:22.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:29:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:29:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:29:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:29:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:29:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:29:22.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:29:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:29:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:29:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:29:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:29:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:29:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.56855935000000002, 2024-09-17T07:29:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:29:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.516 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:29:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:29:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:29:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:29:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:29:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:29:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:29:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:29:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:29:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:29:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:29:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:29:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:29:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:29:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:29:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.520 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.520 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:bench [] [] [] [] 2024-09-17T07:29:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs debug_inject_read_zeros 2024-09-17T07:29:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs files list 2024-09-17T07:29:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs stats 2024-09-17T07:29:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:bluestore allocator dump block 2024-09-17T07:29:22.522 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: invalid command 2024-09-17T07:29:22.523 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:22.521+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:22.523 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:22.524 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:22.524 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T07:29:22.525 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-17T07:29:22.622 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:22.621+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:22.668 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:22.668 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:22.668 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:22.668 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:22.676 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:22.702 INFO:teuthology.orchestra.run.smithi045.stdout:1979 punch from 0x5bea7de to 0x5bf0fcb, (0x67ed bytes) 2024-09-17T07:29:22.703 INFO:teuthology.orchestra.run.smithi045.stdout:1980 read 0x9aa2e2c thru 0x9ab1ea5 (0xf07a bytes) 2024-09-17T07:29:22.703 INFO:teuthology.orchestra.run.smithi045.stdout:1981 write 0x457b863 thru 0x4582d24 (0x74c2 bytes) 2024-09-17T07:29:22.723 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:22.723+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:22.749 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:22.749 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:22.750 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:22.750 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:22.750 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:22.750 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:22.750 INFO:teuthology.orchestra.run.smithi045.stderr:bench [] [] [] [] 2024-09-17T07:29:22.750 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs debug_inject_read_zeros 2024-09-17T07:29:22.750 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs files list 2024-09-17T07:29:22.750 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs stats 2024-09-17T07:29:22.750 INFO:teuthology.orchestra.run.smithi045.stderr:bluestore allocator dump block 2024-09-17T07:29:22.750 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: invalid command 2024-09-17T07:29:22.752 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:22.753 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T07:29:22.753 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-17T07:29:22.821 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:22.821 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:22.821 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:22.821 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:22.821 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:22.831 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:22.832 INFO:teuthology.orchestra.run.smithi045.stdout:1982 write 0xc458ccc thru 0xc45add8 (0x210d bytes) 2024-09-17T07:29:22.832 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:22.832+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:22.833 INFO:teuthology.orchestra.run.smithi045.stdout:1983 punch from 0x37e20e2 to 0x37e2a78, (0x996 bytes) 2024-09-17T07:29:22.833 INFO:teuthology.orchestra.run.smithi045.stdout:1984 write 0x9055303 thru 0x905904e (0x3d4c bytes) 2024-09-17T07:29:22.926 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:22.924+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:22.936 INFO:teuthology.orchestra.run.smithi045.stdout:1986 punch from 0x556b69c to 0x556bc9c, (0x600 bytes) 2024-09-17T07:29:22.936 INFO:teuthology.orchestra.run.smithi045.stdout:1987 write 0x496766b thru 0x4974f60 (0xd8f6 bytes) 2024-09-17T07:29:22.938 INFO:teuthology.orchestra.run.smithi045.stdout:1988 read 0x5adbfdd thru 0x5aeb43f (0xf463 bytes) 2024-09-17T07:29:22.938 INFO:teuthology.orchestra.run.smithi045.stdout:1989 read 0x314776a thru 0x3156ad4 (0xf36b bytes) 2024-09-17T07:29:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:22.940 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:bench [] [] [] [] 2024-09-17T07:29:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs debug_inject_read_zeros 2024-09-17T07:29:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs files list 2024-09-17T07:29:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs stats 2024-09-17T07:29:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:bluestore allocator dump block 2024-09-17T07:29:22.941 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: invalid command 2024-09-17T07:29:22.941 INFO:teuthology.orchestra.run.smithi045.stdout:1990 writesame 0xbcd89b5 thru 0xbcd8a85 data_size 0x1(0xd1 bytes) 2024-09-17T07:29:22.943 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:22.943 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T07:29:22.943 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-17T07:29:22.945 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:22.945 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:22.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:22.945 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:22.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.945 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:29:22.945 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:22.945 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.609080659, 2024-09-17T07:29:22.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:29:22.945 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.945 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.945 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:22.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:29:22.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:29:22.946 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:29:22.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.946 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:22.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:22.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:29:22.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:29:22.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:29:22.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:29:22.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:29:22.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:29:22.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:29:22.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:29:22.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:29:22.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:29:22.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:29:22.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:29:22.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:29:22.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:29:22.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:29:22.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:29:22.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:29:22.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:29:22.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:29:22.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:29:22.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:22.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:29:22.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:29:22.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:22.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:29:22.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:29:22.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:22.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:29:22.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:29:22.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:29:22.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:29:22.951 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.951 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.951 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.951 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7378 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7176192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:22.952 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:22.952 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.020038625000002, 2024-09-17T07:29:22.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073003980999999996, 2024-09-17T07:29:22.952 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.952 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.952 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.952 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7378 2024-09-17T07:29:22.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.952 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:22.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1799999999999999e-07 2024-09-17T07:29:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923713+0000", 2024-09-17T07:29:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.057e-06 2024-09-17T07:29:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923716+0000", 2024-09-17T07:29:22.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0599999999999999e-06 2024-09-17T07:29:22.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923719+0000", 2024-09-17T07:29:22.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8219999999999998e-06 2024-09-17T07:29:22.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996401+0000", 2024-09-17T07:29:22.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072681467999999999 2024-09-17T07:29:22.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996452+0000", 2024-09-17T07:29:22.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.134e-05 2024-09-17T07:29:22.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996716+0000", 2024-09-17T07:29:22.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000264016 2024-09-17T07:29:22.956 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.956 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.956 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.957 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:22.957 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:22.957 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.020028508999999, 2024-09-17T07:29:22.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:29:22.957 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.957 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.957 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.957 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.957 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:29:22.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.957 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:22.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:22.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:29:22.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:29:22.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:29:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:29:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:29:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:29:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:29:22.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:29:22.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:29:22.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:29:22.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:29:22.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:29:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:29:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:29:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7398 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6389760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.014759975, 2024-09-17T07:29:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072891580999999997, 2024-09-17T07:29:22.961 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7398 2024-09-17T07:29:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928995+0000", 2024-09-17T07:29:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0759999999999996e-06 2024-09-17T07:29:22.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928999+0000", 2024-09-17T07:29:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1609999999999996e-06 2024-09-17T07:29:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929002+0000", 2024-09-17T07:29:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8449999999999999e-06 2024-09-17T07:29:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001598+0000", 2024-09-17T07:29:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072596318000000007 2024-09-17T07:29:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001663+0000", 2024-09-17T07:29:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4995999999999999e-05 2024-09-17T07:29:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001882+0000", 2024-09-17T07:29:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021897299999999999 2024-09-17T07:29:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.965 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.014745392999998, 2024-09-17T07:29:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:29:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:29:22.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.967 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:22.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:22.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:22.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:29:22.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:29:22.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:29:22.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:29:22.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:29:22.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:29:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:29:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:29:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:29:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:29:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:29:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:29:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:29:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.014714413, 2024-09-17T07:29:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:29:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:29:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:29:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:29:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:29:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:29:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:29:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:29:22.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:29:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:29:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:29:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:29:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:29:22.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:29:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:29:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:29:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.014684364000001, 2024-09-17T07:29:22.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:29:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:29:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:29:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:29:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:29:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:29:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:29:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:29:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:29:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:29:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:29:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:29:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:29:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:29:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:29:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:29:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.982 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.014655956999999, 2024-09-17T07:29:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:29:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:29:22.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:29:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:29:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:29:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:29:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:29:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:29:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:29:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:29:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:29:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:29:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:29:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:29:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.014636136, 2024-09-17T07:29:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:29:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.987 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:29:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:29:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:29:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:29:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:29:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:29:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:29:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:29:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:29:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:29:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:29:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:29:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:29:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:29:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:29:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.991 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.014611884000001, 2024-09-17T07:29:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:29:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:29:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:29:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:29:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:29:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:29:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:29:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:29:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:29:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:29:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:29:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:29:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:29:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:29:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:29:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:29:22.995 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7418 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7700480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.009205861000002, 2024-09-17T07:29:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072752965000000003, 2024-09-17T07:29:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:22.996 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7418 2024-09-17T07:29:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3599999999999999e-07 2024-09-17T07:29:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934547+0000", 2024-09-17T07:29:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3350000000000001e-06 2024-09-17T07:29:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934554+0000", 2024-09-17T07:29:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9859999999999997e-06 2024-09-17T07:29:22.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934557+0000", 2024-09-17T07:29:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.818e-06 2024-09-17T07:29:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007104+0000", 2024-09-17T07:29:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072546779000000006 2024-09-17T07:29:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007147+0000", 2024-09-17T07:29:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2904000000000001e-05 2024-09-17T07:29:22.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007298+0000", 2024-09-17T07:29:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015080700000000001 2024-09-17T07:29:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.009189295999999, 2024-09-17T07:29:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:29:23.000 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:29:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:29:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:29:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:29:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:29:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:29:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:29:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:29:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:29:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:29:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:29:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:29:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:29:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:29:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:29:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.004 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.009142421, 2024-09-17T07:29:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:29:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:23.005 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:29:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:29:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:29:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:29:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:29:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:29:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:29:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:29:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:29:23.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:29:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:29:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:29:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:29:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.009 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.009127716999998, 2024-09-17T07:29:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:29:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:29:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:29:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:29:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:29:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:29:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:29:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:29:23.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:29:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:29:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:29:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:29:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:29:23.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:29:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.009104621999999, 2024-09-17T07:29:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:29:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:23.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:29:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:29:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:29:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:29:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:29:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:29:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:29:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:29:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:29:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:29:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:29:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:29:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:29:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:29:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.009073247, 2024-09-17T07:29:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:29:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:23.018 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:29:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:23.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:29:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:29:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:29:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:29:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:29:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:29:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:29:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:29:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:29:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:29:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:29:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:29:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:29:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:29:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.022 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.009057986999998, 2024-09-17T07:29:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:29:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:29:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:23.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:29:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:29:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:29:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:29:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:29:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:29:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:29:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:29:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:29:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:29:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:29:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:29:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:29:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.026 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.009037917000001, 2024-09-17T07:29:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:29:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:29:23.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:23.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:29:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:29:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:29:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:29:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:29:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:29:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:29:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:29:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:29:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:29:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:29:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:23.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:29:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:29:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.009025977, 2024-09-17T07:29:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:29:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:23.031 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:29:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:23.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:29:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:29:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:23.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:29:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:29:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:29:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:29:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:29:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:29:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:29:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:29:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:29:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:29:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:23.035 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.009002818999999, 2024-09-17T07:29:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:29:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:29:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:23.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:29:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:29:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:29:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:29:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:29:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:29:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:29:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:29:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:29:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:29:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:29:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:29:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:29:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:29:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.039 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:23.040 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:23.040 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:23.042 INFO:teuthology.orchestra.run.smithi045.stdout:1991 punch from 0xd96c07b to 0xd970cd1, (0x4c56 bytes) 2024-09-17T07:29:23.042 INFO:teuthology.orchestra.run.smithi045.stdout:1992 read 0x8155ccc thru 0x815a77d (0x4ab2 bytes) 2024-09-17T07:29:23.042 INFO:teuthology.orchestra.run.smithi045.stdout:1993 read 0x88405b0 thru 0x884ea6a (0xe4bb bytes) 2024-09-17T07:29:23.042 INFO:teuthology.orchestra.run.smithi045.stdout:1994 write 0x804a021 thru 0x804e985 (0x4965 bytes) 2024-09-17T07:29:23.042 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:23.042 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:23.042 INFO:teuthology.orchestra.run.smithi045.stdout:1997 punch from 0xa8c86a2 to 0xa8d3117, (0xaa75 bytes) 2024-09-17T07:29:23.042 INFO:teuthology.orchestra.run.smithi045.stdout:1998 clone 21 order 22 su 524288 sc 8 2024-09-17T07:29:23.043 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:23.040+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:23.043 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:23.142 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:23.141+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:23.178 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:bench [] [] [] [] 2024-09-17T07:29:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs debug_inject_read_zeros 2024-09-17T07:29:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs files list 2024-09-17T07:29:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs stats 2024-09-17T07:29:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:bluestore allocator dump block 2024-09-17T07:29:23.179 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: invalid command 2024-09-17T07:29:23.182 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:23.183 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T07:29:23.194 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:23.195 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:23.203 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:23.245 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:23.244+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:23.311 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone20 from 0xea0dfe0 (overlap 0xea0dfe0) to 0x463765c 2024-09-17T07:29:23.343 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:23.343+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:23.359 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:23.360 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:23.368 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:23.444 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:23.443+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:23.530 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:23.531 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:23.531 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:23.531 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:23.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.531 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:23.531 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:23.531 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 25.396956224, 2024-09-17T07:29:23.531 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:29:23.531 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:23.531 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.532 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.532 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.532 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:29:23.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.532 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.532 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.532 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:23.532 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.532 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:23.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:29:23.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:29:23.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:29:23.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:29:23.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:29:23.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:29:23.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:29:23.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:29:23.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:29:23.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:29:23.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.535 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:29:23.535 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:29:23.535 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:29:23.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.535 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.535 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:29:23.535 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:29:23.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.535 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:23.535 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:29:23.535 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:29:23.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.536 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.536 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:29:23.536 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:29:23.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.536 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:23.536 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:29:23.536 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:29:23.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.536 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.536 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:29:23.537 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:29:23.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.537 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.537 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.537 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:23.537 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090989520000001, 2024-09-17T07:29:23.537 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:29:23.537 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.537 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.537 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:23.538 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.538 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.538 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.538 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:29:23.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.538 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.538 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.538 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.538 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:23.538 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.538 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.538 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.538 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:23.539 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.539 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.539 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:29:23.539 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:29:23.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.539 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.539 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:29:23.539 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:29:23.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.539 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.540 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:29:23.540 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:29:23.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.540 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.540 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:29:23.540 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:29:23.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.540 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.540 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:29:23.541 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:29:23.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.541 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:23.541 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:29:23.541 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:29:23.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.541 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.541 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:29:23.541 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:29:23.541 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.542 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.542 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.542 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.542 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:23.542 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090967405000001, 2024-09-17T07:29:23.542 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:29:23.542 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.542 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.542 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.542 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.542 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.542 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:29:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.544 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.544 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:29:23.544 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:29:23.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.544 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.544 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:29:23.544 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:29:23.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.544 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.544 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:29:23.544 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:29:23.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.545 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:29:23.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:29:23.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.545 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:29:23.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:29:23.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.545 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:29:23.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:29:23.546 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.546 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.546 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.546 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.546 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:23.546 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090943300999999, 2024-09-17T07:29:23.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:29:23.546 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.546 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.546 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.547 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.547 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.547 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.547 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:29:23.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.547 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:23.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:23.548 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.548 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.548 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:29:23.548 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:29:23.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.548 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.549 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:29:23.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:29:23.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.549 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.549 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:29:23.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:29:23.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.549 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.549 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:29:23.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:29:23.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.550 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.550 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:29:23.550 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:29:23.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.550 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.550 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:29:23.550 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:29:23.550 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.550 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.550 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.551 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.551 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:23.551 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090919211999999, 2024-09-17T07:29:23.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:29:23.551 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.551 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.551 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.551 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.551 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.551 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.551 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:29:23.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.551 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.551 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:23.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:23.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:23.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:29:23.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:29:23.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:29:23.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:29:23.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:29:23.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:29:23.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:29:23.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:29:23.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:29:23.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:29:23.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:29:23.554 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.554 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.554 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.555 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.555 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:23.555 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090894793, 2024-09-17T07:29:23.555 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:29:23.555 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.555 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.555 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.555 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.555 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.555 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.555 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:29:23.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.555 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.556 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.556 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:23.556 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.556 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.556 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:23.556 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.556 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.556 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:23.556 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:29:23.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:29:23.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:29:23.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:29:23.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:29:23.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:29:23.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:29:23.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.558 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:29:23.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:29:23.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.558 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:29:23.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:29:23.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.558 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.559 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.559 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:23.559 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090873093000001, 2024-09-17T07:29:23.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:29:23.559 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.559 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.559 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.559 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.559 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.559 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.559 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:29:23.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.560 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.560 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.560 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:23.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.560 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.560 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:23.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.561 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:23.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:29:23.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.561 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:29:23.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:29:23.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.561 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:29:23.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:29:23.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.562 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:29:23.562 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:29:23.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.562 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:29:23.562 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:29:23.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.562 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:29:23.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:29:23.563 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.563 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.563 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.563 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.563 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:23.563 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090828361, 2024-09-17T07:29:23.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:29:23.563 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.563 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.563 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.563 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.564 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.564 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.564 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:29:23.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.564 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:23.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:23.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:29:23.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:29:23.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:29:23.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:29:23.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:29:23.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:29:23.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:29:23.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:29:23.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:29:23.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:29:23.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.567 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.567 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:29:23.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:29:23.567 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.567 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.567 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.567 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.567 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:23.567 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090799895, 2024-09-17T07:29:23.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:29:23.567 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.567 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.568 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.568 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.568 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.568 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.568 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:29:23.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.568 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:23.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:23.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:23.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:29:23.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:29:23.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:29:23.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:29:23.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:29:23.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:29:23.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:29:23.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:29:23.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:29:23.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.571 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:29:23.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:29:23.571 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.571 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.571 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.571 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.571 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:23.571 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090682642999999, 2024-09-17T07:29:23.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:29:23.571 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.572 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.572 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.572 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.572 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:29:23.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.572 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:23.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.573 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:23.573 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.573 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.573 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:29:23.573 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:29:23.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.573 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.573 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:29:23.573 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:29:23.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.573 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.574 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:29:23.574 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:29:23.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.574 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.574 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:29:23.574 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:29:23.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.574 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.574 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:29:23.574 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:29:23.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.575 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.575 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:29:23.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:29:23.575 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.575 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.575 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.575 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.575 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:23.575 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090256541, 2024-09-17T07:29:23.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:29:23.575 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.575 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.576 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.576 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.576 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.576 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.576 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:29:23.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.576 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.576 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.576 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:23.576 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.576 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:23.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.577 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:29:23.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:23.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.577 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:29:23.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:29:23.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.578 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.578 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:29:23.578 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:29:23.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.578 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.578 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:29:23.578 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:29:23.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.578 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.578 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:29:23.578 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:29:23.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.579 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.579 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:29:23.579 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:29:23.579 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.579 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.579 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.579 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.579 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:23.579 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.09023687, 2024-09-17T07:29:23.579 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:29:23.580 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.580 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.580 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.580 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.580 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.580 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.580 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:29:23.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.580 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.581 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.581 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:23.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.581 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.581 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:23.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.581 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.581 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:23.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:29:23.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.582 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.582 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:29:23.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:29:23.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.582 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.582 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:29:23.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:29:23.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:29:23.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:29:23.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:29:23.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:29:23.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:29:23.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:29:23.583 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.584 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.584 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.584 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.584 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:23.584 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090214006, 2024-09-17T07:29:23.584 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:29:23.584 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.584 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.584 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.584 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.584 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.585 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:29:23.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.585 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.585 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.585 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:23.585 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.585 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.585 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:23.585 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.586 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.586 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:23.586 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:29:23.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.586 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.586 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:29:23.586 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:29:23.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.586 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.586 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:29:23.586 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:29:23.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:29:23.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:29:23.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:29:23.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:29:23.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:29:23.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:29:23.588 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.588 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.588 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.588 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.588 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:23.588 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090192705, 2024-09-17T07:29:23.588 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:29:23.588 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.588 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.588 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.588 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.588 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.589 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.589 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:29:23.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.589 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.589 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.589 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:23.589 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.589 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.589 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:23.589 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.590 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.590 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:29:23.590 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:29:23.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.590 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.590 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:29:23.590 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:29:23.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.590 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.591 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:29:23.591 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:29:23.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.591 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.591 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:29:23.591 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:29:23.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.591 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.591 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:29:23.591 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:29:23.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.592 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.592 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:29:23.592 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:29:23.592 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.592 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.592 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.592 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.592 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:23.592 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090144917, 2024-09-17T07:29:23.592 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:29:23.592 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.592 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.593 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.593 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.593 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.593 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.593 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:29:23.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.593 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.593 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.593 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:23.593 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.594 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.594 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:23.594 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.594 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.594 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:29:23.594 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:29:23.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.594 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.594 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:29:23.594 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:29:23.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.595 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.595 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:29:23.595 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:29:23.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.595 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.595 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.595 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:29:23.595 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:29:23.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.595 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.595 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.595 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:29:23.595 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:29:23.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.596 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.596 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.596 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:29:23.596 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:29:23.596 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.596 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.596 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.596 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.596 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.596 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:23.596 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090117695, 2024-09-17T07:29:23.596 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:29:23.597 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.597 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.597 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.597 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.597 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.597 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.597 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:29:23.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.597 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.597 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.597 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:23.597 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.598 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.598 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:23.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.598 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.598 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:23.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:29:23.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.598 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.598 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:29:23.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:29:23.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.599 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:29:23.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:29:23.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.599 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:29:23.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:29:23.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.599 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:29:23.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:29:23.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.600 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.600 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:29:23.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:29:23.600 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.600 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.600 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.600 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.600 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:23.600 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090097308000001, 2024-09-17T07:29:23.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:29:23.601 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.601 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.601 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.601 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.601 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.601 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.601 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:29:23.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.601 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:23.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:23.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:29:23.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:29:23.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:29:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:29:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:29:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:29:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:29:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:29:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.604 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:29:23.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:29:23.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.604 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.604 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:29:23.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:29:23.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.604 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.604 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.604 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:23.605 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090077602999999, 2024-09-17T07:29:23.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:29:23.605 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.605 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.605 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.605 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.605 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.605 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.605 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:29:23.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.605 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:23.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:23.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:29:23.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:29:23.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:29:23.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:29:23.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:29:23.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:29:23.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:29:23.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:29:23.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.608 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:29:23.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:29:23.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:29:23.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:29:23.609 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.609 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.609 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.609 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.609 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:23.609 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090054524999999, 2024-09-17T07:29:23.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:29:23.609 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.610 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.610 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.610 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.610 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.610 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:29:23.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.610 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:23.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:23.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:29:23.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:29:23.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:29:23.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:29:23.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:29:23.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:29:23.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:29:23.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:29:23.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:29:23.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:29:23.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:29:23.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:29:23.613 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.613 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.613 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.613 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:23.613 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:23.613 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 15.090033963, 2024-09-17T07:29:23.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:29:23.613 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.613 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.614 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:23.614 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.614 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.614 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.614 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:29:23.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.614 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:23.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:23.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:23.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:29:23.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:29:23.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:29:23.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:29:23.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:29:23.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:29:23.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:29:23.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:29:23.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:29:23.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:23.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:29:23.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:29:23.617 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.617 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.617 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.617 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.617 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.617 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:23.618 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:29:23.619 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:23.618+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:23.620 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:23.690 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-17T07:29:23.691 INFO:teuthology.orchestra.run.smithi045.stdout:1999 write 0x548885d thru 0x548be0d (0x35b1 bytes) 2024-09-17T07:29:23.698 INFO:teuthology.orchestra.run.smithi045.stdout:2000 writesame 0x3123719 thru 0x312a06b data_size 0x1(0x6953 bytes) 2024-09-17T07:29:23.720 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:23.718+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:23.743 INFO:teuthology.orchestra.run.smithi045.stdout:2001 read 0x2194495 thru 0x219c38f (0x7efb bytes) 2024-09-17T07:29:23.744 INFO:teuthology.orchestra.run.smithi045.stdout:2002 write 0x70b96ae thru 0x70bd0aa (0x39fd bytes) 2024-09-17T07:29:23.748 INFO:teuthology.orchestra.run.smithi045.stdout:2003 punch from 0x776973a to 0x776bc3e, (0x2504 bytes) 2024-09-17T07:29:23.748 INFO:teuthology.orchestra.run.smithi045.stdout:2004 read 0x431d02e thru 0x43263c5 (0x9398 bytes) 2024-09-17T07:29:23.748 INFO:teuthology.orchestra.run.smithi045.stdout:2005 write 0xb4d9d69 thru 0xb4dd514 (0x37ac bytes) 2024-09-17T07:29:23.770 INFO:teuthology.orchestra.run.smithi045.stdout:2006 trunc from 0xea0dfe0 to 0x525622 2024-09-17T07:29:23.779 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:23.780 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:23.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.780 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54833 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.780 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775572+0000", 2024-09-17T07:29:23.780 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0022862479999999998, 2024-09-17T07:29:23.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023061499999999999, 2024-09-17T07:29:23.780 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.780 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.780 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.780 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.780 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.781 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.781 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54833 2024-09-17T07:29:23.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.781 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775572+0000", 2024-09-17T07:29:23.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775572+0000", 2024-09-17T07:29:23.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.782 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775573+0000", 2024-09-17T07:29:23.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8599999999999998e-07 2024-09-17T07:29:23.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.782 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775575+0000", 2024-09-17T07:29:23.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5500000000000001e-06 2024-09-17T07:29:23.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775582+0000", 2024-09-17T07:29:23.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8460000000000002e-06 2024-09-17T07:29:23.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775590+0000", 2024-09-17T07:29:23.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8259999999999993e-06 2024-09-17T07:29:23.783 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.783 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.784 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.784 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54836 3.0 3.51b53d00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.784 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775624+0000", 2024-09-17T07:29:23.784 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0022348659999999998, 2024-09-17T07:29:23.784 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002303032, 2024-09-17T07:29:23.784 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.785 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.785 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.785 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.785 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.785 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54836 2024-09-17T07:29:23.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.785 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775624+0000", 2024-09-17T07:29:23.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775624+0000", 2024-09-17T07:29:23.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775624+0000", 2024-09-17T07:29:23.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4799999999999998e-07 2024-09-17T07:29:23.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775628+0000", 2024-09-17T07:29:23.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.827e-06 2024-09-17T07:29:23.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775635+0000", 2024-09-17T07:29:23.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5739999999999998e-06 2024-09-17T07:29:23.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775641+0000", 2024-09-17T07:29:23.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4979999999999997e-06 2024-09-17T07:29:23.790 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.790 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.790 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.790 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54837 3.0 3.51b53d00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.790 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775650+0000", 2024-09-17T07:29:23.790 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0022082709999999999, 2024-09-17T07:29:23.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023027859999999998, 2024-09-17T07:29:23.791 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.791 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.791 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.791 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.791 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.791 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.791 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54837 2024-09-17T07:29:23.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.791 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775650+0000", 2024-09-17T07:29:23.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.792 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.792 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775650+0000", 2024-09-17T07:29:23.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.792 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.792 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775651+0000", 2024-09-17T07:29:23.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5300000000000004e-07 2024-09-17T07:29:23.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.792 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.792 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775653+0000", 2024-09-17T07:29:23.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3690000000000001e-06 2024-09-17T07:29:23.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775660+0000", 2024-09-17T07:29:23.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8900000000000001e-06 2024-09-17T07:29:23.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775666+0000", 2024-09-17T07:29:23.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8209999999999998e-06 2024-09-17T07:29:23.793 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.793 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.793 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.794 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54840 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.794 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775678+0000", 2024-09-17T07:29:23.794 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0021800000000000001, 2024-09-17T07:29:23.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022978479999999999, 2024-09-17T07:29:23.794 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.794 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.794 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.794 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.794 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.794 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.794 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54840 2024-09-17T07:29:23.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.794 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775678+0000", 2024-09-17T07:29:23.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775678+0000", 2024-09-17T07:29:23.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775679+0000", 2024-09-17T07:29:23.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7e-07 2024-09-17T07:29:23.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775681+0000", 2024-09-17T07:29:23.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5380000000000001e-06 2024-09-17T07:29:23.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775688+0000", 2024-09-17T07:29:23.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0249999999999997e-06 2024-09-17T07:29:23.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775695+0000", 2024-09-17T07:29:23.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1349999999999997e-06 2024-09-17T07:29:23.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.797 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.797 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54841 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.797 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775710+0000", 2024-09-17T07:29:23.797 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0021480190000000001, 2024-09-17T07:29:23.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022890440000000001, 2024-09-17T07:29:23.797 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.797 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54841 2024-09-17T07:29:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775710+0000", 2024-09-17T07:29:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775710+0000", 2024-09-17T07:29:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775711+0000", 2024-09-17T07:29:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6899999999999998e-07 2024-09-17T07:29:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775713+0000", 2024-09-17T07:29:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5380000000000001e-06 2024-09-17T07:29:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775722+0000", 2024-09-17T07:29:23.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1240000000000005e-06 2024-09-17T07:29:23.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775727+0000", 2024-09-17T07:29:23.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.801e-06 2024-09-17T07:29:23.800 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.800 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.800 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.801 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54844 3.0 3.51b53d00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.801 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775736+0000", 2024-09-17T07:29:23.801 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0021227139999999999, 2024-09-17T07:29:23.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022964320000000002, 2024-09-17T07:29:23.801 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.801 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.801 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.801 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.801 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.801 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.801 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54844 2024-09-17T07:29:23.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.801 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775736+0000", 2024-09-17T07:29:23.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775736+0000", 2024-09-17T07:29:23.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775736+0000", 2024-09-17T07:29:23.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:29:23.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775738+0000", 2024-09-17T07:29:23.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3520000000000001e-06 2024-09-17T07:29:23.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775746+0000", 2024-09-17T07:29:23.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2230000000000001e-06 2024-09-17T07:29:23.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775751+0000", 2024-09-17T07:29:23.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7760000000000003e-06 2024-09-17T07:29:23.804 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.804 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.804 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.804 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54845 3.0 3.51b53d00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.804 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775760+0000", 2024-09-17T07:29:23.804 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.00209866, 2024-09-17T07:29:23.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022957469999999999, 2024-09-17T07:29:23.804 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.804 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.804 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.804 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.805 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.805 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54845 2024-09-17T07:29:23.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.805 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775760+0000", 2024-09-17T07:29:23.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775760+0000", 2024-09-17T07:29:23.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775760+0000", 2024-09-17T07:29:23.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6199999999999998e-07 2024-09-17T07:29:23.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775763+0000", 2024-09-17T07:29:23.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4880000000000001e-06 2024-09-17T07:29:23.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775769+0000", 2024-09-17T07:29:23.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4060000000000002e-06 2024-09-17T07:29:23.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775774+0000", 2024-09-17T07:29:23.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3159999999999996e-06 2024-09-17T07:29:23.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.807 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.807 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54848 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.807 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775783+0000", 2024-09-17T07:29:23.807 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0020750199999999999, 2024-09-17T07:29:23.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022951669999999999, 2024-09-17T07:29:23.807 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.808 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.808 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.808 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.808 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54848 2024-09-17T07:29:23.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.808 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775783+0000", 2024-09-17T07:29:23.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775783+0000", 2024-09-17T07:29:23.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775784+0000", 2024-09-17T07:29:23.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4700000000000002e-07 2024-09-17T07:29:23.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775786+0000", 2024-09-17T07:29:23.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4420000000000001e-06 2024-09-17T07:29:23.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775798+0000", 2024-09-17T07:29:23.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1601e-05 2024-09-17T07:29:23.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775804+0000", 2024-09-17T07:29:23.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3969999999999999e-06 2024-09-17T07:29:23.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.811 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.811 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54849 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.811 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775813+0000", 2024-09-17T07:29:23.811 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0020455920000000002, 2024-09-17T07:29:23.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023047240000000002, 2024-09-17T07:29:23.811 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.811 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.811 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.811 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.811 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54849 2024-09-17T07:29:23.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.812 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.812 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775813+0000", 2024-09-17T07:29:23.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.812 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775813+0000", 2024-09-17T07:29:23.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775813+0000", 2024-09-17T07:29:23.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:29:23.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775816+0000", 2024-09-17T07:29:23.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.05e-06 2024-09-17T07:29:23.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775824+0000", 2024-09-17T07:29:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.413e-06 2024-09-17T07:29:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775829+0000", 2024-09-17T07:29:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4389999999999999e-06 2024-09-17T07:29:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54852 3.0 3.51b53d00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775843+0000", 2024-09-17T07:29:23.814 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.00201527, 2024-09-17T07:29:23.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023007689999999998, 2024-09-17T07:29:23.815 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.815 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.815 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.815 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.815 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54852 2024-09-17T07:29:23.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.815 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775843+0000", 2024-09-17T07:29:23.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775843+0000", 2024-09-17T07:29:23.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775844+0000", 2024-09-17T07:29:23.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5799999999999999e-07 2024-09-17T07:29:23.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775846+0000", 2024-09-17T07:29:23.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5119999999999998e-06 2024-09-17T07:29:23.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775855+0000", 2024-09-17T07:29:23.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0299999999999999e-06 2024-09-17T07:29:23.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775860+0000", 2024-09-17T07:29:23.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5739999999999997e-06 2024-09-17T07:29:23.817 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.817 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.817 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.818 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54853 3.0 3.51b53d00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.818 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775868+0000", 2024-09-17T07:29:23.818 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0019900730000000002, 2024-09-17T07:29:23.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022982889999999998, 2024-09-17T07:29:23.818 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.818 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.818 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.818 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.818 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54853 2024-09-17T07:29:23.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.818 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775868+0000", 2024-09-17T07:29:23.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775868+0000", 2024-09-17T07:29:23.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775869+0000", 2024-09-17T07:29:23.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:29:23.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775871+0000", 2024-09-17T07:29:23.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6819999999999999e-06 2024-09-17T07:29:23.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775879+0000", 2024-09-17T07:29:23.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8089999999999993e-06 2024-09-17T07:29:23.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775885+0000", 2024-09-17T07:29:23.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5389999999999998e-06 2024-09-17T07:29:23.821 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.821 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.821 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.821 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54856 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.821 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775908+0000", 2024-09-17T07:29:23.821 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0019503859999999999, 2024-09-17T07:29:23.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022816640000000001, 2024-09-17T07:29:23.821 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.822 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.822 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.822 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.822 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54856 2024-09-17T07:29:23.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.822 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775908+0000", 2024-09-17T07:29:23.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775908+0000", 2024-09-17T07:29:23.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775909+0000", 2024-09-17T07:29:23.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:29:23.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775911+0000", 2024-09-17T07:29:23.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7690000000000001e-06 2024-09-17T07:29:23.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775921+0000", 2024-09-17T07:29:23.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9490000000000008e-06 2024-09-17T07:29:23.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775927+0000", 2024-09-17T07:29:23.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6790000000000002e-06 2024-09-17T07:29:23.824 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.824 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.824 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.824 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54857 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.825 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775936+0000", 2024-09-17T07:29:23.825 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.001922574, 2024-09-17T07:29:23.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022769069999999999, 2024-09-17T07:29:23.825 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.825 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.825 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.825 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54857 2024-09-17T07:29:23.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.825 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775936+0000", 2024-09-17T07:29:23.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775936+0000", 2024-09-17T07:29:23.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775937+0000", 2024-09-17T07:29:23.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7100000000000001e-07 2024-09-17T07:29:23.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775939+0000", 2024-09-17T07:29:23.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4710000000000001e-06 2024-09-17T07:29:23.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775948+0000", 2024-09-17T07:29:23.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0159999999999997e-06 2024-09-17T07:29:23.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775954+0000", 2024-09-17T07:29:23.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8309999999999997e-06 2024-09-17T07:29:23.827 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.828 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.828 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.828 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54860 3.0 3.51b53d00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.828 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775962+0000", 2024-09-17T07:29:23.828 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.001896254, 2024-09-17T07:29:23.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022736950000000001, 2024-09-17T07:29:23.828 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.828 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.828 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.828 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.829 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.829 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54860 2024-09-17T07:29:23.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.829 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775962+0000", 2024-09-17T07:29:23.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775962+0000", 2024-09-17T07:29:23.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775963+0000", 2024-09-17T07:29:23.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9100000000000004e-07 2024-09-17T07:29:23.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775965+0000", 2024-09-17T07:29:23.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.39e-06 2024-09-17T07:29:23.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775972+0000", 2024-09-17T07:29:23.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8290000000000002e-06 2024-09-17T07:29:23.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775978+0000", 2024-09-17T07:29:23.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.489e-06 2024-09-17T07:29:23.831 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.831 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.831 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.831 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54861 3.0 3.51b53d00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.831 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775985+0000", 2024-09-17T07:29:23.831 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0018729319999999999, 2024-09-17T07:29:23.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022736150000000001, 2024-09-17T07:29:23.832 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.832 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.832 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.832 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.832 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.832 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54861 2024-09-17T07:29:23.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.832 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775985+0000", 2024-09-17T07:29:23.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775985+0000", 2024-09-17T07:29:23.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775986+0000", 2024-09-17T07:29:23.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8799999999999998e-07 2024-09-17T07:29:23.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775988+0000", 2024-09-17T07:29:23.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0360000000000001e-06 2024-09-17T07:29:23.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775995+0000", 2024-09-17T07:29:23.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1169999999999998e-06 2024-09-17T07:29:23.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776001+0000", 2024-09-17T07:29:23.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9540000000000001e-06 2024-09-17T07:29:23.834 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.834 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.835 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.835 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54864 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.835 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.776009+0000", 2024-09-17T07:29:23.835 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0018492280000000001, 2024-09-17T07:29:23.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00227283, 2024-09-17T07:29:23.835 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.835 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.835 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.835 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.835 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.835 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54864 2024-09-17T07:29:23.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.836 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776009+0000", 2024-09-17T07:29:23.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776009+0000", 2024-09-17T07:29:23.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776010+0000", 2024-09-17T07:29:23.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2300000000000002e-07 2024-09-17T07:29:23.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776012+0000", 2024-09-17T07:29:23.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0779999999999998e-06 2024-09-17T07:29:23.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776020+0000", 2024-09-17T07:29:23.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5210000000000002e-06 2024-09-17T07:29:23.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776025+0000", 2024-09-17T07:29:23.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1050000000000003e-06 2024-09-17T07:29:23.838 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.838 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.838 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.838 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54865 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.838 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.776033+0000", 2024-09-17T07:29:23.838 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.001825576, 2024-09-17T07:29:23.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022723169999999998, 2024-09-17T07:29:23.838 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.839 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.839 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.839 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.839 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.839 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54865 2024-09-17T07:29:23.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.839 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776033+0000", 2024-09-17T07:29:23.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776033+0000", 2024-09-17T07:29:23.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776033+0000", 2024-09-17T07:29:23.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.08e-07 2024-09-17T07:29:23.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776035+0000", 2024-09-17T07:29:23.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0959999999999999e-06 2024-09-17T07:29:23.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776044+0000", 2024-09-17T07:29:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2439999999999993e-06 2024-09-17T07:29:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776049+0000", 2024-09-17T07:29:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.276e-06 2024-09-17T07:29:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54868 3.0 3.51b53d00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.776057+0000", 2024-09-17T07:29:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0018013230000000001, 2024-09-17T07:29:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002274084, 2024-09-17T07:29:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54868 2024-09-17T07:29:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776057+0000", 2024-09-17T07:29:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776057+0000", 2024-09-17T07:29:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776058+0000", 2024-09-17T07:29:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.75e-07 2024-09-17T07:29:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776060+0000", 2024-09-17T07:29:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5220000000000002e-06 2024-09-17T07:29:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776067+0000", 2024-09-17T07:29:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8519999999999998e-06 2024-09-17T07:29:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776072+0000", 2024-09-17T07:29:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8199999999999996e-06 2024-09-17T07:29:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54869 3.0 3.51b53d00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.776079+0000", 2024-09-17T07:29:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0017792190000000001, 2024-09-17T07:29:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022750829999999998, 2024-09-17T07:29:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54869 2024-09-17T07:29:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776079+0000", 2024-09-17T07:29:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776079+0000", 2024-09-17T07:29:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776080+0000", 2024-09-17T07:29:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:29:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776082+0000", 2024-09-17T07:29:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.26e-06 2024-09-17T07:29:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776089+0000", 2024-09-17T07:29:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5969999999999997e-06 2024-09-17T07:29:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776094+0000", 2024-09-17T07:29:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5899999999999998e-06 2024-09-17T07:29:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54872 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.776101+0000", 2024-09-17T07:29:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.001757397, 2024-09-17T07:29:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022763869999999999, 2024-09-17T07:29:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54872 2024-09-17T07:29:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776101+0000", 2024-09-17T07:29:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776101+0000", 2024-09-17T07:29:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776101+0000", 2024-09-17T07:29:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9000000000000002e-07 2024-09-17T07:29:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776103+0000", 2024-09-17T07:29:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:29:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776110+0000", 2024-09-17T07:29:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4670000000000001e-06 2024-09-17T07:29:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776115+0000", 2024-09-17T07:29:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2310000000000004e-06 2024-09-17T07:29:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54873 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.776122+0000", 2024-09-17T07:29:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0017366899999999999, 2024-09-17T07:29:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002278314, 2024-09-17T07:29:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54873 2024-09-17T07:29:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776122+0000", 2024-09-17T07:29:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776122+0000", 2024-09-17T07:29:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776122+0000", 2024-09-17T07:29:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1300000000000001e-07 2024-09-17T07:29:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776139+0000", 2024-09-17T07:29:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6833e-05 2024-09-17T07:29:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776147+0000", 2024-09-17T07:29:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7440000000000007e-06 2024-09-17T07:29:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776152+0000", 2024-09-17T07:29:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4759999999999997e-06 2024-09-17T07:29:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.856 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54876 3.0 3.51b53d00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.856 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.776160+0000", 2024-09-17T07:29:23.856 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.001698176, 2024-09-17T07:29:23.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022625309999999999, 2024-09-17T07:29:23.856 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.856 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.856 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.856 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.856 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54876 2024-09-17T07:29:23.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.857 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776160+0000", 2024-09-17T07:29:23.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776160+0000", 2024-09-17T07:29:23.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776161+0000", 2024-09-17T07:29:23.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3799999999999998e-07 2024-09-17T07:29:23.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776163+0000", 2024-09-17T07:29:23.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0899999999999999e-06 2024-09-17T07:29:23.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776170+0000", 2024-09-17T07:29:23.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6179999999999997e-06 2024-09-17T07:29:23.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776175+0000", 2024-09-17T07:29:23.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1139999999999997e-06 2024-09-17T07:29:23.859 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.859 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.859 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.859 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54877 3.0 3.51b53d00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.859 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.776182+0000", 2024-09-17T07:29:23.859 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.001676547, 2024-09-17T07:29:23.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022638200000000002, 2024-09-17T07:29:23.859 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.859 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.860 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.860 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.860 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.860 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54877 2024-09-17T07:29:23.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.860 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776182+0000", 2024-09-17T07:29:23.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776182+0000", 2024-09-17T07:29:23.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776182+0000", 2024-09-17T07:29:23.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:29:23.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776184+0000", 2024-09-17T07:29:23.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7859999999999999e-06 2024-09-17T07:29:23.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776191+0000", 2024-09-17T07:29:23.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6950000000000003e-06 2024-09-17T07:29:23.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776196+0000", 2024-09-17T07:29:23.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2100000000000001e-06 2024-09-17T07:29:23.862 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.862 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.862 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.862 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54880 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.862 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.776202+0000", 2024-09-17T07:29:23.863 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.001655927, 2024-09-17T07:29:23.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022659550000000001, 2024-09-17T07:29:23.863 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.863 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.863 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.863 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.863 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.863 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54880 2024-09-17T07:29:23.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.863 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776202+0000", 2024-09-17T07:29:23.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776202+0000", 2024-09-17T07:29:23.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776203+0000", 2024-09-17T07:29:23.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0900000000000002e-07 2024-09-17T07:29:23.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776205+0000", 2024-09-17T07:29:23.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9309999999999998e-06 2024-09-17T07:29:23.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776211+0000", 2024-09-17T07:29:23.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1870000000000002e-06 2024-09-17T07:29:23.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776216+0000", 2024-09-17T07:29:23.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1139999999999997e-06 2024-09-17T07:29:23.865 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.866 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.866 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.866 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54881 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.866 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.776224+0000", 2024-09-17T07:29:23.866 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.001634679, 2024-09-17T07:29:23.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022679699999999998, 2024-09-17T07:29:23.866 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.866 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.866 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.866 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.866 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.866 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54881 2024-09-17T07:29:23.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.867 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776224+0000", 2024-09-17T07:29:23.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776224+0000", 2024-09-17T07:29:23.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776224+0000", 2024-09-17T07:29:23.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:29:23.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776226+0000", 2024-09-17T07:29:23.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1670000000000002e-06 2024-09-17T07:29:23.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776233+0000", 2024-09-17T07:29:23.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:29:23.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776238+0000", 2024-09-17T07:29:23.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8539999999999996e-06 2024-09-17T07:29:23.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.869 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.869 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54884 3.0 3.51b53d00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.869 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.776245+0000", 2024-09-17T07:29:23.869 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.001613607, 2024-09-17T07:29:23.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00226991, 2024-09-17T07:29:23.869 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.869 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.869 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.870 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.870 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.870 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.870 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54884 2024-09-17T07:29:23.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.870 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776245+0000", 2024-09-17T07:29:23.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776245+0000", 2024-09-17T07:29:23.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776245+0000", 2024-09-17T07:29:23.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0699999999999997e-07 2024-09-17T07:29:23.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776248+0000", 2024-09-17T07:29:23.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4660000000000002e-06 2024-09-17T07:29:23.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776255+0000", 2024-09-17T07:29:23.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0149999999999998e-06 2024-09-17T07:29:23.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776260+0000", 2024-09-17T07:29:23.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8690000000000003e-06 2024-09-17T07:29:23.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.872 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.872 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54885 3.0 3.51b53d00 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.873 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.776266+0000", 2024-09-17T07:29:23.873 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.001592266, 2024-09-17T07:29:23.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022714889999999998, 2024-09-17T07:29:23.873 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.873 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.873 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.873 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.873 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54885 2024-09-17T07:29:23.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.873 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776266+0000", 2024-09-17T07:29:23.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776266+0000", 2024-09-17T07:29:23.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776267+0000", 2024-09-17T07:29:23.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.39e-07 2024-09-17T07:29:23.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776269+0000", 2024-09-17T07:29:23.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:29:23.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776275+0000", 2024-09-17T07:29:23.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:29:23.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776279+0000", 2024-09-17T07:29:23.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7740000000000004e-06 2024-09-17T07:29:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54888 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.776286+0000", 2024-09-17T07:29:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.001572869, 2024-09-17T07:29:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002274987, 2024-09-17T07:29:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.876 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54888 2024-09-17T07:29:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776286+0000", 2024-09-17T07:29:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776286+0000", 2024-09-17T07:29:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776286+0000", 2024-09-17T07:29:23.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1800000000000002e-07 2024-09-17T07:29:23.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776288+0000", 2024-09-17T07:29:23.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8789999999999999e-06 2024-09-17T07:29:23.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776293+0000", 2024-09-17T07:29:23.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6999999999999996e-06 2024-09-17T07:29:23.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.879 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776298+0000", 2024-09-17T07:29:23.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9130000000000002e-06 2024-09-17T07:29:23.879 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.879 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.879 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.879 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54889 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.879 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.776304+0000", 2024-09-17T07:29:23.879 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.001554127, 2024-09-17T07:29:23.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022789429999999999, 2024-09-17T07:29:23.880 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.880 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.880 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.880 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.880 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.880 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.880 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54889 2024-09-17T07:29:23.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.880 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776304+0000", 2024-09-17T07:29:23.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776304+0000", 2024-09-17T07:29:23.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776305+0000", 2024-09-17T07:29:23.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6100000000000002e-07 2024-09-17T07:29:23.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776306+0000", 2024-09-17T07:29:23.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.716e-06 2024-09-17T07:29:23.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776312+0000", 2024-09-17T07:29:23.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7309999999999998e-06 2024-09-17T07:29:23.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.776317+0000", 2024-09-17T07:29:23.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4580000000000003e-06 2024-09-17T07:29:23.882 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.882 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.882 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.883 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54829 3.0 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:17 [sparse-read 4194304~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.883 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775500+0000", 2024-09-17T07:29:23.883 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0023581040000000002, 2024-09-17T07:29:23.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003109236, 2024-09-17T07:29:23.883 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.883 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.883 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:23.883 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.883 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.883 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.883 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54829 2024-09-17T07:29:23.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775500+0000", 2024-09-17T07:29:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775500+0000", 2024-09-17T07:29:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775501+0000", 2024-09-17T07:29:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9999999999999997e-07 2024-09-17T07:29:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775504+0000", 2024-09-17T07:29:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1599999999999998e-06 2024-09-17T07:29:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775518+0000", 2024-09-17T07:29:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3874e-05 2024-09-17T07:29:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775526+0000", 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.622e-06 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.777414+0000", 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001888182 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.777484+0000", 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9962000000000005e-05 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.886 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.887 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:54832 3.0 3.9d007f20 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-17T07:29:23.887 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.775546+0000", 2024-09-17T07:29:23.887 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0023124909999999999, 2024-09-17T07:29:23.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0030917219999999999, 2024-09-17T07:29:23.887 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:23.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:23.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:23.887 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:23.887 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:23.887 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:23.887 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 54832 2024-09-17T07:29:23.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.888 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:23.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:23.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775546+0000", 2024-09-17T07:29:23.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:23.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775546+0000", 2024-09-17T07:29:23.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:23.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:23.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775546+0000", 2024-09-17T07:29:23.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999996e-07 2024-09-17T07:29:23.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:23.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775549+0000", 2024-09-17T07:29:23.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9450000000000002e-06 2024-09-17T07:29:23.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:23.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775557+0000", 2024-09-17T07:29:23.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7000000000000008e-06 2024-09-17T07:29:23.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:23.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:23.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:23.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.775563+0000", 2024-09-17T07:29:23.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8560000000000003e-06 2024-09-17T07:29:23.890 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.890 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:23.890 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.890 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:23.890 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:29:23.890 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 31 2024-09-17T07:29:23.890 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:23.892 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:23.892 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:23.892 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:23.893 INFO:teuthology.orchestra.run.smithi045.stdout:2007 read 0x2d8d40 thru 0x2df168 (0x6429 bytes) 2024-09-17T07:29:23.893 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:23.893 INFO:teuthology.orchestra.run.smithi045.stdout:2009 read 0x15c964 thru 0x162bb4 (0x6251 bytes) 2024-09-17T07:29:23.893 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:23.893 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:23.894 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:23.894 INFO:teuthology.orchestra.run.smithi045.stdout:2013 write 0xe56dcb2 thru 0xe579222 (0xb571 bytes) 2024-09-17T07:29:23.894 INFO:teuthology.orchestra.run.smithi045.stdout:2014 write 0x89522a2 thru 0x8961ed3 (0xfc32 bytes) 2024-09-17T07:29:23.895 INFO:teuthology.orchestra.run.smithi045.stdout:2015 writesame 0xcaf34c6 thru 0xcafce0b data_size 0x17(0x9946 bytes) 2024-09-17T07:29:23.895 INFO:teuthology.orchestra.run.smithi045.stdout:2016 read 0xc1a07cd thru 0xc1a4b31 (0x4365 bytes) 2024-09-17T07:29:23.895 INFO:teuthology.orchestra.run.smithi045.stdout:2017 read 0xb6b4b35 thru 0xb6bbe8c (0x7358 bytes) 2024-09-17T07:29:23.895 INFO:teuthology.orchestra.run.smithi045.stdout:2019 writesame 0xddefe46 thru 0xddf4413 data_size 0x5(0x45ce bytes) 2024-09-17T07:29:23.895 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:23.895 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:23.895 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:23.896 INFO:teuthology.orchestra.run.smithi045.stdout:2021 read 0x8ede06b thru 0x8ee22a8 (0x423e bytes) 2024-09-17T07:29:23.896 INFO:teuthology.orchestra.run.smithi045.stdout:2023 trunc from 0xe579223 to 0xde0743e 2024-09-17T07:29:23.896 INFO:teuthology.orchestra.run.smithi045.stdout:2024 punch from 0x5f1bf34 to 0x5f1eb15, (0x2be1 bytes) 2024-09-17T07:29:23.896 INFO:teuthology.orchestra.run.smithi045.stdout:2026 writesame 0x95c9572 thru 0x95d9393 data_size 0x15(0xfe22 bytes) 2024-09-17T07:29:23.897 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:23.897 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:23.897 INFO:teuthology.orchestra.run.smithi045.stdout:2027 trunc from 0xde0743e to 0x39af9b8 2024-09-17T07:29:23.897 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:23.898 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:23.898 INFO:teuthology.orchestra.run.smithi045.stdout:2028 trunc from 0x39af9b8 to 0xabdf008 2024-09-17T07:29:23.898 INFO:teuthology.orchestra.run.smithi045.stdout:2031 read 0x9e35833 thru 0x9e37085 (0x1853 bytes) 2024-09-17T07:29:23.898 INFO:teuthology.orchestra.run.smithi045.stdout:2033 write 0x7e4f97a thru 0x7e53a25 (0x40ac bytes) 2024-09-17T07:29:23.898 INFO:teuthology.orchestra.run.smithi045.stdout:2035 read 0x6d966e9 thru 0x6da4c65 (0xe57d bytes) 2024-09-17T07:29:23.899 INFO:teuthology.orchestra.run.smithi045.stdout:2036 punch from 0xa79fbfc to 0xa7acfa8, (0xd3ac bytes) 2024-09-17T07:29:23.899 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:23.891+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:23.899 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:23.899 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:23.901 INFO:teuthology.orchestra.run.smithi045.stdout:2037 write 0xe0b9171 thru 0xe0ba2cf (0x115f bytes) 2024-09-17T07:29:23.903 INFO:teuthology.orchestra.run.smithi045.stdout:2038 read 0xa7151a2 thru 0xa720ce4 (0xbb43 bytes) 2024-09-17T07:29:23.903 INFO:teuthology.orchestra.run.smithi045.stdout:2039 write 0xd7c3674 thru 0xd7d20d2 (0xea5f bytes) 2024-09-17T07:29:23.906 INFO:teuthology.orchestra.run.smithi045.stdout:2040 write 0xa039600 thru 0xa03c557 (0x2f58 bytes) 2024-09-17T07:29:23.910 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:23.910 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:23.910 INFO:teuthology.orchestra.run.smithi045.stdout:2041 write 0xb7b24d0 thru 0xb7bf38c (0xcebd bytes) 2024-09-17T07:29:23.913 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:23.913 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:23.915 INFO:teuthology.orchestra.run.smithi045.stdout:2043 trunc from 0xe0ba2d0 to 0xd9cd7c3 2024-09-17T07:29:23.925 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:23.925 INFO:teuthology.orchestra.run.smithi045.stdout:2047 punch from 0xbb0287 to 0xbb9c30, (0x99a9 bytes) 2024-09-17T07:29:23.925 INFO:teuthology.orchestra.run.smithi045.stdout:2049 writesame 0x83c2ab5 thru 0x83d0b6a data_size 0x31(0xe0b6 bytes) 2024-09-17T07:29:23.927 INFO:teuthology.orchestra.run.smithi045.stdout:2050 writesame 0xb3530e5 thru 0xb3615f5 data_size 0xb(0xe511 bytes) 2024-09-17T07:29:23.931 INFO:teuthology.orchestra.run.smithi045.stdout:2053 writesame 0x77a838 thru 0x786ca2 data_size 0x25(0xc46b bytes) 2024-09-17T07:29:23.933 INFO:teuthology.orchestra.run.smithi045.stdout:2054 write 0x818cbd3 thru 0x81935f1 (0x6a1f bytes) 2024-09-17T07:29:23.936 INFO:teuthology.orchestra.run.smithi045.stdout:2055 read 0x6f28acb thru 0x6f319c3 (0x8ef9 bytes) 2024-09-17T07:29:23.936 INFO:teuthology.orchestra.run.smithi045.stdout:2056 writesame 0xb0a9e9e thru 0xb0b51c7 data_size 0x47(0xb32a bytes) 2024-09-17T07:29:23.938 INFO:teuthology.orchestra.run.smithi045.stdout:2057 punch from 0xadf03e3 to 0xadfe279, (0xde96 bytes) 2024-09-17T07:29:23.939 INFO:teuthology.orchestra.run.smithi045.stdout:2058 write 0x7215d5b thru 0x721eba1 (0x8e47 bytes) 2024-09-17T07:29:23.940 INFO:teuthology.orchestra.run.smithi045.stdout:2059 punch from 0x7027a82 to 0x70299c7, (0x1f45 bytes) 2024-09-17T07:29:23.940 INFO:teuthology.orchestra.run.smithi045.stdout:2060 punch from 0xd5a3579 to 0xd5ad27d, (0x9d04 bytes) 2024-09-17T07:29:23.940 INFO:teuthology.orchestra.run.smithi045.stdout:2061 write 0x3d69c25 thru 0x3d72498 (0x8874 bytes) 2024-09-17T07:29:23.943 INFO:teuthology.orchestra.run.smithi045.stdout:2062 trunc from 0xd9cd7c3 to 0xe227545 2024-09-17T07:29:23.947 INFO:teuthology.orchestra.run.smithi045.stdout:2063 write 0x590a5ed thru 0x590a7e6 (0x1fa bytes) 2024-09-17T07:29:23.956 INFO:teuthology.orchestra.run.smithi045.stdout:2066 trunc from 0xe227545 to 0x901bdfe 2024-09-17T07:29:23.965 INFO:teuthology.orchestra.run.smithi045.stdout:2067 read 0x6639dd thru 0x670759 (0xcd7d bytes) 2024-09-17T07:29:23.965 INFO:teuthology.orchestra.run.smithi045.stdout:2068 punch from 0x1b55cd1 to 0x1b5da3c, (0x7d6b bytes) 2024-09-17T07:29:23.969 INFO:teuthology.orchestra.run.smithi045.stdout:2069 trunc from 0x901bdfe to 0x62f4e76 2024-09-17T07:29:23.978 INFO:teuthology.orchestra.run.smithi045.stdout:2070 trunc from 0x62f4e76 to 0x644b84e 2024-09-17T07:29:23.982 INFO:teuthology.orchestra.run.smithi045.stdout:2071 read 0x2cc4285 thru 0x2ccbec3 (0x7c3f bytes) 2024-09-17T07:29:23.982 INFO:teuthology.orchestra.run.smithi045.stdout:2072 read 0x2e17ddb thru 0x2e27966 (0xfb8c bytes) 2024-09-17T07:29:23.988 INFO:teuthology.orchestra.run.smithi045.stdout:2073 trunc from 0x644b84e to 0x1079a8e 2024-09-17T07:29:23.993 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:23.992+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:24.003 INFO:teuthology.orchestra.run.smithi045.stdout:2074 write 0x107fa71 thru 0x1089b0e (0xa09e bytes) 2024-09-17T07:29:24.016 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:24.016 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:24.019 INFO:teuthology.orchestra.run.smithi045.stdout:2075 write 0x8c679d9 thru 0x8c70250 (0x8878 bytes) 2024-09-17T07:29:24.022 INFO:teuthology.orchestra.run.smithi045.stdout:2076 read 0x578666c thru 0x5789cc9 (0x365e bytes) 2024-09-17T07:29:24.022 INFO:teuthology.orchestra.run.smithi045.stdout:2077 read 0xe25d9a thru 0xe26e4d (0x10b4 bytes) 2024-09-17T07:29:24.023 INFO:teuthology.orchestra.run.smithi045.stdout:2080 punch from 0x76d5a4b to 0x76e098b, (0xaf40 bytes) 2024-09-17T07:29:24.023 INFO:teuthology.orchestra.run.smithi045.stdout:2081 writesame 0x1be68f5 thru 0x1bea013 data_size 0x1(0x371f bytes) 2024-09-17T07:29:24.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:24.030 INFO:teuthology.orchestra.run.smithi045.stdout:2082 write 0x88dda0f thru 0x88e0f37 (0x3529 bytes) 2024-09-17T07:29:24.039 INFO:teuthology.orchestra.run.smithi045.stdout:2084 trunc from 0x8c70251 to 0x38d3d91 2024-09-17T07:29:24.055 INFO:teuthology.orchestra.run.smithi045.stdout:2085 write 0x5c45b55 thru 0x5c50ee2 (0xb38e bytes) 2024-09-17T07:29:24.058 INFO:teuthology.orchestra.run.smithi045.stdout:2087 read 0x1240631 thru 0x12459c2 (0x5392 bytes) 2024-09-17T07:29:24.058 INFO:teuthology.orchestra.run.smithi045.stdout:2089 read 0x2ba19e4 thru 0x2baee0a (0xd427 bytes) 2024-09-17T07:29:24.058 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:24.059 INFO:teuthology.orchestra.run.smithi045.stdout:2091 write 0x3380ac3 thru 0x33843dc (0x391a bytes) 2024-09-17T07:29:24.066 INFO:teuthology.orchestra.run.smithi045.stdout:2092 write 0x66f605f thru 0x670422c (0xe1ce bytes) 2024-09-17T07:29:24.079 INFO:teuthology.orchestra.run.smithi045.stdout:2095 writesame 0xbaaecbb thru 0xbaaf625 data_size 0x1(0x96b bytes) 2024-09-17T07:29:24.079 INFO:teuthology.orchestra.run.smithi045.stdout:2096 write 0x9bcd928 thru 0x9bce246 (0x91f bytes) 2024-09-17T07:29:24.081 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:24.081 INFO:teuthology.orchestra.run.smithi045.stdout:2100 punch from 0x7945aa to 0x7a35da, (0xf030 bytes) 2024-09-17T07:29:24.082 INFO:teuthology.orchestra.run.smithi045.stdout:2101 write 0x5c10123 thru 0x5c1aafc (0xa9da bytes) 2024-09-17T07:29:24.083 INFO:teuthology.orchestra.run.smithi045.stdout:2102 read 0xb6bf84f thru 0xb6c5346 (0x5af8 bytes) 2024-09-17T07:29:24.083 INFO:teuthology.orchestra.run.smithi045.stdout:2104 clone 22 order 19 su 65536 sc 5 2024-09-17T07:29:24.094 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:24.094+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:24.141 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:24.142 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:24.150 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:24.198 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:24.196+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:24.250 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T07:29:24.298 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:24.297+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:24.307 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:24.307 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:24.317 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:24.330 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone21 from 0xbaaf626 (overlap 0x525622) to 0x0 2024-09-17T07:29:24.402 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:24.400+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:24.433 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-17T07:29:24.441 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:24.441 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:24.455 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:24.531 INFO:teuthology.orchestra.run.smithi045.stdout:2105 write 0x8e130f6 thru 0x8e2189d (0xe7a8 bytes) 2024-09-17T07:29:24.531 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:24.531+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:24.546 INFO:teuthology.orchestra.run.smithi045.stdout:2107 trunc from 0xbaaf626 to 0x5673ec7 2024-09-17T07:29:24.565 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:24.565 INFO:teuthology.orchestra.run.smithi045.stdout:2109 read 0x2587916 thru 0x258cd67 (0x5452 bytes) 2024-09-17T07:29:24.571 INFO:teuthology.orchestra.run.smithi045.stdout:2110 trunc from 0x5673ec7 to 0x32402d 2024-09-17T07:29:24.592 INFO:teuthology.orchestra.run.smithi045.stdout:2112 write 0x28f938 thru 0x29bca8 (0xc371 bytes) 2024-09-17T07:29:24.593 INFO:teuthology.orchestra.run.smithi045.stdout:2113 punch from 0x29f751 to 0x2a4866, (0x5115 bytes) 2024-09-17T07:29:24.606 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:24.605+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:24.610 INFO:teuthology.orchestra.run.smithi045.stdout:2117 trunc from 0x32402d to 0xec43944 2024-09-17T07:29:24.614 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:24.614 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:24.615 INFO:teuthology.orchestra.run.smithi045.stdout:2118 write 0xd4cca17 thru 0xd4d0dd9 (0x43c3 bytes) 2024-09-17T07:29:24.621 INFO:teuthology.orchestra.run.smithi045.stdout:2119 write 0xa7b8101 thru 0xa7bcb60 (0x4a60 bytes) 2024-09-17T07:29:24.624 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:24.624 INFO:teuthology.orchestra.run.smithi045.stdout:2122 write 0x8f4221 thru 0x8fde6c (0x9c4c bytes) 2024-09-17T07:29:24.627 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:24.627 INFO:teuthology.orchestra.run.smithi045.stdout:2124 write 0xaa24fe3 thru 0xaa30229 (0xb247 bytes) 2024-09-17T07:29:24.632 INFO:teuthology.orchestra.run.smithi045.stdout:2125 read 0xbffbc1e thru 0xc0063e0 (0xa7c3 bytes) 2024-09-17T07:29:24.632 INFO:teuthology.orchestra.run.smithi045.stdout:2129 writesame 0x27bc3f thru 0x27da7b data_size 0x1(0x1e3d bytes) 2024-09-17T07:29:24.638 INFO:teuthology.orchestra.run.smithi045.stdout:2130 read 0x75f32f7 thru 0x7600bc2 (0xd8cc bytes) 2024-09-17T07:29:24.645 INFO:teuthology.orchestra.run.smithi045.stdout:2131 trunc from 0xec43944 to 0x8e2432f 2024-09-17T07:29:24.655 INFO:teuthology.orchestra.run.smithi045.stdout:2132 read 0x17aadf5 thru 0x17b1879 (0x6a85 bytes) 2024-09-17T07:29:24.663 INFO:teuthology.orchestra.run.smithi045.stdout:2133 write 0xbd6ac81 thru 0xbd725b9 (0x7939 bytes) 2024-09-17T07:29:24.667 INFO:teuthology.orchestra.run.smithi045.stdout:2135 writesame 0x94b5522 thru 0x94bd94f data_size 0x7(0x842e bytes) 2024-09-17T07:29:24.671 INFO:teuthology.orchestra.run.smithi045.stdout:2136 read 0xb509455 thru 0xb512e73 (0x9a1f bytes) 2024-09-17T07:29:24.671 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:24.671 INFO:teuthology.orchestra.run.smithi045.stdout:2138 read 0x47816d4 thru 0x4784bf3 (0x3520 bytes) 2024-09-17T07:29:24.680 INFO:teuthology.orchestra.run.smithi045.stdout:2140 write 0xeb3b180 thru 0xeb47f0b (0xcd8c bytes) 2024-09-17T07:29:24.683 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:24.684 INFO:teuthology.orchestra.run.smithi045.stdout:2142 writesame 0x60338bd thru 0x603b602 data_size 0xf(0x7d46 bytes) 2024-09-17T07:29:24.687 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:24.693 INFO:teuthology.orchestra.run.smithi045.stdout:2144 trunc from 0xeb47f0c to 0x9794374 2024-09-17T07:29:24.702 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:24.703 INFO:teuthology.orchestra.run.smithi045.stdout:2147 trunc from 0x9794374 to 0xac8b52b 2024-09-17T07:29:24.706 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:24.705+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:24.709 INFO:teuthology.orchestra.run.smithi045.stdout:2148 writesame 0xa1441f9 thru 0xa14b836 data_size 0xf(0x763e bytes) 2024-09-17T07:29:24.712 INFO:teuthology.orchestra.run.smithi045.stdout:2149 read 0x2c32852 thru 0x2c42563 (0xfd12 bytes) 2024-09-17T07:29:24.713 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:24.723 INFO:teuthology.orchestra.run.smithi045.stdout:2151 trunc from 0xac8b52b to 0x1de3545 2024-09-17T07:29:24.732 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:24.732 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:24.732 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:24.732 INFO:teuthology.orchestra.run.smithi045.stdout:2157 read 0x13c098e thru 0x13cb740 (0xadb3 bytes) 2024-09-17T07:29:24.732 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:24.732 INFO:teuthology.orchestra.run.smithi045.stdout:2159 read 0x3fae60 thru 0x3fbb0a (0xcab bytes) 2024-09-17T07:29:24.736 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:24.736 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:24.745 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:24.747 INFO:teuthology.orchestra.run.smithi045.stdout:2160 write 0xa908351 thru 0xa91309f (0xad4f bytes) 2024-09-17T07:29:24.754 INFO:teuthology.orchestra.run.smithi045.stdout:2161 writesame 0x6784bce thru 0x678d872 data_size 0x13(0x8ca5 bytes) 2024-09-17T07:29:24.767 INFO:teuthology.orchestra.run.smithi045.stdout:2162 write 0xd76579c thru 0xd76a9e0 (0x5245 bytes) 2024-09-17T07:29:24.771 INFO:teuthology.orchestra.run.smithi045.stdout:2163 write 0x69d1f4e thru 0x69d3a33 (0x1ae6 bytes) 2024-09-17T07:29:24.788 INFO:teuthology.orchestra.run.smithi045.stdout:2164 trunc from 0xd76a9e1 to 0x20ddf3 2024-09-17T07:29:24.810 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:24.810+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:24.812 INFO:teuthology.orchestra.run.smithi045.stdout:2167 punch from 0x1026b6 to 0x105833, (0x317d bytes) 2024-09-17T07:29:24.821 INFO:teuthology.orchestra.run.smithi045.stdout:2168 writesame 0x2f45ce4 thru 0x2f48413 data_size 0x30(0x2730 bytes) 2024-09-17T07:29:24.831 INFO:teuthology.orchestra.run.smithi045.stdout:2169 writesame 0x74bb3f8 thru 0x74c7c2d data_size 0x31(0xc836 bytes) 2024-09-17T07:29:24.841 INFO:teuthology.orchestra.run.smithi045.stdout:2170 write 0xb88b329 thru 0xb88c1b3 (0xe8b bytes) 2024-09-17T07:29:24.845 INFO:teuthology.orchestra.run.smithi045.stdout:2171 read 0x28ff4b9 thru 0x290487e (0x53c6 bytes) 2024-09-17T07:29:24.845 INFO:teuthology.orchestra.run.smithi045.stdout:2172 write 0xa821736 thru 0xa827614 (0x5edf bytes) 2024-09-17T07:29:24.859 INFO:teuthology.orchestra.run.smithi045.stdout:2173 trunc from 0xb88c1b4 to 0x3a209cd 2024-09-17T07:29:24.877 INFO:teuthology.orchestra.run.smithi045.stdout:2174 write 0x3b26689 thru 0x3b28559 (0x1ed1 bytes) 2024-09-17T07:29:24.884 INFO:teuthology.orchestra.run.smithi045.stdout:2175 trunc from 0x3b2855a to 0x7d3a4a0 2024-09-17T07:29:24.890 INFO:teuthology.orchestra.run.smithi045.stdout:2176 read 0x3301eba thru 0x3307df9 (0x5f40 bytes) 2024-09-17T07:29:24.890 INFO:teuthology.orchestra.run.smithi045.stdout:2179 write 0x304041d thru 0x304d37a (0xcf5e bytes) 2024-09-17T07:29:24.892 INFO:teuthology.orchestra.run.smithi045.stdout:2180 writesame 0x6d862bc thru 0x6d95680 data_size 0x7(0xf3c5 bytes) 2024-09-17T07:29:24.897 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:24.897 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:24.900 INFO:teuthology.orchestra.run.smithi045.stdout:2181 writesame 0x7b0fa65 thru 0x7b1a131 data_size 0x1(0xa6cd bytes) 2024-09-17T07:29:24.906 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:24.924 INFO:teuthology.orchestra.run.smithi045.stdout:2182 read 0x71103c6 thru 0x7113adc (0x3717 bytes) 2024-09-17T07:29:24.924 INFO:teuthology.orchestra.run.smithi045.stdout:2183 punch from 0x3b9f67d to 0x3ba35f0, (0x3f73 bytes) 2024-09-17T07:29:24.924 INFO:teuthology.orchestra.run.smithi045.stdout:2184 write 0x2aa47ba thru 0x2aa57ca (0x1011 bytes) 2024-09-17T07:29:24.936 INFO:teuthology.orchestra.run.smithi045.stdout:2185 write 0xb5cb3f6 thru 0xb5d6201 (0xae0c bytes) 2024-09-17T07:29:24.946 INFO:teuthology.orchestra.run.smithi045.stdout:2187 write 0xcb2992f thru 0xcb2b7ee (0x1ec0 bytes) 2024-09-17T07:29:24.949 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:24.948+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:24.950 INFO:teuthology.orchestra.run.smithi045.stdout:2190 read 0xc633c73 thru 0xc63601e (0x23ac bytes) 2024-09-17T07:29:24.950 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:24.950 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:24.956 INFO:teuthology.orchestra.run.smithi045.stdout:2194 writesame 0xe285e40 thru 0xe28fb24 data_size 0x1d(0x9ce5 bytes) 2024-09-17T07:29:24.956 INFO:teuthology.orchestra.run.smithi045.stdout:2195 punch from 0x963532d to 0x963b12f, (0x5e02 bytes) 2024-09-17T07:29:24.956 INFO:teuthology.orchestra.run.smithi045.stdout:2197 write 0x6bb6e66 thru 0x6bbaa8e (0x3c29 bytes) 2024-09-17T07:29:24.957 INFO:teuthology.orchestra.run.smithi045.stdout:2199 write 0x4cee0a9 thru 0x4cf544d (0x73a5 bytes) 2024-09-17T07:29:24.961 INFO:teuthology.orchestra.run.smithi045.stdout:2200 punch from 0x536a623 to 0x5373e4d, (0x982a bytes) 2024-09-17T07:29:24.961 INFO:teuthology.orchestra.run.smithi045.stdout:2201 write 0x89d66e1 thru 0x89e4c55 (0xe575 bytes) 2024-09-17T07:29:24.964 INFO:teuthology.orchestra.run.smithi045.stdout:2202 read 0xdb79b5d thru 0xdb7f420 (0x58c4 bytes) 2024-09-17T07:29:24.964 INFO:teuthology.orchestra.run.smithi045.stdout:2204 read 0x5d46858 thru 0x5d48c10 (0x23b9 bytes) 2024-09-17T07:29:24.973 INFO:teuthology.orchestra.run.smithi045.stdout:2206 trunc from 0xe28fb25 to 0x67d645e 2024-09-17T07:29:24.993 INFO:teuthology.orchestra.run.smithi045.stdout:2207 write 0x88d12ed thru 0x88da1f2 (0x8f06 bytes) 2024-09-17T07:29:25.002 INFO:teuthology.orchestra.run.smithi045.stdout:2208 writesame 0x97a9355 thru 0x97b1179 data_size 0x1(0x7e25 bytes) 2024-09-17T07:29:25.003 INFO:teuthology.orchestra.run.smithi045.stdout:2209 read 0x446350b thru 0x4467229 (0x3d1f bytes) 2024-09-17T07:29:25.003 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:25.003 INFO:teuthology.orchestra.run.smithi045.stdout:2213 write 0x68de6d7 thru 0x68e59cd (0x72f7 bytes) 2024-09-17T07:29:25.006 INFO:teuthology.orchestra.run.smithi045.stdout:2214 punch from 0x56e770d to 0x56f6fad, (0xf8a0 bytes) 2024-09-17T07:29:25.006 INFO:teuthology.orchestra.run.smithi045.stdout:2216 write 0x979d808 thru 0x97a4d66 (0x755f bytes) 2024-09-17T07:29:25.010 INFO:teuthology.orchestra.run.smithi045.stdout:2217 read 0x180780e thru 0x180d9fa (0x61ed bytes) 2024-09-17T07:29:25.010 INFO:teuthology.orchestra.run.smithi045.stdout:2218 punch from 0xe4ec55 to 0xe57a07, (0x8db2 bytes) 2024-09-17T07:29:25.010 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:25.011 INFO:teuthology.orchestra.run.smithi045.stdout:2220 writesame 0x606f61b thru 0x606f9ef data_size 0x3(0x3d5 bytes) 2024-09-17T07:29:25.013 INFO:teuthology.orchestra.run.smithi045.stdout:2221 read 0x10c55bf thru 0x10c7160 (0x1ba2 bytes) 2024-09-17T07:29:25.013 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:25.011+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:25.015 INFO:teuthology.orchestra.run.smithi045.stdout:2222 trunc from 0x97b117a to 0xbae093a 2024-09-17T07:29:25.019 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:25.020 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:25.020 INFO:teuthology.orchestra.run.smithi045.stdout:2224 writesame 0x9044b92 thru 0x9044feb data_size 0x1(0x45a bytes) 2024-09-17T07:29:25.023 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:25.023 INFO:teuthology.orchestra.run.smithi045.stdout:2228 write 0x5867901 thru 0x586a49b (0x2b9b bytes) 2024-09-17T07:29:25.026 INFO:teuthology.orchestra.run.smithi045.stdout:2229 write 0x7779a06 thru 0x777dfaa (0x45a5 bytes) 2024-09-17T07:29:25.029 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:25.030 INFO:teuthology.orchestra.run.smithi045.stdout:2230 read 0x85b796b thru 0x85c4853 (0xcee9 bytes) 2024-09-17T07:29:25.030 INFO:teuthology.orchestra.run.smithi045.stdout:2232 read 0x4cdb9ea thru 0x4ce8f3f (0xd556 bytes) 2024-09-17T07:29:25.031 INFO:teuthology.orchestra.run.smithi045.stdout:2235 read 0x72a41b3 thru 0x72a5618 (0x1466 bytes) 2024-09-17T07:29:25.031 INFO:teuthology.orchestra.run.smithi045.stdout:2236 write 0x7b669b9 thru 0x7b6d1bb (0x6803 bytes) 2024-09-17T07:29:25.033 INFO:teuthology.orchestra.run.smithi045.stdout:2237 read 0x63c72c6 thru 0x63d3acf (0xc80a bytes) 2024-09-17T07:29:25.033 INFO:teuthology.orchestra.run.smithi045.stdout:2238 writesame 0x8bf593d thru 0x8c0101c data_size 0x9a(0xb6e0 bytes) 2024-09-17T07:29:25.036 INFO:teuthology.orchestra.run.smithi045.stdout:2239 read 0xba45788 thru 0xba4d687 (0x7f00 bytes) 2024-09-17T07:29:25.037 INFO:teuthology.orchestra.run.smithi045.stdout:2241 trunc from 0xbae093a to 0xb312eab 2024-09-17T07:29:25.042 INFO:teuthology.orchestra.run.smithi045.stdout:2242 writesame 0x7199cad thru 0x719ee62 data_size 0x1(0x51b6 bytes) 2024-09-17T07:29:25.054 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:25.055 INFO:teuthology.orchestra.run.smithi045.stdout:2245 read 0x41eb390 thru 0x41f0a19 (0x568a bytes) 2024-09-17T07:29:25.055 INFO:teuthology.orchestra.run.smithi045.stdout:2246 read 0x40958c8 thru 0x4098a06 (0x313f bytes) 2024-09-17T07:29:25.055 INFO:teuthology.orchestra.run.smithi045.stdout:2247 writesame 0x7837d22 thru 0x783ad9a data_size 0x1(0x3079 bytes) 2024-09-17T07:29:25.060 INFO:teuthology.orchestra.run.smithi045.stdout:2248 write 0x11e5a48 thru 0x11eca7f (0x7038 bytes) 2024-09-17T07:29:25.064 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:25.064 INFO:teuthology.orchestra.run.smithi045.stdout:2250 writesame 0x205465a thru 0x2063f36 data_size 0x1(0xf8dd bytes) 2024-09-17T07:29:25.086 INFO:teuthology.orchestra.run.smithi045.stdout:2252 write 0x9ac7f70 thru 0x9ad787b (0xf90c bytes) 2024-09-17T07:29:25.100 INFO:teuthology.orchestra.run.smithi045.stdout:2253 write 0xd52dc26 thru 0xd53013c (0x2517 bytes) 2024-09-17T07:29:25.109 INFO:teuthology.orchestra.run.smithi045.stdout:2254 write 0xb76f713 thru 0xb7799da (0xa2c8 bytes) 2024-09-17T07:29:25.114 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:25.113+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:25.126 INFO:teuthology.orchestra.run.smithi045.stdout:2255 trunc from 0xd53013d to 0x973adfa 2024-09-17T07:29:25.149 INFO:teuthology.orchestra.run.smithi045.stdout:2256 writesame 0xeb26b3b thru 0xeb31edb data_size 0x11(0xb3a1 bytes) 2024-09-17T07:29:25.149 INFO:teuthology.orchestra.run.smithi045.stdout:2259 write 0x129193f thru 0x1293994 (0x2056 bytes) 2024-09-17T07:29:25.152 INFO:teuthology.orchestra.run.smithi045.stdout:2261 write 0xa56b24c thru 0xa576388 (0xb13d bytes) 2024-09-17T07:29:25.156 INFO:teuthology.orchestra.run.smithi045.stdout:2264 punch from 0xa4c0ad7 to 0xa4ca82d, (0x9d56 bytes) 2024-09-17T07:29:25.156 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:25.156 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:25.156 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:25.157 INFO:teuthology.orchestra.run.smithi045.stdout:2270 read 0x59d4261 thru 0x59d89cb (0x476b bytes) 2024-09-17T07:29:25.157 INFO:teuthology.orchestra.run.smithi045.stdout:2273 write 0x8a9372 thru 0x8af826 (0x64b5 bytes) 2024-09-17T07:29:25.159 INFO:teuthology.orchestra.run.smithi045.stdout:2275 read 0x3083fd6 thru 0x3092560 (0xe58b bytes) 2024-09-17T07:29:25.159 INFO:teuthology.orchestra.run.smithi045.stdout:2277 write 0xb189c6 thru 0xb2671f (0xdd5a bytes) 2024-09-17T07:29:25.163 INFO:teuthology.orchestra.run.smithi045.stdout:2279 punch from 0xd923528 to 0xd927d21, (0x47f9 bytes) 2024-09-17T07:29:25.163 INFO:teuthology.orchestra.run.smithi045.stdout:2280 trunc from 0xeb31edc to 0xe56b74e 2024-09-17T07:29:25.175 INFO:teuthology.orchestra.run.smithi045.stdout:2282 trunc from 0xe56b74e to 0x85f2c68 2024-09-17T07:29:25.184 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:25.184 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:25.187 INFO:teuthology.orchestra.run.smithi045.stdout:2284 write 0x353e13d thru 0x3548184 (0xa048 bytes) 2024-09-17T07:29:25.191 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:25.191 INFO:teuthology.orchestra.run.smithi045.stdout:2286 writesame 0x56ddf32 thru 0x56e2802 data_size 0x1(0x48d1 bytes) 2024-09-17T07:29:25.194 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:25.210 INFO:teuthology.orchestra.run.smithi045.stdout:2287 write 0xd225ce8 thru 0xd22bbf4 (0x5f0d bytes) 2024-09-17T07:29:25.228 INFO:teuthology.orchestra.run.smithi045.stdout:2288 trunc from 0xd22bbf5 to 0x521f9e 2024-09-17T07:29:25.238 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:25.237+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:25.241 INFO:teuthology.orchestra.run.smithi045.stdout:2289 write 0x35b25 thru 0x3a304 (0x47e0 bytes) 2024-09-17T07:29:25.250 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-17T07:29:25.250 INFO:teuthology.orchestra.run.smithi045.stdout:2291 write 0x1755802 thru 0x17576d8 (0x1ed7 bytes) 2024-09-17T07:29:25.254 INFO:teuthology.orchestra.run.smithi045.stdout:2292 read 0x4b55a thru 0x5919e (0xdc45 bytes) 2024-09-17T07:29:25.255 INFO:teuthology.orchestra.run.smithi045.stdout:2293 punch from 0x347609 to 0x349c45, (0x263c bytes) 2024-09-17T07:29:25.255 INFO:teuthology.orchestra.run.smithi045.stdout:2294 punch from 0x151cc1f to 0x151e7d1, (0x1bb2 bytes) 2024-09-17T07:29:25.255 INFO:teuthology.orchestra.run.smithi045.stdout:2295 punch from 0x4900a1 to 0x499c7c, (0x9bdb bytes) 2024-09-17T07:29:25.255 INFO:teuthology.orchestra.run.smithi045.stdout:2296 read 0x124a542 thru 0x1258fd2 (0xea91 bytes) 2024-09-17T07:29:25.274 INFO:teuthology.orchestra.run.smithi045.stdout:2300 write 0xd4fd5a3 thru 0xd50308c (0x5aea bytes) 2024-09-17T07:29:25.279 INFO:teuthology.orchestra.run.smithi045.stdout:2302 write 0x3330da0 thru 0x33331c7 (0x2428 bytes) 2024-09-17T07:29:25.281 INFO:teuthology.orchestra.run.smithi045.stdout:2303 write 0x4caab59 thru 0x4cb994f (0xedf7 bytes) 2024-09-17T07:29:25.296 INFO:teuthology.orchestra.run.smithi045.stdout:2304 trunc from 0xd50308d to 0x43e530f 2024-09-17T07:29:25.305 INFO:teuthology.orchestra.run.smithi045.stdout:2306 punch from 0x17a8d1d to 0x17b7739, (0xea1c bytes) 2024-09-17T07:29:25.305 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:25.307 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:25.307 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:25.316 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:25.316 INFO:teuthology.orchestra.run.smithi045.stdout:2310 write 0x8e8fcc5 thru 0x8e97fae (0x82ea bytes) 2024-09-17T07:29:25.320 INFO:teuthology.orchestra.run.smithi045.stdout:2311 writesame 0x499f653 thru 0x49a3353 data_size 0x17(0x3d01 bytes) 2024-09-17T07:29:25.326 INFO:teuthology.orchestra.run.smithi045.stdout:2312 clone 23 order 23 su 65536 sc 13 2024-09-17T07:29:25.357 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:25.355+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:25.422 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:25.420+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:25.459 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:25.459 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:25.467 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:25.523 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:25.522+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:25.611 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:25.611 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:25.619 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:25.623 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:25.623+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:25.721 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:25.721 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:25.725 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:25.724+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:25.729 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:25.829 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:25.824+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:25.832 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:25.832 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:25.840 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:25.915 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:29:25.926 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:25.925+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:25.984 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:25.984 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:25.992 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:26.027 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:26.026+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:26.049 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:26.049 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:26.066 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:26.066 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:26.082 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:26.100 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:26.100 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:26.116 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:26.117 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:26.128 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:26.126+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:26.133 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:26.133 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:26.137 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:26.137 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:26.146 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:26.150 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:26.151 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:26.154 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:26.154 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:26.230 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:26.229+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:26.296 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:29:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.960456309999998, 2024-09-17T07:29:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:29:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.297 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:29:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:29:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:29:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:29:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:29:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:29:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:29:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:29:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:29:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:29:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:29:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:29:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:29:26.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:29:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:29:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:29:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:29:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:29:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:29:26.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:29:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:29:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:29:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:29:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:29:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:29:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:29:26.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:29:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:29:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:29:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:29:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:29:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7378 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7176192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.371414275999999, 2024-09-17T07:29:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073003980999999996, 2024-09-17T07:29:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7378 2024-09-17T07:29:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.304 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1799999999999999e-07 2024-09-17T07:29:26.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923713+0000", 2024-09-17T07:29:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.057e-06 2024-09-17T07:29:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923716+0000", 2024-09-17T07:29:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0599999999999999e-06 2024-09-17T07:29:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923719+0000", 2024-09-17T07:29:26.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8219999999999998e-06 2024-09-17T07:29:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996401+0000", 2024-09-17T07:29:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072681467999999999 2024-09-17T07:29:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996452+0000", 2024-09-17T07:29:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.134e-05 2024-09-17T07:29:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996716+0000", 2024-09-17T07:29:26.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000264016 2024-09-17T07:29:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.371404160000001, 2024-09-17T07:29:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:29:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:29:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:29:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:29:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:29:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:29:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:29:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:29:26.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:29:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:29:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:29:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:29:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:29:26.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:29:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:29:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:29:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7398 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6389760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.366135625999998, 2024-09-17T07:29:26.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072891580999999997, 2024-09-17T07:29:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7398 2024-09-17T07:29:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928995+0000", 2024-09-17T07:29:26.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0759999999999996e-06 2024-09-17T07:29:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928999+0000", 2024-09-17T07:29:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1609999999999996e-06 2024-09-17T07:29:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929002+0000", 2024-09-17T07:29:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8449999999999999e-06 2024-09-17T07:29:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001598+0000", 2024-09-17T07:29:26.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072596318000000007 2024-09-17T07:29:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001663+0000", 2024-09-17T07:29:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4995999999999999e-05 2024-09-17T07:29:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001882+0000", 2024-09-17T07:29:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021897299999999999 2024-09-17T07:29:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.366121044, 2024-09-17T07:29:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:29:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.317 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:29:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:29:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:29:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:29:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:29:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:29:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:29:26.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:29:26.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:29:26.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:29:26.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:29:26.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:29:26.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:29:26.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:29:26.321 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.321 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.321 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.321 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.321 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:26.321 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.366090064000002, 2024-09-17T07:29:26.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:29:26.321 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.322 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.322 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.322 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.322 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.322 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.322 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:29:26.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.322 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:26.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:26.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:29:26.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:29:26.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:29:26.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:29:26.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:29:26.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:29:26.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:29:26.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:29:26.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:29:26.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:29:26.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:29:26.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:29:26.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:29:26.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:29:26.325 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.326 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.326 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.326 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.326 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:26.326 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.366060014999999, 2024-09-17T07:29:26.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:29:26.326 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.326 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.326 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.326 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.326 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.326 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.327 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:29:26.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.327 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:26.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:26.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:29:26.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:29:26.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:29:26.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:29:26.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:29:26.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:29:26.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:29:26.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:29:26.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:29:26.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:29:26.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:29:26.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:29:26.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:29:26.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:29:26.330 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.330 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.330 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.330 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.330 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:26.331 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.366031608, 2024-09-17T07:29:26.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:29:26.331 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.331 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.331 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.331 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.331 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.331 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.331 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:29:26.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.331 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:26.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:26.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:26.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:26.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:29:26.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:29:26.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:29:26.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:29:26.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:29:26.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:29:26.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:29:26.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:29:26.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:29:26.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:29:26.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:29:26.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:29:26.334 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.335 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.335 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.335 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.335 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:26.335 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.366011787000001, 2024-09-17T07:29:26.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:29:26.335 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.335 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.335 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.335 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:29:26.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.336 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:26.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:26.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:29:26.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:29:26.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:29:26.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:29:26.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:29:26.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:29:26.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:29:26.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:29:26.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:29:26.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:29:26.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:29:26.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:29:26.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:29:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:29:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.365987534999999, 2024-09-17T07:29:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:29:26.339 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:29:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:26.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:29:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:29:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:29:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:29:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:29:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:29:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:29:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:29:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:29:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:29:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:29:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:29:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:29:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:29:26.343 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7418 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7700480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.360581512, 2024-09-17T07:29:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072752965000000003, 2024-09-17T07:29:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.344 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7418 2024-09-17T07:29:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3599999999999999e-07 2024-09-17T07:29:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934547+0000", 2024-09-17T07:29:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3350000000000001e-06 2024-09-17T07:29:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934554+0000", 2024-09-17T07:29:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9859999999999997e-06 2024-09-17T07:29:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934557+0000", 2024-09-17T07:29:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.818e-06 2024-09-17T07:29:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007104+0000", 2024-09-17T07:29:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072546779000000006 2024-09-17T07:29:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007147+0000", 2024-09-17T07:29:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2904000000000001e-05 2024-09-17T07:29:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007298+0000", 2024-09-17T07:29:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015080700000000001 2024-09-17T07:29:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.360564947, 2024-09-17T07:29:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:29:26.348 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:29:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:29:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:29:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:29:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:29:26.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:29:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:29:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:29:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:29:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:29:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:29:26.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:29:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:29:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:29:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:29:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.360518072000001, 2024-09-17T07:29:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:29:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:29:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:29:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:29:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:29:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:29:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:29:26.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:29:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:29:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:29:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:29:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:29:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:29:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:29:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.360503368, 2024-09-17T07:29:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:29:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.357 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:29:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:26.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:29:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:29:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:29:26.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:29:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:29:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:29:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:29:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:29:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:29:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:29:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:29:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:29:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.361 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.360480273, 2024-09-17T07:29:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:29:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:29:26.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:26.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:29:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:29:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:29:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:29:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:29:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:29:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:29:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:29:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:29:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:29:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:29:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:29:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:29:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.366 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.360448898000001, 2024-09-17T07:29:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:29:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:29:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:26.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:29:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:29:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:29:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:29:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:29:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:29:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:29:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:29:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:29:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:29:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:29:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:29:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:29:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:29:26.370 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.360433638, 2024-09-17T07:29:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:29:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:29:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:29:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:29:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:29:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:29:26.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:29:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:29:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:29:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:29:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:29:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:29:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:29:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:29:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:29:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.360413567999998, 2024-09-17T07:29:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:29:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:29:26.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:29:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:29:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:29:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:29:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:29:26.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:29:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:29:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:29:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:29:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:29:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:29:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:29:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:29:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.360401627999998, 2024-09-17T07:29:26.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:29:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:29:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:26.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:29:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:29:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:29:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:29:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:29:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:29:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:29:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:29:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:29:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:29:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:29:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:29:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.384 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.360378470000001, 2024-09-17T07:29:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:29:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:29:26.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:29:26.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:29:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:29:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:29:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:29:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:29:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:29:26.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:26.389 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:26.389 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:26.390 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone22 from 0x8e97faf (overlap 0x20ddf3) to 0x9dc2f 2024-09-17T07:29:26.390 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:26.389+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:26.391 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:26.433 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-17T07:29:26.434 INFO:teuthology.orchestra.run.smithi045.stdout:2313 read 0x8e0186d thru 0x8e0ddbc (0xc550 bytes) 2024-09-17T07:29:26.435 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:26.435 INFO:teuthology.orchestra.run.smithi045.stdout:2315 read 0x6ec9aa4 thru 0x6ecd8e7 (0x3e44 bytes) 2024-09-17T07:29:26.435 INFO:teuthology.orchestra.run.smithi045.stdout:2316 punch from 0x44974a6 to 0x4499528, (0x2082 bytes) 2024-09-17T07:29:26.438 INFO:teuthology.orchestra.run.smithi045.stdout:2317 read 0x8cee991 thru 0x8cfd2b3 (0xe923 bytes) 2024-09-17T07:29:26.438 INFO:teuthology.orchestra.run.smithi045.stdout:2318 read 0x28fc25d thru 0x28fe7c8 (0x256c bytes) 2024-09-17T07:29:26.445 INFO:teuthology.orchestra.run.smithi045.stdout:2319 trunc from 0x8e97faf to 0x2adedda 2024-09-17T07:29:26.466 INFO:teuthology.orchestra.run.smithi045.stdout:2320 read 0x33e23 thru 0x37f0f (0x40ed bytes) 2024-09-17T07:29:26.466 INFO:teuthology.orchestra.run.smithi045.stdout:2322 write 0xcb2efb thru 0xcb6439 (0x353f bytes) 2024-09-17T07:29:26.478 INFO:teuthology.orchestra.run.smithi045.stdout:2324 trunc from 0x2adedda to 0xc1d32cd 2024-09-17T07:29:26.483 INFO:teuthology.orchestra.run.smithi045.stdout:2325 read 0x1d6ca85 thru 0x1d783dd (0xb959 bytes) 2024-09-17T07:29:26.483 INFO:teuthology.orchestra.run.smithi045.stdout:2326 read 0x60f54c7 thru 0x61051e7 (0xfd21 bytes) 2024-09-17T07:29:26.484 INFO:teuthology.orchestra.run.smithi045.stdout:2329 read 0x78c5a0f thru 0x78d2dbe (0xd3b0 bytes) 2024-09-17T07:29:26.484 INFO:teuthology.orchestra.run.smithi045.stdout:2330 write 0x8a906d2 thru 0x8a97e6f (0x779e bytes) 2024-09-17T07:29:26.486 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:26.491 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:26.491+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:26.494 INFO:teuthology.orchestra.run.smithi045.stdout:2332 writesame 0xed449e0 thru 0xed4e018 data_size 0x1(0x9639 bytes) 2024-09-17T07:29:26.494 INFO:teuthology.orchestra.run.smithi045.stdout:2333 punch from 0x4a24992 to 0x4a29a78, (0x50e6 bytes) 2024-09-17T07:29:26.494 INFO:teuthology.orchestra.run.smithi045.stdout:2334 punch from 0xd75dbda to 0xd7686a2, (0xaac8 bytes) 2024-09-17T07:29:26.494 INFO:teuthology.orchestra.run.smithi045.stdout:2336 writesame 0x5c849ff thru 0x5c921e5 data_size 0x1(0xd7e7 bytes) 2024-09-17T07:29:26.522 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:56615 3.17 3:ea92f193:::rbd_data.118344704f8f.000000000000000b:head [stat out=16b,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-17T07:29:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:26.494912+0000", 2024-09-17T07:29:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.027264400000000001, 2024-09-17T07:29:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.027285331999999999, 2024-09-17T07:29:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "waiting for sub ops", 2024-09-17T07:29:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 56615 2024-09-17T07:29:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:26.494912+0000", 2024-09-17T07:29:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:26.494912+0000", 2024-09-17T07:29:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:26.494912+0000", 2024-09-17T07:29:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7199999999999999e-07 2024-09-17T07:29:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:26.494916+0000", 2024-09-17T07:29:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.501e-06 2024-09-17T07:29:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:26.494924+0000", 2024-09-17T07:29:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8149999999999995e-06 2024-09-17T07:29:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:26.494930+0000", 2024-09-17T07:29:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8969999999999996e-06 2024-09-17T07:29:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:26.494991+0000", 2024-09-17T07:29:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0195999999999998e-05 2024-09-17T07:29:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:26.495080+0000", 2024-09-17T07:29:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8988999999999998e-05 2024-09-17T07:29:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:29:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:26.500648+0000", 2024-09-17T07:29:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0055681469999999999 2024-09-17T07:29:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop_reply(client.4483.0:56615 3.17 e63/23)", 2024-09-17T07:29:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:26.511423+0000", 2024-09-17T07:29:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.010753247, 2024-09-17T07:29:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010815373, 2024-09-17T07:29:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:26.511423+0000", 2024-09-17T07:29:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:26.511423+0000", 2024-09-17T07:29:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:26.511424+0000", 2024-09-17T07:29:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.226e-06 2024-09-17T07:29:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:26.511428+0000", 2024-09-17T07:29:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8029999999999998e-06 2024-09-17T07:29:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:26.511441+0000", 2024-09-17T07:29:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3010000000000001e-05 2024-09-17T07:29:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:26.511451+0000", 2024-09-17T07:29:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084999999999999e-05 2024-09-17T07:29:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:29:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 2 2024-09-17T07:29:26.530 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:26.531 INFO:teuthology.orchestra.run.smithi045.stdout:2337 read 0xcb01a68 thru 0xcb0dafb (0xc094 bytes) 2024-09-17T07:29:26.531 INFO:teuthology.orchestra.run.smithi045.stdout:2338 write 0x73ac438 thru 0x73acb6b (0x734 bytes) 2024-09-17T07:29:26.531 INFO:teuthology.orchestra.run.smithi045.stdout:2340 read 0x5ad2453 thru 0x5ad693d (0x44eb bytes) 2024-09-17T07:29:26.531 INFO:teuthology.orchestra.run.smithi045.stdout:2341 write 0xa2d2fba thru 0xa2dfaf4 (0xcb3b bytes) 2024-09-17T07:29:26.531 INFO:teuthology.orchestra.run.smithi045.stdout:2343 read 0x639e945 thru 0x63a5eac (0x7568 bytes) 2024-09-17T07:29:26.531 INFO:teuthology.orchestra.run.smithi045.stdout:2344 clone 24 order 25 su 2097152 sc 5 2024-09-17T07:29:26.532 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:26.592 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:26.591+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:26.679 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:26.680 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:26.688 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:26.692 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:26.691+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:26.793 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:26.792+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:26.797 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.663281795, 2024-09-17T07:29:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:29:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:29:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:29:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:29:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:29:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:29:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:29:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:29:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:29:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:29:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:29:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:29:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:29:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:29:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:29:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:29:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:29:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:29:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:29:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:29:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:29:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:29:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:29:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:29:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:29:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.357315091, 2024-09-17T07:29:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:29:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:29:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:29:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:29:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:29:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:29:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:29:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:29:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:29:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:29:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:29:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:29:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:29:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:29:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:29:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:29:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.357292976, 2024-09-17T07:29:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:29:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:29:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:29:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:29:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:29:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:29:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:29:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:29:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:29:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:29:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:29:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:29:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:29:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:29:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.357268871999999, 2024-09-17T07:29:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:29:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:29:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:29:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:29:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:29:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:29:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:29:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:29:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:29:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:29:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:29:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:29:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:29:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:29:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.357244782999999, 2024-09-17T07:29:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:29:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:29:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:29:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:29:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:29:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:29:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:29:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:29:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:29:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:29:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:29:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:29:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:29:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.357220364, 2024-09-17T07:29:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:29:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:29:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:29:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:29:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:29:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:29:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:29:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:29:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:29:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:29:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:29:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:29:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:29:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.357198663999998, 2024-09-17T07:29:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:29:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:29:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:29:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:29:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:29:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:29:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:29:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:29:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:29:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:29:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:29:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:29:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:29:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.357153931999999, 2024-09-17T07:29:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:29:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:29:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:29:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:29:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:29:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:29:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:29:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:29:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:29:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:29:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:29:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:29:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:29:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:29:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.357125465999999, 2024-09-17T07:29:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:29:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:29:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:29:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:29:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:29:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:29:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:29:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:29:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:29:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:29:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:29:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:29:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:29:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.357008214, 2024-09-17T07:29:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:29:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:29:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:29:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:29:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:29:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:29:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:29:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:29:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:29:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:29:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:29:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:29:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:29:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:29:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.356582112000002, 2024-09-17T07:29:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:29:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:29:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:29:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:29:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:29:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:29:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:29:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:29:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:29:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:29:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:29:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:29:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:29:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.356562441000001, 2024-09-17T07:29:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:29:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:29:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:29:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:29:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:29:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:29:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:29:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:29:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:29:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:29:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:29:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:29:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:29:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.356539576999999, 2024-09-17T07:29:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:29:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:29:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:29:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:29:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:29:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:29:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:29:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:29:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:29:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:29:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:29:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:29:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:29:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.356518275999999, 2024-09-17T07:29:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:29:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:29:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:29:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:29:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:29:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:29:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:29:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:29:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:29:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:29:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:29:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:29:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:29:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:29:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.356470487999999, 2024-09-17T07:29:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:29:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:29:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:29:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:29:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:29:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:29:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:29:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:29:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:29:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:29:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:29:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:29:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:29:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:29:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.356443265999999, 2024-09-17T07:29:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:29:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:29:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:29:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:29:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:29:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:29:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:29:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:29:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:29:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:29:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:29:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:29:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:29:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.356422879, 2024-09-17T07:29:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:29:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:29:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:29:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:29:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:29:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:29:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:29:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:29:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:29:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:29:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:29:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:29:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:29:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:29:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.356403174, 2024-09-17T07:29:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:29:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:29:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:29:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:29:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:29:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:29:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:29:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:29:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:29:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:29:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:29:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:29:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:29:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:29:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.356380095999999, 2024-09-17T07:29:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:29:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:29:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:29:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:29:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:29:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:29:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:29:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:29:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:29:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:29:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:29:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:29:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:29:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:29:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.356359533999999, 2024-09-17T07:29:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:29:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:29:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:29:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:29:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:29:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:29:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:29:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:29:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:29:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:29:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:29:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:29:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:29:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:26.887 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:26.888 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:26.894 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:26.893+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:26.994 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:26.994+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:26.995 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:26.996 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:27.004 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:27.095 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:27.094+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:27.152 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:27.152 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:27.160 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:27.196 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:27.195+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:27.270 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T07:29:27.272 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.137269577000001, 2024-09-17T07:29:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:29:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:29:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:29:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:29:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:29:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:29:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:29:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:29:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:29:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:29:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:29:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:29:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:29:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:29:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:29:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:29:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:29:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:29:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:29:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:29:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:29:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:29:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:29:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:29:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.13723268, 2024-09-17T07:29:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:29:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:29:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:29:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:29:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:29:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:29:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:29:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:29:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:29:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:29:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:29:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:29:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:29:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:29:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:29:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:29:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:29:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:29:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:29:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:29:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:29:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:29:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:29:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.137206713000001, 2024-09-17T07:29:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:29:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:29:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:29:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:29:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:29:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:29:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:29:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:29:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:29:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:29:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:29:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:29:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:29:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:29:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:29:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:29:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:29:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:29:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:29:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:29:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:29:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:29:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:29:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.137182117999998, 2024-09-17T07:29:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:29:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:29:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:29:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:29:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:29:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:29:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:29:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:29:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:29:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:29:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:29:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:29:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:29:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:29:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:29:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:29:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:29:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:29:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:29:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:29:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:29:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:29:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:29:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.137158630999998, 2024-09-17T07:29:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:29:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:29:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:29:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:29:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:29:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:29:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:29:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:29:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:29:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:29:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:29:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:29:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:29:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:29:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:29:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:29:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:29:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:29:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:29:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:29:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:29:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:29:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.137144892999999, 2024-09-17T07:29:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:29:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:29:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:29:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:29:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:29:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:29:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:29:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:29:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:29:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:29:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:29:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:29:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:29:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:29:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:29:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:29:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.137114703999998, 2024-09-17T07:29:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:29:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:29:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:29:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:29:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:29:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:29:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:29:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:29:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:29:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:29:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:29:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:29:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:29:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:29:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:29:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:29:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:29:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:29:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:29:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:29:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:29:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:29:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.137078199000001, 2024-09-17T07:29:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:29:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:29:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:29:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:29:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:29:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:29:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:29:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:29:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:29:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:29:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:29:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:29:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:29:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:29:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:29:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:29:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:29:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:29:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:29:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:29:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:29:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:29:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.137050439999999, 2024-09-17T07:29:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:29:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:29:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:29:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:29:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:29:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:29:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:29:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:29:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:29:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:29:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:29:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:29:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:29:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:29:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:29:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:29:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:29:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:29:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:29:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:29:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:29:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:29:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.137028279999999, 2024-09-17T07:29:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:29:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:29:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:29:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:29:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:29:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:29:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:29:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:29:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:29:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:29:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:29:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:29:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:29:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:29:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:29:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:29:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:29:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:29:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:29:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:29:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:29:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:29:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.137015119000001, 2024-09-17T07:29:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:29:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:29:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:29:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:29:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:29:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:29:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:29:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:29:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:29:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:29:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:29:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:29:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:29:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:29:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:29:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:29:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:29:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:29:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:29:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:29:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:29:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:29:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.136996531000001, 2024-09-17T07:29:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:29:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:29:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:29:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:29:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:29:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:29:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:29:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:29:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:29:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:29:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:29:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:29:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:29:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:29:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:29:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:29:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:29:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:29:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:29:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:29:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.136963720000001, 2024-09-17T07:29:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:29:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:29:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:29:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:29:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:29:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:29:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:29:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:29:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:29:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:29:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:29:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:29:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:29:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:29:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:29:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:29:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:29:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:29:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:29:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:29:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.136944663000001, 2024-09-17T07:29:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:29:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:29:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:29:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:29:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:29:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:29:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:29:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:29:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:29:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:29:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:29:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:29:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:29:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:29:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:29:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:29:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:29:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:29:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:29:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:29:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:29:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:29:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:29:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:29:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.136922554000002, 2024-09-17T07:29:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:29:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:29:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:29:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:29:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:29:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:29:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:29:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:29:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:29:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:29:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:29:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:29:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:29:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:29:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:29:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:29:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:29:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:29:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:29:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:29:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:29:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:29:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:29:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.136896447000002, 2024-09-17T07:29:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:29:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:29:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:29:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:29:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:29:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:29:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:29:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:29:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:29:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:29:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:29:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:29:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:29:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:29:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:29:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:29:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:29:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:29:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:29:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:29:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:29:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:29:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:29:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.136867301999999, 2024-09-17T07:29:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:29:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:29:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:29:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:29:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:29:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:29:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:29:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:29:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:29:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:29:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:29:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:29:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:29:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:29:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:29:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:29:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:29:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:29:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:29:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:29:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:29:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:29:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.136835499, 2024-09-17T07:29:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:29:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:29:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:29:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:29:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:29:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:29:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:29:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:29:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:29:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:29:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:29:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:29:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:29:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:29:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:29:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:29:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:29:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:29:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:29:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:29:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:29:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:29:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:29:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.136808552000002, 2024-09-17T07:29:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:29:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:29:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:27.384 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:29:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:29:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:29:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:29:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:29:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:29:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:29:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:29:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:29:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:29:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:29:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:29:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:29:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:29:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:29:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:29:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:29:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:29:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:29:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:29:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 29.136767602999999, 2024-09-17T07:29:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:29:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:29:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:29:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:29:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:29:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:29:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:29:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:29:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:29:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:29:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:29:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:29:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:29:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:29:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:29:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:29:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:29:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:29:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:29:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:29:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:29:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:29:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:29:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:29:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.396 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:27.397 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:27.398 INFO:teuthology.orchestra.run.smithi045.stdout:leaving image image_client.0-clone23 intact 2024-09-17T07:29:27.398 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:27.396+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:27.498 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:27.497+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:27.556 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:27.556 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:27.565 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:27.599 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:27.597+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:27.616 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-17T07:29:27.621 INFO:teuthology.orchestra.run.smithi045.stdout:2345 read 0xa4d0a9a thru 0xa4d2fa6 (0x250d bytes) 2024-09-17T07:29:27.622 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:27.622 INFO:teuthology.orchestra.run.smithi045.stdout:2347 write 0x8e086a9 thru 0x8e180c1 (0xfa19 bytes) 2024-09-17T07:29:27.637 INFO:teuthology.orchestra.run.smithi045.stdout:2348 write 0xcb48888 thru 0xcb561ee (0xd967 bytes) 2024-09-17T07:29:27.643 INFO:teuthology.orchestra.run.smithi045.stdout:2349 read 0xd29fe49 thru 0xd2a526d (0x5425 bytes) 2024-09-17T07:29:27.644 INFO:teuthology.orchestra.run.smithi045.stdout:2351 clone 25 order 21 su 524288 sc 3 2024-09-17T07:29:27.678 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:27.678 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:27.679 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:27.679 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:27.687 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:27.700 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:27.699+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:27.795 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:4848 3.9 e33/23 3:97363921:::rbd_data.1183cff49e3a.0000000000000004:head v 33'102, mlcod=32'100)", 2024-09-17T07:29:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 33.554891462000001, 2024-09-17T07:29:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023715110000000002, 2024-09-17T07:29:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239497+0000", 2024-09-17T07:29:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.53e-06 2024-09-17T07:29:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240142+0000", 2024-09-17T07:29:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00064481300000000005 2024-09-17T07:29:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240160+0000", 2024-09-17T07:29:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8349999999999999e-05 2024-09-17T07:29:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240169+0000", 2024-09-17T07:29:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7690000000000002e-06 2024-09-17T07:29:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240246+0000", 2024-09-17T07:29:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6277000000000002e-05 2024-09-17T07:29:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240270+0000", 2024-09-17T07:29:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4575000000000001e-05 2024-09-17T07:29:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.241813+0000", 2024-09-17T07:29:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015432860000000001 2024-09-17T07:29:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.241867+0000", 2024-09-17T07:29:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3910999999999999e-05 2024-09-17T07:29:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:5460 3.9 e33/23 3:90773955:::rbd_data.1183cff49e3a.000000000000000a:head v 33'106, mlcod=33'104)", 2024-09-17T07:29:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 33.267794876000004, 2024-09-17T07:29:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020825850000000001, 2024-09-17T07:29:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526593+0000", 2024-09-17T07:29:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.54e-07 2024-09-17T07:29:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527050+0000", 2024-09-17T07:29:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000456522 2024-09-17T07:29:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527061+0000", 2024-09-17T07:29:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1722000000000001e-05 2024-09-17T07:29:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527067+0000", 2024-09-17T07:29:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0599999999999998e-06 2024-09-17T07:29:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527112+0000", 2024-09-17T07:29:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999000000000001e-05 2024-09-17T07:29:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527128+0000", 2024-09-17T07:29:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6920999999999999e-05 2024-09-17T07:29:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528636+0000", 2024-09-17T07:29:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001507836 2024-09-17T07:29:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528675+0000", 2024-09-17T07:29:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8671000000000003e-05 2024-09-17T07:29:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:6005 3.0 e33/23 3:037cd38f:::rbd_data.1183cff49e3a.0000000000000012:head v 33'25, mlcod=32'23)", 2024-09-17T07:29:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 33.169761741999999, 2024-09-17T07:29:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019123899999999999, 2024-09-17T07:29:27.805 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624627+0000", 2024-09-17T07:29:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5600000000000001e-06 2024-09-17T07:29:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624777+0000", 2024-09-17T07:29:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015030800000000001 2024-09-17T07:29:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624792+0000", 2024-09-17T07:29:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5089000000000001e-05 2024-09-17T07:29:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624799+0000", 2024-09-17T07:29:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7939999999999997e-06 2024-09-17T07:29:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624855+0000", 2024-09-17T07:29:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5782999999999998e-05 2024-09-17T07:29:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624871+0000", 2024-09-17T07:29:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6192e-05 2024-09-17T07:29:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.626507+0000", 2024-09-17T07:29:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016354500000000001 2024-09-17T07:29:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.626538+0000", 2024-09-17T07:29:27.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1214000000000003e-05 2024-09-17T07:29:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8595 3.4 e33/23 3:237b2a73:::rbd_data.1183cff49e3a.0000000000000049:head v 33'37, mlcod=33'35)", 2024-09-17T07:29:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.719744783000003, 2024-09-17T07:29:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020228939999999999, 2024-09-17T07:29:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074643+0000", 2024-09-17T07:29:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3099999999999997e-07 2024-09-17T07:29:27.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074834+0000", 2024-09-17T07:29:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000191137 2024-09-17T07:29:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074850+0000", 2024-09-17T07:29:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6101999999999999e-05 2024-09-17T07:29:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074859+0000", 2024-09-17T07:29:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2169999999999994e-06 2024-09-17T07:29:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074919+0000", 2024-09-17T07:29:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.003e-05 2024-09-17T07:29:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074944+0000", 2024-09-17T07:29:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5761e-05 2024-09-17T07:29:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076622+0000", 2024-09-17T07:29:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001677977 2024-09-17T07:29:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076665+0000", 2024-09-17T07:29:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2938999999999997e-05 2024-09-17T07:29:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17069 3.0 e41/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 41'110, mlcod=40'108)", 2024-09-17T07:29:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 25.054635058999999, 2024-09-17T07:29:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011408908000000001, 2024-09-17T07:29:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739753+0000", 2024-09-17T07:29:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:29:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743532+0000", 2024-09-17T07:29:27.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037794510000000001 2024-09-17T07:29:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743557+0000", 2024-09-17T07:29:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5253e-05 2024-09-17T07:29:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743569+0000", 2024-09-17T07:29:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1161999999999999e-05 2024-09-17T07:29:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743650+0000", 2024-09-17T07:29:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1087999999999993e-05 2024-09-17T07:29:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743674+0000", 2024-09-17T07:29:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4562e-05 2024-09-17T07:29:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751095+0000", 2024-09-17T07:29:27.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074211140000000004 2024-09-17T07:29:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751161+0000", 2024-09-17T07:29:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5707e-05 2024-09-17T07:29:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17240 3.0 e42/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 42'113, mlcod=41'110)", 2024-09-17T07:29:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 24.568976575000001, 2024-09-17T07:29:27.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064392440000000002, 2024-09-17T07:29:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:29:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225418+0000", 2024-09-17T07:29:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8360000000000003e-06 2024-09-17T07:29:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225426+0000", 2024-09-17T07:29:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2670000000000006e-06 2024-09-17T07:29:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225434+0000", 2024-09-17T07:29:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0560000000000005e-06 2024-09-17T07:29:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225514+0000", 2024-09-17T07:29:27.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9979000000000001e-05 2024-09-17T07:29:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225535+0000", 2024-09-17T07:29:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0965e-05 2024-09-17T07:29:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231797+0000", 2024-09-17T07:29:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062621129999999997 2024-09-17T07:29:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231850+0000", 2024-09-17T07:29:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2611999999999997e-05 2024-09-17T07:29:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20054 3.9 e44/23 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:head v 44'131, mlcod=44'129)", 2024-09-17T07:29:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.526825459000001, 2024-09-17T07:29:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028510570000000002, 2024-09-17T07:29:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.824 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267563+0000", 2024-09-17T07:29:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.584e-06 2024-09-17T07:29:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268269+0000", 2024-09-17T07:29:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00070567399999999999 2024-09-17T07:29:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268287+0000", 2024-09-17T07:29:27.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.84e-05 2024-09-17T07:29:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268297+0000", 2024-09-17T07:29:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9059999999999997e-06 2024-09-17T07:29:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268368+0000", 2024-09-17T07:29:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0562999999999995e-05 2024-09-17T07:29:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268393+0000", 2024-09-17T07:29:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4958999999999999e-05 2024-09-17T07:29:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270330+0000", 2024-09-17T07:29:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001936644 2024-09-17T07:29:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270413+0000", 2024-09-17T07:29:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3326999999999997e-05 2024-09-17T07:29:27.828 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20055 3.9 e44/23 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:head v 44'133, mlcod=44'129)", 2024-09-17T07:29:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.268886+0000", 2024-09-17T07:29:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.525501387999999, 2024-09-17T07:29:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019232920000000001, 2024-09-17T07:29:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268886+0000", 2024-09-17T07:29:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268886+0000", 2024-09-17T07:29:27.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268887+0000", 2024-09-17T07:29:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.85e-07 2024-09-17T07:29:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268905+0000", 2024-09-17T07:29:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8263000000000001e-05 2024-09-17T07:29:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268916+0000", 2024-09-17T07:29:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0876000000000001e-05 2024-09-17T07:29:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268922+0000", 2024-09-17T07:29:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7579999999999998e-06 2024-09-17T07:29:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.269201+0000", 2024-09-17T07:29:27.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027979200000000001 2024-09-17T07:29:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.269224+0000", 2024-09-17T07:29:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2838999999999999e-05 2024-09-17T07:29:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270774+0000", 2024-09-17T07:29:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015495629999999999 2024-09-17T07:29:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270809+0000", 2024-09-17T07:29:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5416000000000001e-05 2024-09-17T07:29:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20163 3.9 e44/23 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:head v 44'134, mlcod=44'131)", 2024-09-17T07:29:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.294194+0000", 2024-09-17T07:29:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.500193036999999, 2024-09-17T07:29:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001874824, 2024-09-17T07:29:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.834 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294194+0000", 2024-09-17T07:29:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294194+0000", 2024-09-17T07:29:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294196+0000", 2024-09-17T07:29:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3599999999999999e-06 2024-09-17T07:29:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294663+0000", 2024-09-17T07:29:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00046696200000000002 2024-09-17T07:29:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294683+0000", 2024-09-17T07:29:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0829999999999999e-05 2024-09-17T07:29:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294697+0000", 2024-09-17T07:29:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3524e-05 2024-09-17T07:29:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294758+0000", 2024-09-17T07:29:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0618999999999997e-05 2024-09-17T07:29:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294776+0000", 2024-09-17T07:29:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8431e-05 2024-09-17T07:29:27.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.296042+0000", 2024-09-17T07:29:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012656329999999999 2024-09-17T07:29:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.296069+0000", 2024-09-17T07:29:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7464999999999999e-05 2024-09-17T07:29:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.1f 51/50 [PushOp(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head, version: 40'83, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head@40'83, size: 19583394, copy_subset: [(0, 19583394)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:29:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:13.339735+0000", 2024-09-17T07:29:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.45465269, 2024-09-17T07:29:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0018873760000000001, 2024-09-17T07:29:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.339735+0000", 2024-09-17T07:29:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.339735+0000", 2024-09-17T07:29:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.339736+0000", 2024-09-17T07:29:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.392e-06 2024-09-17T07:29:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340637+0000", 2024-09-17T07:29:27.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00090133599999999998 2024-09-17T07:29:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340668+0000", 2024-09-17T07:29:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0488e-05 2024-09-17T07:29:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340683+0000", 2024-09-17T07:29:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5315000000000002e-05 2024-09-17T07:29:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340777+0000", 2024-09-17T07:29:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3456999999999996e-05 2024-09-17T07:29:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340786+0000", 2024-09-17T07:29:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2220000000000008e-06 2024-09-17T07:29:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.341622+0000", 2024-09-17T07:29:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083616599999999995 2024-09-17T07:29:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.842 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.1f 51/50 [PushOp(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head, version: 40'83, data_included: [3407872~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head@40'83, size: 19583394, copy_subset: [(0, 19583394)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4456448, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3407872, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:29:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:13.350296+0000", 2024-09-17T07:29:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.444091745, 2024-09-17T07:29:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00191949, 2024-09-17T07:29:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.350296+0000", 2024-09-17T07:29:27.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.350296+0000", 2024-09-17T07:29:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.350297+0000", 2024-09-17T07:29:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4440000000000001e-06 2024-09-17T07:29:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351304+0000", 2024-09-17T07:29:27.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001007319 2024-09-17T07:29:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351331+0000", 2024-09-17T07:29:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6701000000000001e-05 2024-09-17T07:29:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351346+0000", 2024-09-17T07:29:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4911e-05 2024-09-17T07:29:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351425+0000", 2024-09-17T07:29:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9328999999999999e-05 2024-09-17T07:29:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351436+0000", 2024-09-17T07:29:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0467e-05 2024-09-17T07:29:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.352215+0000", 2024-09-17T07:29:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000779319 2024-09-17T07:29:27.846 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:32543 3.0 e55/23 3:03bfd97d:::rbd_data.11839a97b753.0000000000000007:head v 55'150, mlcod=54'148)", 2024-09-17T07:29:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 10.339750322, 2024-09-17T07:29:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031813480000000001, 2024-09-17T07:29:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5199999999999998e-07 2024-09-17T07:29:27.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455083+0000", 2024-09-17T07:29:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044629199999999999 2024-09-17T07:29:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455098+0000", 2024-09-17T07:29:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4183e-05 2024-09-17T07:29:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455106+0000", 2024-09-17T07:29:27.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:29:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455173+0000", 2024-09-17T07:29:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6346000000000005e-05 2024-09-17T07:29:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455200+0000", 2024-09-17T07:29:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7319000000000001e-05 2024-09-17T07:29:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457774+0000", 2024-09-17T07:29:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025742650000000001 2024-09-17T07:29:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457818+0000", 2024-09-17T07:29:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3891999999999999e-05 2024-09-17T07:29:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33735 3.1f e55/52 3:fea54097:::rbd_data.11839a97b753.0000000000000000:head v 55'260, mlcod=55'258)", 2024-09-17T07:29:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:29:27.851 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.9698176939999996, 2024-09-17T07:29:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035215200000000002, 2024-09-17T07:29:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:29:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:29:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824571+0000", 2024-09-17T07:29:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.164e-06 2024-09-17T07:29:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825118+0000", 2024-09-17T07:29:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054696400000000004 2024-09-17T07:29:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825134+0000", 2024-09-17T07:29:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6342999999999999e-05 2024-09-17T07:29:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825142+0000", 2024-09-17T07:29:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5270000000000001e-06 2024-09-17T07:29:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825200+0000", 2024-09-17T07:29:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.893e-05 2024-09-17T07:29:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825218+0000", 2024-09-17T07:29:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7935999999999999e-05 2024-09-17T07:29:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828037+0000", 2024-09-17T07:29:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002818961 2024-09-17T07:29:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828091+0000", 2024-09-17T07:29:27.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3695e-05 2024-09-17T07:29:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33869 3.9 e55/23 3:907b81b5:::rbd_data.11839a97b753.0000000000000001:head v 55'171, mlcod=55'169)", 2024-09-17T07:29:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:29:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.9527618709999999, 2024-09-17T07:29:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033990859999999999, 2024-09-17T07:29:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:29:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:29:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841626+0000", 2024-09-17T07:29:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.023e-06 2024-09-17T07:29:27.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842067+0000", 2024-09-17T07:29:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000440612 2024-09-17T07:29:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842080+0000", 2024-09-17T07:29:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.29e-05 2024-09-17T07:29:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842086+0000", 2024-09-17T07:29:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2090000000000002e-06 2024-09-17T07:29:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842144+0000", 2024-09-17T07:29:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7627999999999998e-05 2024-09-17T07:29:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842171+0000", 2024-09-17T07:29:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7214000000000001e-05 2024-09-17T07:29:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.844973+0000", 2024-09-17T07:29:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002802232 2024-09-17T07:29:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.845024+0000", 2024-09-17T07:29:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1267999999999998e-05 2024-09-17T07:29:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51432 3.0 e59/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 59'177, mlcod=58'174)", 2024-09-17T07:29:27.860 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:29:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.9731648540000002, 2024-09-17T07:29:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010790086000000001, 2024-09-17T07:29:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:29:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:29:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821224+0000", 2024-09-17T07:29:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.511e-06 2024-09-17T07:29:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825174+0000", 2024-09-17T07:29:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0039497949999999999 2024-09-17T07:29:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825196+0000", 2024-09-17T07:29:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2643999999999999e-05 2024-09-17T07:29:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825206+0000", 2024-09-17T07:29:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4320000000000005e-06 2024-09-17T07:29:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825272+0000", 2024-09-17T07:29:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5829999999999998e-05 2024-09-17T07:29:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825299+0000", 2024-09-17T07:29:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7107e-05 2024-09-17T07:29:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.831939+0000", 2024-09-17T07:29:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0066398510000000004 2024-09-17T07:29:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.832012+0000", 2024-09-17T07:29:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3916000000000002e-05 2024-09-17T07:29:27.864 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51969 3.0 e59/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 59'179, mlcod=59'177)", 2024-09-17T07:29:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:29:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.8642964759999998, 2024-09-17T07:29:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060297249999999997, 2024-09-17T07:29:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:29:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:29:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930093+0000", 2024-09-17T07:29:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.764e-06 2024-09-17T07:29:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931779+0000", 2024-09-17T07:29:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001686379 2024-09-17T07:29:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931802+0000", 2024-09-17T07:29:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2682000000000001e-05 2024-09-17T07:29:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931811+0000", 2024-09-17T07:29:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5640000000000002e-06 2024-09-17T07:29:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931877+0000", 2024-09-17T07:29:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6155999999999995e-05 2024-09-17T07:29:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931896+0000", 2024-09-17T07:29:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9111e-05 2024-09-17T07:29:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936063+0000", 2024-09-17T07:29:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004166711 2024-09-17T07:29:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936121+0000", 2024-09-17T07:29:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7358000000000002e-05 2024-09-17T07:29:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54417 3.0 e60/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 60'182, mlcod=59'179)", 2024-09-17T07:29:27.869 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:29:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.3567794849999997, 2024-09-17T07:29:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0059885219999999996, 2024-09-17T07:29:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:29:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:29:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437609+0000", 2024-09-17T07:29:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5829999999999999e-06 2024-09-17T07:29:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437623+0000", 2024-09-17T07:29:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3348e-05 2024-09-17T07:29:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437643+0000", 2024-09-17T07:29:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0463999999999999e-05 2024-09-17T07:29:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437653+0000", 2024-09-17T07:29:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6290000000000005e-06 2024-09-17T07:29:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437719+0000", 2024-09-17T07:29:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6428999999999997e-05 2024-09-17T07:29:27.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437745+0000", 2024-09-17T07:29:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5690999999999998e-05 2024-09-17T07:29:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443549+0000", 2024-09-17T07:29:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0058038259999999998 2024-09-17T07:29:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443596+0000", 2024-09-17T07:29:27.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7552000000000001e-05 2024-09-17T07:29:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54422 3.1f e60/52 3:fef8599a:::rbd_data.1183a58293a3.0000000000000007:head v 60'280, mlcod=58'277)", 2024-09-17T07:29:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.441086+0000", 2024-09-17T07:29:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.3533011100000003, 2024-09-17T07:29:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001916933, 2024-09-17T07:29:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441086+0000", 2024-09-17T07:29:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441086+0000", 2024-09-17T07:29:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441088+0000", 2024-09-17T07:29:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.517e-06 2024-09-17T07:29:27.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441100+0000", 2024-09-17T07:29:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2532e-05 2024-09-17T07:29:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441118+0000", 2024-09-17T07:29:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7798000000000001e-05 2024-09-17T07:29:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441127+0000", 2024-09-17T07:29:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4749999999999999e-06 2024-09-17T07:29:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441675+0000", 2024-09-17T07:29:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054747699999999999 2024-09-17T07:29:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441693+0000", 2024-09-17T07:29:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7621e-05 2024-09-17T07:29:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.442956+0000", 2024-09-17T07:29:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012633900000000001 2024-09-17T07:29:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443003+0000", 2024-09-17T07:29:27.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7123000000000003e-05 2024-09-17T07:29:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54419 3.0 e60/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 60'184, mlcod=59'179)", 2024-09-17T07:29:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:29:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.3504271279999998, 2024-09-17T07:29:27.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004706844, 2024-09-17T07:29:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:29:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:29:27.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443961+0000", 2024-09-17T07:29:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:29:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443967+0000", 2024-09-17T07:29:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.268e-06 2024-09-17T07:29:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443983+0000", 2024-09-17T07:29:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5248999999999999e-05 2024-09-17T07:29:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443990+0000", 2024-09-17T07:29:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5120000000000002e-06 2024-09-17T07:29:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444056+0000", 2024-09-17T07:29:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6197000000000003e-05 2024-09-17T07:29:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444081+0000", 2024-09-17T07:29:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4791e-05 2024-09-17T07:29:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448616+0000", 2024-09-17T07:29:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045346129999999998 2024-09-17T07:29:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448667+0000", 2024-09-17T07:29:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-17T07:29:27.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54895 3.1f e60/52 3:fa1c1177:::rbd_data.1183ad87379e.0000000000000001:head v 60'283, mlcod=60'281)", 2024-09-17T07:29:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:29:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.0029461319999999, 2024-09-17T07:29:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0029495210000000001, 2024-09-17T07:29:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:29:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:29:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791443+0000", 2024-09-17T07:29:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.153e-06 2024-09-17T07:29:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792619+0000", 2024-09-17T07:29:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001176011 2024-09-17T07:29:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:27.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792637+0000", 2024-09-17T07:29:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-05 2024-09-17T07:29:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792649+0000", 2024-09-17T07:29:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1712000000000001e-05 2024-09-17T07:29:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792722+0000", 2024-09-17T07:29:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3545000000000001e-05 2024-09-17T07:29:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792738+0000", 2024-09-17T07:29:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5223e-05 2024-09-17T07:29:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.794330+0000", 2024-09-17T07:29:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015922740000000001 2024-09-17T07:29:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:27.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.794391+0000", 2024-09-17T07:29:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0803e-05 2024-09-17T07:29:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:27.891 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:27.892 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:27.894 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:27.894 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:27.893+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:27.994 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:27.993+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:28.009 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:28.009 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:28.009 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:28.009 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:28.018 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:28.095 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:28.094+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:28.122 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:28.122 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:28.123 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:28.123 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:28.123 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:28.132 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:28.155 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:29:28.184 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-17T07:29:28.184 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:29:28.185 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-17T07:29:28.185 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T07:29:28.185 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T07:29:28.185 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-17T07:29:28.200 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:28.200+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:28.239 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:28.239 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:28.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:28.240 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:28.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.240 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:30525 3.1b 3:def7c8f0:::rbd_data.118381ffbbab.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T07:29:28.240 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:28.240 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.715057444999999, 2024-09-17T07:29:28.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55515850300000003, 2024-09-17T07:29:28.240 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.240 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.240 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:28.240 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.240 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.240 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.241 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30525 2024-09-17T07:29:28.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.241 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:28.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:28.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522300+0000", 2024-09-17T07:29:28.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:29:28.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522305+0000", 2024-09-17T07:29:28.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-17T07:29:28.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522318+0000", 2024-09-17T07:29:28.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3030999999999999e-05 2024-09-17T07:29:28.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522326+0000", 2024-09-17T07:29:28.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4200000000000001e-06 2024-09-17T07:29:28.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522385+0000", 2024-09-17T07:29:28.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9005000000000002e-05 2024-09-17T07:29:28.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:29:28.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522393+0000", 2024-09-17T07:29:28.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-06 2024-09-17T07:29:28.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075347+0000", 2024-09-17T07:29:28.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55295417800000002 2024-09-17T07:29:28.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075502+0000", 2024-09-17T07:29:28.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154272 2024-09-17T07:29:28.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:28.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075630+0000", 2024-09-17T07:29:28.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012816300000000001 2024-09-17T07:29:28.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:28.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077221+0000", 2024-09-17T07:29:28.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015912929999999999 2024-09-17T07:29:28.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:28.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077353+0000", 2024-09-17T07:29:28.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013215699999999999 2024-09-17T07:29:28.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:28.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077413+0000", 2024-09-17T07:29:28.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0034999999999997e-05 2024-09-17T07:29:28.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077457+0000", 2024-09-17T07:29:28.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4017999999999998e-05 2024-09-17T07:29:28.247 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.247 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.247 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.247 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44181 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13922304~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.247 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:29:28.247 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5998787969999997, 2024-09-17T07:29:28.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046010502000000002, 2024-09-17T07:29:28.247 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.247 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.247 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.247 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.247 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44181 2024-09-17T07:29:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:29:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:29:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637478+0000", 2024-09-17T07:29:28.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:29:28.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637480+0000", 2024-09-17T07:29:28.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.843e-06 2024-09-17T07:29:28.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637487+0000", 2024-09-17T07:29:28.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6159999999999996e-06 2024-09-17T07:29:28.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637492+0000", 2024-09-17T07:29:28.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5360000000000003e-06 2024-09-17T07:29:28.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683351+0000", 2024-09-17T07:29:28.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045858758999999999 2024-09-17T07:29:28.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683422+0000", 2024-09-17T07:29:28.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1705999999999994e-05 2024-09-17T07:29:28.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683488+0000", 2024-09-17T07:29:28.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5612999999999997e-05 2024-09-17T07:29:28.251 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.251 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.251 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.251 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44182 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13938688~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.251 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:28.251 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.599858126, 2024-09-17T07:29:28.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046153118999999999, 2024-09-17T07:29:28.252 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.252 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.252 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.252 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44182 2024-09-17T07:29:28.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.252 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:28.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:28.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:28.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:29:28.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637506+0000", 2024-09-17T07:29:28.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1520000000000003e-06 2024-09-17T07:29:28.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637512+0000", 2024-09-17T07:29:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-09-17T07:29:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637517+0000", 2024-09-17T07:29:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-09-17T07:29:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683514+0000", 2024-09-17T07:29:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045997616999999998 2024-09-17T07:29:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683584+0000", 2024-09-17T07:29:28.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9942000000000002e-05 2024-09-17T07:29:28.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683651+0000", 2024-09-17T07:29:28.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6951000000000003e-05 2024-09-17T07:29:28.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.259 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.259 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44183 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13955072~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.260 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:29:28.260 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5998328940000004, 2024-09-17T07:29:28.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046339793999999997, 2024-09-17T07:29:28.260 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.260 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.260 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.260 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44183 2024-09-17T07:29:28.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.260 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:29:28.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:29:28.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639583+0000", 2024-09-17T07:29:28.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002059412 2024-09-17T07:29:28.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639597+0000", 2024-09-17T07:29:28.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3832000000000001e-05 2024-09-17T07:29:28.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639607+0000", 2024-09-17T07:29:28.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485e-05 2024-09-17T07:29:28.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639614+0000", 2024-09-17T07:29:28.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0849999999999999e-06 2024-09-17T07:29:28.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683705+0000", 2024-09-17T07:29:28.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044091261 2024-09-17T07:29:28.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683792+0000", 2024-09-17T07:29:28.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6197000000000002e-05 2024-09-17T07:29:28.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683863+0000", 2024-09-17T07:29:28.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1521999999999997e-05 2024-09-17T07:29:28.264 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.264 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.264 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.264 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44199 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14217216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.264 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:28.264 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5963153420000005, 2024-09-17T07:29:28.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045995623999999999, 2024-09-17T07:29:28.264 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.264 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.264 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.265 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44199 2024-09-17T07:29:28.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.265 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:28.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:28.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:28.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:29:28.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641043+0000", 2024-09-17T07:29:28.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.624e-06 2024-09-17T07:29:28.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641048+0000", 2024-09-17T07:29:28.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4029999999999998e-06 2024-09-17T07:29:28.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641053+0000", 2024-09-17T07:29:28.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6670000000000004e-06 2024-09-17T07:29:28.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686918+0000", 2024-09-17T07:29:28.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045865131000000003 2024-09-17T07:29:28.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686986+0000", 2024-09-17T07:29:28.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8299000000000005e-05 2024-09-17T07:29:28.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687036+0000", 2024-09-17T07:29:28.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0164999999999998e-05 2024-09-17T07:29:28.268 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.268 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.268 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.268 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44200 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14233600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.269 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:28.269 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5962975129999997, 2024-09-17T07:29:28.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046183984999999997, 2024-09-17T07:29:28.269 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.269 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.269 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.269 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.269 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.269 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.269 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44200 2024-09-17T07:29:28.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.269 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:28.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:28.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:28.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:28.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641061+0000", 2024-09-17T07:29:28.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.767e-06 2024-09-17T07:29:28.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641067+0000", 2024-09-17T07:29:28.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7379999999999999e-06 2024-09-17T07:29:28.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641071+0000", 2024-09-17T07:29:28.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-09-17T07:29:28.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687078+0000", 2024-09-17T07:29:28.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046007095999999997 2024-09-17T07:29:28.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687158+0000", 2024-09-17T07:29:28.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0374e-05 2024-09-17T07:29:28.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687243+0000", 2024-09-17T07:29:28.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4221999999999994e-05 2024-09-17T07:29:28.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.273 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.273 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44201 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14249984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.273 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:29:28.273 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5962798669999998, 2024-09-17T07:29:28.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046303182999999998, 2024-09-17T07:29:28.273 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.273 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.273 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.273 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.273 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.274 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44201 2024-09-17T07:29:28.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.274 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:29:28.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:29:28.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641120+0000", 2024-09-17T07:29:28.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3933000000000001e-05 2024-09-17T07:29:28.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641123+0000", 2024-09-17T07:29:28.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.191e-06 2024-09-17T07:29:28.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641133+0000", 2024-09-17T07:29:28.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4020000000000008e-06 2024-09-17T07:29:28.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641138+0000", 2024-09-17T07:29:28.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6559999999999997e-06 2024-09-17T07:29:28.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687279+0000", 2024-09-17T07:29:28.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046140278999999999 2024-09-17T07:29:28.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687338+0000", 2024-09-17T07:29:28.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9268999999999998e-05 2024-09-17T07:29:28.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687379+0000", 2024-09-17T07:29:28.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1452999999999998e-05 2024-09-17T07:29:28.277 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.277 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.277 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.278 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44202 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14266368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.278 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:29:28.278 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5962091010000004, 2024-09-17T07:29:28.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046484299, 2024-09-17T07:29:28.278 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.278 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.278 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.278 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44202 2024-09-17T07:29:28.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.278 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:29:28.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:29:28.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642810+0000", 2024-09-17T07:29:28.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016627980000000001 2024-09-17T07:29:28.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642815+0000", 2024-09-17T07:29:28.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.091e-06 2024-09-17T07:29:28.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642829+0000", 2024-09-17T07:29:28.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4047e-05 2024-09-17T07:29:28.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642844+0000", 2024-09-17T07:29:28.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4515000000000001e-05 2024-09-17T07:29:28.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687426+0000", 2024-09-17T07:29:28.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044582201000000002 2024-09-17T07:29:28.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687529+0000", 2024-09-17T07:29:28.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103078 2024-09-17T07:29:28.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687631+0000", 2024-09-17T07:29:28.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102569 2024-09-17T07:29:28.282 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.282 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.282 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.282 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44209 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14381056~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.282 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:29:28.282 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5943109490000005, 2024-09-17T07:29:28.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045944444000000001, 2024-09-17T07:29:28.282 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.282 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.282 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.282 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.282 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.283 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44209 2024-09-17T07:29:28.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.283 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:29:28.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:29:28.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643046+0000", 2024-09-17T07:29:28.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:29:28.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643048+0000", 2024-09-17T07:29:28.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:29:28.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643054+0000", 2024-09-17T07:29:28.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.032e-06 2024-09-17T07:29:28.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643065+0000", 2024-09-17T07:29:28.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1861999999999999e-05 2024-09-17T07:29:28.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688806+0000", 2024-09-17T07:29:28.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045740617999999997 2024-09-17T07:29:28.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688927+0000", 2024-09-17T07:29:28.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000121327 2024-09-17T07:29:28.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688990+0000", 2024-09-17T07:29:28.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2293999999999997e-05 2024-09-17T07:29:28.287 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.287 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.287 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.287 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44210 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14397440~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.287 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:29:28.287 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5942840230000002, 2024-09-17T07:29:28.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046104635999999997, 2024-09-17T07:29:28.288 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.288 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.288 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.288 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.288 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.288 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.288 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44210 2024-09-17T07:29:28.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.288 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:29:28.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:29:28.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643073+0000", 2024-09-17T07:29:28.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.77e-07 2024-09-17T07:29:28.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643075+0000", 2024-09-17T07:29:28.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-09-17T07:29:28.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643082+0000", 2024-09-17T07:29:28.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7789999999999996e-06 2024-09-17T07:29:28.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643087+0000", 2024-09-17T07:29:28.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.967e-06 2024-09-17T07:29:28.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689022+0000", 2024-09-17T07:29:28.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045934349999999999 2024-09-17T07:29:28.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689112+0000", 2024-09-17T07:29:28.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0039999999999999e-05 2024-09-17T07:29:28.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689177+0000", 2024-09-17T07:29:28.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5041000000000003e-05 2024-09-17T07:29:28.291 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.292 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.292 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.292 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44211 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14413824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.292 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:29:28.292 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5942628899999995, 2024-09-17T07:29:28.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046309003000000001, 2024-09-17T07:29:28.292 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.292 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.292 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.293 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44211 2024-09-17T07:29:28.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.293 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:29:28.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:29:28.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643094+0000", 2024-09-17T07:29:28.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:29:28.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643095+0000", 2024-09-17T07:29:28.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-17T07:29:28.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643101+0000", 2024-09-17T07:29:28.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8730000000000004e-06 2024-09-17T07:29:28.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643106+0000", 2024-09-17T07:29:28.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6859999999999997e-06 2024-09-17T07:29:28.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689220+0000", 2024-09-17T07:29:28.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046113945000000003 2024-09-17T07:29:28.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689309+0000", 2024-09-17T07:29:28.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9560000000000003e-05 2024-09-17T07:29:28.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689402+0000", 2024-09-17T07:29:28.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2786000000000002e-05 2024-09-17T07:29:28.296 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.296 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.296 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.296 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44212 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14430208~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.296 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:28.296 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5942183280000002, 2024-09-17T07:29:28.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046448912000000002, 2024-09-17T07:29:28.296 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.297 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.297 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.297 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.297 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.297 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.297 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44212 2024-09-17T07:29:28.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.297 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:28.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:28.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:28.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2200000000000004e-07 2024-09-17T07:29:28.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643141+0000", 2024-09-17T07:29:28.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5210000000000001e-06 2024-09-17T07:29:28.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643149+0000", 2024-09-17T07:29:28.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6490000000000008e-06 2024-09-17T07:29:28.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643161+0000", 2024-09-17T07:29:28.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2241e-05 2024-09-17T07:29:28.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689423+0000", 2024-09-17T07:29:28.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046262323000000001 2024-09-17T07:29:28.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689505+0000", 2024-09-17T07:29:28.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2103999999999999e-05 2024-09-17T07:29:28.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689587+0000", 2024-09-17T07:29:28.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1452000000000006e-05 2024-09-17T07:29:28.300 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.300 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.300 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.301 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44213 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14446592~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.301 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:29:28.301 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5941868289999999, 2024-09-17T07:29:28.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046601780000000002, 2024-09-17T07:29:28.301 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.301 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.301 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.301 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.301 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.301 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.301 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44213 2024-09-17T07:29:28.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:29:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:29:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643170+0000", 2024-09-17T07:29:28.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-17T07:29:28.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643172+0000", 2024-09-17T07:29:28.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1660000000000001e-06 2024-09-17T07:29:28.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643180+0000", 2024-09-17T07:29:28.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-09-17T07:29:28.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643185+0000", 2024-09-17T07:29:28.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-09-17T07:29:28.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689622+0000", 2024-09-17T07:29:28.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046436742000000003 2024-09-17T07:29:28.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689697+0000", 2024-09-17T07:29:28.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5179999999999995e-05 2024-09-17T07:29:28.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689771+0000", 2024-09-17T07:29:28.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3928000000000001e-05 2024-09-17T07:29:28.305 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.305 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.305 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.305 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44214 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14462976~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.305 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:29:28.305 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5941637689999997, 2024-09-17T07:29:28.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046736038000000001, 2024-09-17T07:29:28.305 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.305 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.305 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44214 2024-09-17T07:29:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:29:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:29:28.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643193+0000", 2024-09-17T07:29:28.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:29:28.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643195+0000", 2024-09-17T07:29:28.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0640000000000001e-06 2024-09-17T07:29:28.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643201+0000", 2024-09-17T07:29:28.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2859999999999996e-06 2024-09-17T07:29:28.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643206+0000", 2024-09-17T07:29:28.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-17T07:29:28.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689814+0000", 2024-09-17T07:29:28.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046607742000000001 2024-09-17T07:29:28.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689886+0000", 2024-09-17T07:29:28.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2020999999999994e-05 2024-09-17T07:29:28.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689928+0000", 2024-09-17T07:29:28.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2771000000000001e-05 2024-09-17T07:29:28.309 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.309 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.309 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.309 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44215 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14479360~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.309 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:29:28.310 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5941198720000003, 2024-09-17T07:29:28.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046868543999999998, 2024-09-17T07:29:28.310 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.310 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.310 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.310 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.310 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.310 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.310 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44215 2024-09-17T07:29:28.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.310 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:29:28.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:29:28.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643237+0000", 2024-09-17T07:29:28.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:29:28.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643245+0000", 2024-09-17T07:29:28.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3270000000000001e-06 2024-09-17T07:29:28.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643254+0000", 2024-09-17T07:29:28.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6049999999999998e-06 2024-09-17T07:29:28.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643260+0000", 2024-09-17T07:29:28.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7849999999999997e-06 2024-09-17T07:29:28.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689961+0000", 2024-09-17T07:29:28.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046701196 2024-09-17T07:29:28.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690036+0000", 2024-09-17T07:29:28.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5153000000000004e-05 2024-09-17T07:29:28.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690105+0000", 2024-09-17T07:29:28.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8874000000000005e-05 2024-09-17T07:29:28.313 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.313 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.313 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.314 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44216 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14495744~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.314 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:29:28.314 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.594088159, 2024-09-17T07:29:28.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047003047999999999, 2024-09-17T07:29:28.314 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.314 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.314 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.314 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.314 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.314 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.314 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44216 2024-09-17T07:29:28.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.315 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:29:28.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:29:28.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643269+0000", 2024-09-17T07:29:28.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:29:28.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643271+0000", 2024-09-17T07:29:28.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-09-17T07:29:28.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643278+0000", 2024-09-17T07:29:28.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8850000000000001e-06 2024-09-17T07:29:28.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643282+0000", 2024-09-17T07:29:28.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5979999999999999e-06 2024-09-17T07:29:28.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690133+0000", 2024-09-17T07:29:28.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850470999999998 2024-09-17T07:29:28.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690205+0000", 2024-09-17T07:29:28.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2442999999999997e-05 2024-09-17T07:29:28.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690271+0000", 2024-09-17T07:29:28.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5874e-05 2024-09-17T07:29:28.318 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.318 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.318 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.318 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44217 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14512128~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.318 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:28.318 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5940671469999996, 2024-09-17T07:29:28.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047166485000000001, 2024-09-17T07:29:28.318 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.318 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.319 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.319 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.319 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.319 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.319 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44217 2024-09-17T07:29:28.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.319 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:28.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:28.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:28.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-17T07:29:28.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643291+0000", 2024-09-17T07:29:28.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.962e-06 2024-09-17T07:29:28.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643297+0000", 2024-09-17T07:29:28.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0700000000000003e-06 2024-09-17T07:29:28.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643302+0000", 2024-09-17T07:29:28.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:29:28.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690308+0000", 2024-09-17T07:29:28.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047005354999999999 2024-09-17T07:29:28.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690385+0000", 2024-09-17T07:29:28.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7541999999999997e-05 2024-09-17T07:29:28.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690455+0000", 2024-09-17T07:29:28.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0328999999999997e-05 2024-09-17T07:29:28.322 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.322 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.322 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.323 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44218 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14528512~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.323 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:28.323 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5940474699999996, 2024-09-17T07:29:28.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047321015000000001, 2024-09-17T07:29:28.323 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.323 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.323 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.323 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.323 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.323 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.323 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44218 2024-09-17T07:29:28.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.323 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:28.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:28.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:28.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-17T07:29:28.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643311+0000", 2024-09-17T07:29:28.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.933e-06 2024-09-17T07:29:28.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643318+0000", 2024-09-17T07:29:28.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3980000000000004e-06 2024-09-17T07:29:28.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643322+0000", 2024-09-17T07:29:28.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7609999999999998e-06 2024-09-17T07:29:28.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690484+0000", 2024-09-17T07:29:28.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047161428999999998 2024-09-17T07:29:28.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690564+0000", 2024-09-17T07:29:28.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9845000000000005e-05 2024-09-17T07:29:28.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690630+0000", 2024-09-17T07:29:28.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6021999999999999e-05 2024-09-17T07:29:28.326 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.327 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.327 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.327 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44219 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14544896~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.327 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:28.327 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5940271030000002, 2024-09-17T07:29:28.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048128708999999999, 2024-09-17T07:29:28.327 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.327 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.327 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.327 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.327 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.327 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.328 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44219 2024-09-17T07:29:28.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.328 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:28.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:28.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:28.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:29:28.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643337+0000", 2024-09-17T07:29:28.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.109e-06 2024-09-17T07:29:28.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643350+0000", 2024-09-17T07:29:28.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3046e-05 2024-09-17T07:29:28.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643356+0000", 2024-09-17T07:29:28.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8930000000000002e-06 2024-09-17T07:29:28.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690660+0000", 2024-09-17T07:29:28.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047304061000000001 2024-09-17T07:29:28.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691372+0000", 2024-09-17T07:29:28.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00071281299999999997 2024-09-17T07:29:28.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691458+0000", 2024-09-17T07:29:28.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5352000000000005e-05 2024-09-17T07:29:28.331 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.331 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.331 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.331 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44220 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14561280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:28.331 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:29:28.331 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5939930550000003, 2024-09-17T07:29:28.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048309849000000002, 2024-09-17T07:29:28.331 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:28.332 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:28.332 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:28.332 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:28.332 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:28.332 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:28.332 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44220 2024-09-17T07:29:28.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.332 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:28.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:28.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:29:28.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:28.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:29:28.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:28.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:28.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646869+0000", 2024-09-17T07:29:28.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003506267 2024-09-17T07:29:28.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:28.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646874+0000", 2024-09-17T07:29:28.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7500000000000003e-06 2024-09-17T07:29:28.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:28.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646887+0000", 2024-09-17T07:29:28.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3115999999999999e-05 2024-09-17T07:29:28.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:28.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646894+0000", 2024-09-17T07:29:28.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5289999999999997e-06 2024-09-17T07:29:28.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:28.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691547+0000", 2024-09-17T07:29:28.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044652743000000002 2024-09-17T07:29:28.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:28.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691622+0000", 2024-09-17T07:29:28.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5037999999999995e-05 2024-09-17T07:29:28.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:28.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:28.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:28.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691673+0000", 2024-09-17T07:29:28.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-17T07:29:28.336 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.336 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.336 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.336 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:28.336 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:28.336 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:28.338 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-17T07:29:28.338 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:28.339 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:28.339 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:28.340 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:28.340 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:28.340 INFO:teuthology.orchestra.run.smithi192.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T07:29:28.340 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:28.341 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:28.336+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:28.341 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:28.341 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:28.341 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:28.342 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:28.342 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:28.360 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:28.360 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:28.371 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone24 from 0xed4e019 (overlap 0xed4e019) to 0xa61d011 2024-09-17T07:29:28.380 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:28.411 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:28.412 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:28.415 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:28.416 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:28.456 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:28.456+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:28.481 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:28.481 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:28.495 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:28.539 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:28.539+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:28.594 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-17T07:29:28.602 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:28.602 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:28.611 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:28.654 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:28.653+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:28.714 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:28.715 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:28.723 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:28.742 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:28.741+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:28.842 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:28.842+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:28.851 INFO:teuthology.orchestra.run.smithi045.stdout:2352 write 0x3badc46 thru 0x3bb3ebe (0x6279 bytes) 2024-09-17T07:29:28.861 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:28.862 INFO:teuthology.orchestra.run.smithi045.stdout:2354 trunc from 0xed4e019 to 0xdc33500 2024-09-17T07:29:28.870 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:28.870 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:28.872 INFO:teuthology.orchestra.run.smithi045.stdout:2357 read 0x5a11af5 thru 0x5a12682 (0xb8e bytes) 2024-09-17T07:29:28.873 INFO:teuthology.orchestra.run.smithi045.stdout:2358 writesame 0x76322cd thru 0x763f106 data_size 0x3f(0xce3a bytes) 2024-09-17T07:29:28.878 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:28.878 INFO:teuthology.orchestra.run.smithi045.stdout:2361 write 0xc9882a4 thru 0xc99326a (0xafc7 bytes) 2024-09-17T07:29:28.879 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:28.886 INFO:teuthology.orchestra.run.smithi045.stdout:2364 punch from 0xdac5e21 to 0xdace6b6, (0x8895 bytes) 2024-09-17T07:29:28.889 INFO:teuthology.orchestra.run.smithi045.stdout:2366 trunc from 0xdc33500 to 0xaa2d7fd 2024-09-17T07:29:28.905 INFO:teuthology.orchestra.run.smithi045.stdout:2367 write 0x8613b10 thru 0x8621e12 (0xe303 bytes) 2024-09-17T07:29:28.908 INFO:teuthology.orchestra.run.smithi045.stdout:2368 read 0x13df958 thru 0x13e4bef (0x5298 bytes) 2024-09-17T07:29:28.909 INFO:teuthology.orchestra.run.smithi045.stdout:2369 trunc from 0xaa2d7fd to 0xb2a92a5 2024-09-17T07:29:28.916 INFO:teuthology.orchestra.run.smithi045.stdout:2371 read 0x81d6cb0 thru 0x81d9b93 (0x2ee4 bytes) 2024-09-17T07:29:28.916 INFO:teuthology.orchestra.run.smithi045.stdout:2374 clone 26 order 19 su 16384 sc 10 2024-09-17T07:29:28.943 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:28.942+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:28.983 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:28.984 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:28.993 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:29.077 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:29.075+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:29.136 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:29.136 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:29.145 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:29.187 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:29.186+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:29.248 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:29.247+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:29.293 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:29.294 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:29.303 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:29.388 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:29.387+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:29.401 INFO:teuthology.orchestra.run.smithi045.stdout:flattening image image_client.0-clone25 2024-09-17T07:29:29.450 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:29.449+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:29.489 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:29.489 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:29.498 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:29.551 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:29.550+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:29.646 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:29.646 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:29.651 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:29.651+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:29.655 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:29.676 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-17T07:29:29.753 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:29.753+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:29.784 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:29.784 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:29.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:29.855 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:29.854+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:29.903 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:29.903 INFO:teuthology.orchestra.run.smithi045.stdout:2377 read 0x7cf6057 thru 0x7cfa36a (0x4314 bytes) 2024-09-17T07:29:29.904 INFO:teuthology.orchestra.run.smithi045.stdout:2378 punch from 0x86bf11c to 0x86c191f, (0x2803 bytes) 2024-09-17T07:29:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:29.907 INFO:teuthology.orchestra.run.smithi045.stdout:2381 write 0x3df990c thru 0x3e00c62 (0x7357 bytes) 2024-09-17T07:29:29.912 INFO:teuthology.orchestra.run.smithi045.stdout:2382 read 0x976134b thru 0x976d083 (0xbd39 bytes) 2024-09-17T07:29:29.912 INFO:teuthology.orchestra.run.smithi045.stdout:2383 read 0x2c8240 thru 0x2cabf9 (0x29ba bytes) 2024-09-17T07:29:29.922 INFO:teuthology.orchestra.run.smithi045.stdout:2384 trunc from 0xb2a92a5 to 0x2d6a85d 2024-09-17T07:29:29.942 INFO:teuthology.orchestra.run.smithi045.stdout:2385 read 0x8e20f6 thru 0x8f08da (0xe7e5 bytes) 2024-09-17T07:29:29.942 INFO:teuthology.orchestra.run.smithi045.stdout:2386 read 0x2222dce thru 0x22247a2 (0x19d5 bytes) 2024-09-17T07:29:29.942 INFO:teuthology.orchestra.run.smithi045.stdout:2387 read 0x258c996 thru 0x2593530 (0x6b9b bytes) 2024-09-17T07:29:29.943 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:29.943 INFO:teuthology.orchestra.run.smithi045.stdout:2389 write 0x2a205bb thru 0x2a2dad9 (0xd51f bytes) 2024-09-17T07:29:29.950 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:29.950 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:29.954 INFO:teuthology.orchestra.run.smithi045.stdout:2390 write 0x4b2b8ac thru 0x4b2eb3c (0x3291 bytes) 2024-09-17T07:29:29.955 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:29.954+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:29.960 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:29.967 INFO:teuthology.orchestra.run.smithi045.stdout:2391 write 0x77789e8 thru 0x777fac5 (0x70de bytes) 2024-09-17T07:29:29.972 INFO:teuthology.orchestra.run.smithi045.stdout:2392 read 0x3c05950 thru 0x3c12840 (0xcef1 bytes) 2024-09-17T07:29:29.972 INFO:teuthology.orchestra.run.smithi045.stdout:2393 punch from 0x75d365d to 0x75db5f1, (0x7f94 bytes) 2024-09-17T07:29:29.972 INFO:teuthology.orchestra.run.smithi045.stdout:2394 write 0x111b13c thru 0x111da40 (0x2905 bytes) 2024-09-17T07:29:29.979 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:29.979 INFO:teuthology.orchestra.run.smithi045.stdout:2398 read 0x42face thru 0x431958 (0x1e8b bytes) 2024-09-17T07:29:29.985 INFO:teuthology.orchestra.run.smithi045.stdout:2400 writesame 0x8503a73 thru 0x8507aae data_size 0x2(0x403c bytes) 2024-09-17T07:29:29.985 INFO:teuthology.orchestra.run.smithi045.stdout:2401 punch from 0x3b2cf97 to 0x3b35c64, (0x8ccd bytes) 2024-09-17T07:29:29.985 INFO:teuthology.orchestra.run.smithi045.stdout:2402 read 0xe67e62 thru 0xe73cea (0xbe89 bytes) 2024-09-17T07:29:29.995 INFO:teuthology.orchestra.run.smithi045.stdout:2403 writesame 0xba713c1 thru 0xba7aadc data_size 0x26(0x971c bytes) 2024-09-17T07:29:29.995 INFO:teuthology.orchestra.run.smithi045.stdout:2404 write 0x55e231b thru 0x55e442f (0x2115 bytes) 2024-09-17T07:29:29.999 INFO:teuthology.orchestra.run.smithi045.stdout:2405 write 0x6a7d4cf thru 0x6a81886 (0x43b8 bytes) 2024-09-17T07:29:30.004 INFO:teuthology.orchestra.run.smithi045.stdout:2406 write 0x9c2995d thru 0x9c30c4a (0x72ee bytes) 2024-09-17T07:29:30.009 INFO:teuthology.orchestra.run.smithi045.stdout:2408 writesame 0x3daeeb3 thru 0x3daf40f data_size 0x1(0x55d bytes) 2024-09-17T07:29:30.014 INFO:teuthology.orchestra.run.smithi045.stdout:2409 read 0xadfab0c thru 0xae0856e (0xda63 bytes) 2024-09-17T07:29:30.014 INFO:teuthology.orchestra.run.smithi045.stdout:2410 punch from 0x77fcc3f to 0x77ffcc9, (0x308a bytes) 2024-09-17T07:29:30.014 INFO:teuthology.orchestra.run.smithi045.stdout:2412 read 0x9f38ade thru 0x9f462ad (0xd7d0 bytes) 2024-09-17T07:29:30.014 INFO:teuthology.orchestra.run.smithi045.stdout:2413 punch from 0x2f94077 to 0x2f9e94a, (0xa8d3 bytes) 2024-09-17T07:29:30.025 INFO:teuthology.orchestra.run.smithi045.stdout:2414 trunc from 0xba7aadd to 0x1765270 2024-09-17T07:29:30.056 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.056 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:30.056+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:30.067 INFO:teuthology.orchestra.run.smithi045.stdout:2416 writesame 0x771467b thru 0x771d008 data_size 0x3(0x898e bytes) 2024-09-17T07:29:30.079 INFO:teuthology.orchestra.run.smithi045.stdout:2417 write 0xccd4a9e thru 0xcce07e5 (0xbd48 bytes) 2024-09-17T07:29:30.087 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:30.087 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:30.088 INFO:teuthology.orchestra.run.smithi045.stdout:2418 read 0x26b647f thru 0x26bb033 (0x4bb5 bytes) 2024-09-17T07:29:30.088 INFO:teuthology.orchestra.run.smithi045.stdout:2419 punch from 0x56b1f2c to 0x56bbd5a, (0x9e2e bytes) 2024-09-17T07:29:30.088 INFO:teuthology.orchestra.run.smithi045.stdout:2420 punch from 0x102a9aa to 0x103a73f, (0xfd95 bytes) 2024-09-17T07:29:30.088 INFO:teuthology.orchestra.run.smithi045.stdout:2422 writesame 0x305c11f thru 0x306b469 data_size 0xd(0xf34b bytes) 2024-09-17T07:29:30.096 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:30.097 INFO:teuthology.orchestra.run.smithi045.stdout:2423 punch from 0x5acb1f9 to 0x5ad96b1, (0xe4b8 bytes) 2024-09-17T07:29:30.107 INFO:teuthology.orchestra.run.smithi045.stdout:2426 trunc from 0xcce07e6 to 0x3c45a99 2024-09-17T07:29:30.117 INFO:teuthology.orchestra.run.smithi045.stdout:2427 punch from 0x1cb178a to 0x1cb9885, (0x80fb bytes) 2024-09-17T07:29:30.117 INFO:teuthology.orchestra.run.smithi045.stdout:2428 write 0x23dad57 thru 0x23dba71 (0xd1b bytes) 2024-09-17T07:29:30.125 INFO:teuthology.orchestra.run.smithi045.stdout:2429 write 0x518c877 thru 0x518cd0f (0x499 bytes) 2024-09-17T07:29:30.128 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.140 INFO:teuthology.orchestra.run.smithi045.stdout:2431 write 0xb68e40c thru 0xb6949d3 (0x65c8 bytes) 2024-09-17T07:29:30.145 INFO:teuthology.orchestra.run.smithi045.stdout:2436 trunc from 0xb6949d4 to 0xba9ca41 2024-09-17T07:29:30.150 INFO:teuthology.orchestra.run.smithi045.stdout:2437 writesame 0x1146c72 thru 0x114eb6d data_size 0x2b(0x7efc bytes) 2024-09-17T07:29:30.157 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:30.157+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:30.158 INFO:teuthology.orchestra.run.smithi045.stdout:2438 read 0x4b3d544 thru 0x4b4382e (0x62eb bytes) 2024-09-17T07:29:30.158 INFO:teuthology.orchestra.run.smithi045.stdout:2439 punch from 0x6676e06 to 0x66856e6, (0xe8e0 bytes) 2024-09-17T07:29:30.166 INFO:teuthology.orchestra.run.smithi045.stdout:2440 write 0xdab1988 thru 0xdabfa97 (0xe110 bytes) 2024-09-17T07:29:30.171 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.171 INFO:teuthology.orchestra.run.smithi045.stdout:2443 read 0x109d955 thru 0x10aa432 (0xcade bytes) 2024-09-17T07:29:30.186 INFO:teuthology.orchestra.run.smithi045.stdout:2444 trunc from 0xdabfa98 to 0x221f480 2024-09-17T07:29:30.196 INFO:teuthology.orchestra.run.smithi045.stdout:2446 punch from 0x13e1170 to 0x13e5d1e, (0x4bae bytes) 2024-09-17T07:29:30.196 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.196 INFO:teuthology.orchestra.run.smithi045.stdout:2449 punch from 0x19bafc4 to 0x19bfdf4, (0x4e30 bytes) 2024-09-17T07:29:30.211 INFO:teuthology.orchestra.run.smithi045.stdout:2450 write 0xaa00dbd thru 0xaa0321e (0x2462 bytes) 2024-09-17T07:29:30.219 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:30.219 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:30.222 INFO:teuthology.orchestra.run.smithi045.stdout:2453 trunc from 0xaa0321f to 0x29d7687 2024-09-17T07:29:30.230 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:30.233 INFO:teuthology.orchestra.run.smithi045.stdout:2454 read 0x1b1c954 thru 0x1b1f5ff (0x2cac bytes) 2024-09-17T07:29:30.233 INFO:teuthology.orchestra.run.smithi045.stdout:2456 read 0x251cd4c thru 0x251d67a (0x92f bytes) 2024-09-17T07:29:30.237 INFO:teuthology.orchestra.run.smithi045.stdout:2459 write 0x2f7ddf3 thru 0x2f8dcf6 (0xff04 bytes) 2024-09-17T07:29:30.258 INFO:teuthology.orchestra.run.smithi045.stdout:2461 write 0x658fcfb thru 0x659f0df (0xf3e5 bytes) 2024-09-17T07:29:30.258 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:30.257+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:30.268 INFO:teuthology.orchestra.run.smithi045.stdout:2462 read 0x344cc64 thru 0x3459b17 (0xceb4 bytes) 2024-09-17T07:29:30.268 INFO:teuthology.orchestra.run.smithi045.stdout:2463 punch from 0x6154523 to 0x615b2e4, (0x6dc1 bytes) 2024-09-17T07:29:30.268 INFO:teuthology.orchestra.run.smithi045.stdout:2464 read 0xc5c88b thru 0xc689e7 (0xc15d bytes) 2024-09-17T07:29:30.268 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.280 INFO:teuthology.orchestra.run.smithi045.stdout:2466 writesame 0xcb513e4 thru 0xcb5b7fe data_size 0x1(0xa41b bytes) 2024-09-17T07:29:30.280 INFO:teuthology.orchestra.run.smithi045.stdout:2467 punch from 0x8ac60d to 0x8aeee9, (0x28dc bytes) 2024-09-17T07:29:30.280 INFO:teuthology.orchestra.run.smithi045.stdout:2468 punch from 0xc4d0911 to 0xc4d9c01, (0x92f0 bytes) 2024-09-17T07:29:30.289 INFO:teuthology.orchestra.run.smithi045.stdout:2469 trunc from 0xcb5b7ff to 0x435bf09 2024-09-17T07:29:30.298 INFO:teuthology.orchestra.run.smithi045.stdout:2470 punch from 0x34fe6a3 to 0x35071a8, (0x8b05 bytes) 2024-09-17T07:29:30.308 INFO:teuthology.orchestra.run.smithi045.stdout:2471 write 0x8bad80f thru 0x8bb7c4d (0xa43f bytes) 2024-09-17T07:29:30.312 INFO:teuthology.orchestra.run.smithi045.stdout:2472 punch from 0x4571d30 to 0x457c959, (0xac29 bytes) 2024-09-17T07:29:30.312 INFO:teuthology.orchestra.run.smithi045.stdout:2473 write 0x4b82797 thru 0x4b90e06 (0xe670 bytes) 2024-09-17T07:29:30.330 INFO:teuthology.orchestra.run.smithi045.stdout:2474 write 0xe23254d thru 0xe23d3a9 (0xae5d bytes) 2024-09-17T07:29:30.340 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T07:29:30.348 INFO:teuthology.orchestra.run.smithi045.stdout:2475 trunc from 0xe23d3aa to 0x6995492 2024-09-17T07:29:30.352 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:30.353 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:30.359 INFO:teuthology.orchestra.run.smithi045.stdout:2477 read 0x537cffa thru 0x5382c1f (0x5c26 bytes) 2024-09-17T07:29:30.359 INFO:teuthology.orchestra.run.smithi045.stdout:2478 punch from 0x23ad483 to 0x23b6a22, (0x959f bytes) 2024-09-17T07:29:30.360 INFO:teuthology.orchestra.run.smithi045.stdout:2479 writesame 0x383c414 thru 0x383cc55 data_size 0x7(0x842 bytes) 2024-09-17T07:29:30.360 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:30.358+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:30.363 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:30.374 INFO:teuthology.orchestra.run.smithi045.stdout:2481 write 0xc83c4d0 thru 0xc8422b0 (0x5de1 bytes) 2024-09-17T07:29:30.377 INFO:teuthology.orchestra.run.smithi045.stdout:2482 read 0x398631b thru 0x398e435 (0x811b bytes) 2024-09-17T07:29:30.377 INFO:teuthology.orchestra.run.smithi045.stdout:2483 punch from 0x7190d32 to 0x719b8df, (0xabad bytes) 2024-09-17T07:29:30.377 INFO:teuthology.orchestra.run.smithi045.stdout:2486 punch from 0xa04c1d to 0xa0c895, (0x7c78 bytes) 2024-09-17T07:29:30.377 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.377 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.377 INFO:teuthology.orchestra.run.smithi045.stdout:2489 writesame 0x3289b7f thru 0x3292b7e data_size 0x80(0x9000 bytes) 2024-09-17T07:29:30.382 INFO:teuthology.orchestra.run.smithi045.stdout:2490 read 0xa27ad75 thru 0xa287718 (0xc9a4 bytes) 2024-09-17T07:29:30.382 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.382 INFO:teuthology.orchestra.run.smithi045.stdout:2492 write 0x3364799 thru 0x3372853 (0xe0bb bytes) 2024-09-17T07:29:30.388 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.388 INFO:teuthology.orchestra.run.smithi045.stdout:2495 punch from 0x5d5ef5b to 0x5d6c9ff, (0xdaa4 bytes) 2024-09-17T07:29:30.388 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.388 INFO:teuthology.orchestra.run.smithi045.stdout:2497 writesame 0x864871e thru 0x86569d2 data_size 0x1(0xe2b5 bytes) 2024-09-17T07:29:30.406 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.406 INFO:teuthology.orchestra.run.smithi045.stdout:2499 read 0x123ee78 thru 0x1249713 (0xa89c bytes) 2024-09-17T07:29:30.406 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.413 INFO:teuthology.orchestra.run.smithi045.stdout:2501 write 0xe443908 thru 0xe44fc7d (0xc376 bytes) 2024-09-17T07:29:30.416 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:29:30.419 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.419 INFO:teuthology.orchestra.run.smithi045.stdout:2503 read 0xc88928f thru 0xc891491 (0x8203 bytes) 2024-09-17T07:29:30.420 INFO:teuthology.orchestra.run.smithi045.stdout:2504 trunc from 0xe44fc7e to 0xdc7d47c 2024-09-17T07:29:30.429 INFO:teuthology.orchestra.run.smithi045.stdout:2506 write 0xa3d315f thru 0xa3e2bab (0xfa4d bytes) 2024-09-17T07:29:30.436 INFO:teuthology.orchestra.run.smithi045.stdout:2507 trunc from 0xdc7d47c to 0xd74694b 2024-09-17T07:29:30.441 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.446 INFO:teuthology.orchestra.run.smithi045.stdout:2509 writesame 0xd79940e thru 0xd79bd8d data_size 0x40(0x2980 bytes) 2024-09-17T07:29:30.446 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.446 INFO:teuthology.orchestra.run.smithi045.stdout:2511 read 0x427d36f thru 0x4282e6e (0x5b00 bytes) 2024-09-17T07:29:30.446 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.446 INFO:teuthology.orchestra.run.smithi045.stdout:2513 punch from 0x1cfb0f9 to 0x1d000af, (0x4fb6 bytes) 2024-09-17T07:29:30.446 INFO:teuthology.orchestra.run.smithi045.stdout:2514 writesame 0x4ba3d7b thru 0x4bb317e data_size 0x2e(0xf404 bytes) 2024-09-17T07:29:30.449 INFO:teuthology.orchestra.run.smithi045.stdout:2517 write 0xb5ddf15 thru 0xb5dfac4 (0x1bb0 bytes) 2024-09-17T07:29:30.452 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.452 INFO:teuthology.orchestra.run.smithi045.stdout:2519 write 0x64186ad thru 0x641b88b (0x31df bytes) 2024-09-17T07:29:30.460 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:30.460+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:30.461 INFO:teuthology.orchestra.run.smithi045.stdout:2520 writesame 0xeb787f2 thru 0xeb86a15 data_size 0x52(0xe224 bytes) 2024-09-17T07:29:30.461 INFO:teuthology.orchestra.run.smithi045.stdout:2521 write 0xbff1ca3 thru 0xbff58bd (0x3c1b bytes) 2024-09-17T07:29:30.464 INFO:teuthology.orchestra.run.smithi045.stdout:2523 write 0xa5f334 thru 0xa6b39d (0xc06a bytes) 2024-09-17T07:29:30.474 INFO:teuthology.orchestra.run.smithi045.stdout:2524 write 0x79407db thru 0x7945e0e (0x5634 bytes) 2024-09-17T07:29:30.478 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.478 INFO:teuthology.orchestra.run.smithi045.stdout:2527 writesame 0x60b4dd7 thru 0x60c4c30 data_size 0x7(0xfe5a bytes) 2024-09-17T07:29:30.489 INFO:teuthology.orchestra.run.smithi045.stdout:2529 writesame 0x661d3c0 thru 0x661e4ab data_size 0x26(0x10ec bytes) 2024-09-17T07:29:30.492 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.493 INFO:teuthology.orchestra.run.smithi045.stdout:2531 read 0x3e2de6b thru 0x3e38e2b (0xafc1 bytes) 2024-09-17T07:29:30.493 INFO:teuthology.orchestra.run.smithi045.stdout:2532 write 0xdd5f63c thru 0xdd698e1 (0xa2a6 bytes) 2024-09-17T07:29:30.494 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:30.494 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:30.506 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:30.512 INFO:teuthology.orchestra.run.smithi045.stdout:2533 trunc from 0xeb86a16 to 0x360afd7 2024-09-17T07:29:30.537 INFO:teuthology.orchestra.run.smithi045.stdout:2536 trunc from 0x360afd7 to 0x7c1fa4a 2024-09-17T07:29:30.542 INFO:teuthology.orchestra.run.smithi045.stdout:2537 read 0x15ddf59 thru 0x15e71fc (0x92a4 bytes) 2024-09-17T07:29:30.542 INFO:teuthology.orchestra.run.smithi045.stdout:2539 writesame 0x23d9981 thru 0x23df016 data_size 0x1(0x5696 bytes) 2024-09-17T07:29:30.551 INFO:teuthology.orchestra.run.smithi045.stdout:2541 read 0x31de2e5 thru 0x31e486d (0x6589 bytes) 2024-09-17T07:29:30.552 INFO:teuthology.orchestra.run.smithi045.stdout:2543 read 0x5ae116c thru 0x5aec321 (0xb1b6 bytes) 2024-09-17T07:29:30.552 INFO:teuthology.orchestra.run.smithi045.stdout:2544 punch from 0x31a512a to 0x31a8723, (0x35f9 bytes) 2024-09-17T07:29:30.552 INFO:teuthology.orchestra.run.smithi045.stdout:2545 trunc from 0x7c1fa4a to 0x84fd937 2024-09-17T07:29:30.562 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:30.561+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:30.563 INFO:teuthology.orchestra.run.smithi045.stdout:2546 trunc from 0x84fd937 to 0xcfea3c0 2024-09-17T07:29:30.567 INFO:teuthology.orchestra.run.smithi045.stdout:2547 write 0x45e136f thru 0x45e6f88 (0x5c1a bytes) 2024-09-17T07:29:30.574 INFO:teuthology.orchestra.run.smithi045.stdout:2548 read 0x48d4ef8 thru 0x48e28b7 (0xd9c0 bytes) 2024-09-17T07:29:30.574 INFO:teuthology.orchestra.run.smithi045.stdout:2549 punch from 0xbe84586 to 0xbe88341, (0x3dbb bytes) 2024-09-17T07:29:30.575 INFO:teuthology.orchestra.run.smithi045.stdout:2551 trunc from 0xcfea3c0 to 0xd8f48f9 2024-09-17T07:29:30.577 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:30.577 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:30.580 INFO:teuthology.orchestra.run.smithi045.stdout:2553 write 0xa9935fa thru 0xa9a1f5a (0xe961 bytes) 2024-09-17T07:29:30.586 INFO:teuthology.orchestra.run.smithi045.stdout:2554 read 0x8208ade thru 0x8215f96 (0xd4b9 bytes) 2024-09-17T07:29:30.586 INFO:teuthology.orchestra.run.smithi045.stdout:2556 write 0x91d5475 thru 0x91dc188 (0x6d14 bytes) 2024-09-17T07:29:30.590 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.590 INFO:teuthology.orchestra.run.smithi045.stdout:2558 write 0xc3b7145 thru 0xc3bf54f (0x840b bytes) 2024-09-17T07:29:30.595 INFO:teuthology.orchestra.run.smithi045.stdout:2559 read 0x3b07a44 thru 0x3b09234 (0x17f1 bytes) 2024-09-17T07:29:30.595 INFO:teuthology.orchestra.run.smithi045.stdout:2563 read 0x8438f0d thru 0x843d4eb (0x45df bytes) 2024-09-17T07:29:30.596 INFO:teuthology.orchestra.run.smithi045.stdout:2564 write 0xc9e78f1 thru 0xc9ec1e3 (0x48f3 bytes) 2024-09-17T07:29:30.599 INFO:teuthology.orchestra.run.smithi045.stdout:2565 read 0x2d9d4db thru 0x2da71ec (0x9d12 bytes) 2024-09-17T07:29:30.600 INFO:teuthology.orchestra.run.smithi045.stdout:2566 read 0x3ebc46d thru 0x3ec9609 (0xd19d bytes) 2024-09-17T07:29:30.600 INFO:teuthology.orchestra.run.smithi045.stdout:2567 write 0x32b5f34 thru 0x32c5271 (0xf33e bytes) 2024-09-17T07:29:30.603 INFO:teuthology.orchestra.run.smithi045.stdout:2568 write 0x2655fef thru 0x265dc3f (0x7c51 bytes) 2024-09-17T07:29:30.607 INFO:teuthology.orchestra.run.smithi045.stdout:2569 write 0x1c9b99b thru 0x1ca3d22 (0x8388 bytes) 2024-09-17T07:29:30.610 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:30.616 INFO:teuthology.orchestra.run.smithi045.stdout:2570 punch from 0x8d59894 to 0x8d63880, (0x9fec bytes) 2024-09-17T07:29:30.616 INFO:teuthology.orchestra.run.smithi045.stdout:2571 read 0x7c07fd8 thru 0x7c10593 (0x85bc bytes) 2024-09-17T07:29:30.616 INFO:teuthology.orchestra.run.smithi045.stdout:2574 write 0xa835d0a thru 0xa83f465 (0x975c bytes) 2024-09-17T07:29:30.622 INFO:teuthology.orchestra.run.smithi045.stdout:2575 write 0x323fb2d thru 0x3245e9a (0x636e bytes) 2024-09-17T07:29:30.624 INFO:teuthology.orchestra.run.smithi045.stdout:2576 punch from 0x2c486c1 to 0x2c538f9, (0xb238 bytes) 2024-09-17T07:29:30.625 INFO:teuthology.orchestra.run.smithi045.stdout:2578 write 0xcee2d3d thru 0xcee9028 (0x62ec bytes) 2024-09-17T07:29:30.628 INFO:teuthology.orchestra.run.smithi045.stdout:2579 write 0x24bc2d3 thru 0x24c8d35 (0xca63 bytes) 2024-09-17T07:29:30.636 INFO:teuthology.orchestra.run.smithi045.stdout:2581 read 0x9a4cd64 thru 0x9a5b6be (0xe95b bytes) 2024-09-17T07:29:30.636 INFO:teuthology.orchestra.run.smithi045.stdout:2582 write 0x4382f31 thru 0x439078a (0xd85a bytes) 2024-09-17T07:29:30.645 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:30.646 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:30.646 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:30.662 INFO:teuthology.orchestra.run.smithi045.stdout:2583 trunc from 0xd8f48f9 to 0x10ea44 2024-09-17T07:29:30.662 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:30.662+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:30.662 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:30.663 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:30.663 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:30.663 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:30.665 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:30.665 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:30.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:30.690 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:30.690 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:30.716 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:30.717 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:30.734 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:30.737 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:30.737 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:30.740 INFO:teuthology.orchestra.run.smithi045.stdout:2584 punch from 0xe8496 to 0xf2767, (0xa2d1 bytes) 2024-09-17T07:29:30.755 INFO:teuthology.orchestra.run.smithi045.stdout:2585 write 0x8dbac1e thru 0x8dbccc2 (0x20a5 bytes) 2024-09-17T07:29:30.759 INFO:teuthology.orchestra.run.smithi045.stdout:2586 read 0xb60400 thru 0xb63846 (0x3447 bytes) 2024-09-17T07:29:30.759 INFO:teuthology.orchestra.run.smithi045.stdout:2587 punch from 0x59b0c9c to 0x59c023d, (0xf5a1 bytes) 2024-09-17T07:29:30.759 INFO:teuthology.orchestra.run.smithi045.stdout:2588 read 0x3538657 thru 0x35401c7 (0x7b71 bytes) 2024-09-17T07:29:30.759 INFO:teuthology.orchestra.run.smithi045.stdout:2589 read 0x2b1a2c1 thru 0x2b27fa0 (0xdce0 bytes) 2024-09-17T07:29:30.759 INFO:teuthology.orchestra.run.smithi045.stdout:2592 punch from 0x42c3db8 to 0x42c5bed, (0x1e35 bytes) 2024-09-17T07:29:30.759 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:30.759 INFO:teuthology.orchestra.run.smithi045.stdout:2594 write 0x378b0eb thru 0x3794525 (0x943b bytes) 2024-09-17T07:29:30.763 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:30.762+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:30.765 INFO:teuthology.orchestra.run.smithi045.stdout:2595 trunc from 0x8dbccc3 to 0x7f44e3c 2024-09-17T07:29:30.774 INFO:teuthology.orchestra.run.smithi045.stdout:2596 writesame 0x58a3c70 thru 0x58b19b9 data_size 0x37(0xdd4a bytes) 2024-09-17T07:29:30.781 INFO:teuthology.orchestra.run.smithi045.stdout:2597 read 0x718fb55 thru 0x7191d31 (0x21dd bytes) 2024-09-17T07:29:30.781 INFO:teuthology.orchestra.run.smithi045.stdout:2599 read 0x74bb83e thru 0x74c7e5b (0xc61e bytes) 2024-09-17T07:29:30.793 INFO:teuthology.orchestra.run.smithi045.stdout:2600 write 0xc9a1a3d thru 0xc9abe62 (0xa426 bytes) 2024-09-17T07:29:30.798 INFO:teuthology.orchestra.run.smithi045.stdout:2601 trunc from 0xc9abe63 to 0xc022733 2024-09-17T07:29:30.807 INFO:teuthology.orchestra.run.smithi045.stdout:2602 clone 27 order 23 su 1048576 sc 9 2024-09-17T07:29:30.815 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:30.815 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:30.815 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:30.815 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:30.815 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:30.824 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:30.866 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:30.865+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:30.965 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:30.964+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:30.973 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:4848 3.9 e33/23 3:97363921:::rbd_data.1183cff49e3a.0000000000000004:head v 33'102, mlcod=32'100)", 2024-09-17T07:29:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 36.731949016000002, 2024-09-17T07:29:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023715110000000002, 2024-09-17T07:29:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:30.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:30.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239497+0000", 2024-09-17T07:29:30.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.53e-06 2024-09-17T07:29:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240142+0000", 2024-09-17T07:29:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00064481300000000005 2024-09-17T07:29:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240160+0000", 2024-09-17T07:29:30.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8349999999999999e-05 2024-09-17T07:29:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240169+0000", 2024-09-17T07:29:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7690000000000002e-06 2024-09-17T07:29:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240246+0000", 2024-09-17T07:29:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6277000000000002e-05 2024-09-17T07:29:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240270+0000", 2024-09-17T07:29:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4575000000000001e-05 2024-09-17T07:29:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:30.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.241813+0000", 2024-09-17T07:29:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015432860000000001 2024-09-17T07:29:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.241867+0000", 2024-09-17T07:29:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3910999999999999e-05 2024-09-17T07:29:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:5460 3.9 e33/23 3:90773955:::rbd_data.1183cff49e3a.000000000000000a:head v 33'106, mlcod=33'104)", 2024-09-17T07:29:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:30.979 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 36.444852429999997, 2024-09-17T07:29:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020825850000000001, 2024-09-17T07:29:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:30.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526593+0000", 2024-09-17T07:29:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.54e-07 2024-09-17T07:29:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527050+0000", 2024-09-17T07:29:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000456522 2024-09-17T07:29:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527061+0000", 2024-09-17T07:29:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1722000000000001e-05 2024-09-17T07:29:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527067+0000", 2024-09-17T07:29:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0599999999999998e-06 2024-09-17T07:29:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527112+0000", 2024-09-17T07:29:30.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999000000000001e-05 2024-09-17T07:29:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527128+0000", 2024-09-17T07:29:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6920999999999999e-05 2024-09-17T07:29:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528636+0000", 2024-09-17T07:29:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001507836 2024-09-17T07:29:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:30.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528675+0000", 2024-09-17T07:29:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8671000000000003e-05 2024-09-17T07:29:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:6005 3.0 e33/23 3:037cd38f:::rbd_data.1183cff49e3a.0000000000000012:head v 33'25, mlcod=32'23)", 2024-09-17T07:29:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 36.346819296, 2024-09-17T07:29:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019123899999999999, 2024-09-17T07:29:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:30.984 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624627+0000", 2024-09-17T07:29:30.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5600000000000001e-06 2024-09-17T07:29:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624777+0000", 2024-09-17T07:29:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015030800000000001 2024-09-17T07:29:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624792+0000", 2024-09-17T07:29:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5089000000000001e-05 2024-09-17T07:29:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:30.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624799+0000", 2024-09-17T07:29:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7939999999999997e-06 2024-09-17T07:29:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624855+0000", 2024-09-17T07:29:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5782999999999998e-05 2024-09-17T07:29:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624871+0000", 2024-09-17T07:29:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6192e-05 2024-09-17T07:29:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:30.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.626507+0000", 2024-09-17T07:29:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016354500000000001 2024-09-17T07:29:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.626538+0000", 2024-09-17T07:29:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1214000000000003e-05 2024-09-17T07:29:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8595 3.4 e33/23 3:237b2a73:::rbd_data.1183cff49e3a.0000000000000049:head v 33'37, mlcod=33'35)", 2024-09-17T07:29:30.988 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.896802336999997, 2024-09-17T07:29:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020228939999999999, 2024-09-17T07:29:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074643+0000", 2024-09-17T07:29:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3099999999999997e-07 2024-09-17T07:29:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074834+0000", 2024-09-17T07:29:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000191137 2024-09-17T07:29:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074850+0000", 2024-09-17T07:29:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6101999999999999e-05 2024-09-17T07:29:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:30.991 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074859+0000", 2024-09-17T07:29:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2169999999999994e-06 2024-09-17T07:29:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074919+0000", 2024-09-17T07:29:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.003e-05 2024-09-17T07:29:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074944+0000", 2024-09-17T07:29:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5761e-05 2024-09-17T07:29:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076622+0000", 2024-09-17T07:29:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001677977 2024-09-17T07:29:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076665+0000", 2024-09-17T07:29:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2938999999999997e-05 2024-09-17T07:29:30.993 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17069 3.0 e41/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 41'110, mlcod=40'108)", 2024-09-17T07:29:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.231692613, 2024-09-17T07:29:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011408908000000001, 2024-09-17T07:29:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:30.994 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:30.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739753+0000", 2024-09-17T07:29:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:29:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743532+0000", 2024-09-17T07:29:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037794510000000001 2024-09-17T07:29:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743557+0000", 2024-09-17T07:29:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5253e-05 2024-09-17T07:29:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743569+0000", 2024-09-17T07:29:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1161999999999999e-05 2024-09-17T07:29:30.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743650+0000", 2024-09-17T07:29:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1087999999999993e-05 2024-09-17T07:29:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743674+0000", 2024-09-17T07:29:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4562e-05 2024-09-17T07:29:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751095+0000", 2024-09-17T07:29:30.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074211140000000004 2024-09-17T07:29:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751161+0000", 2024-09-17T07:29:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5707e-05 2024-09-17T07:29:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17240 3.0 e42/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 42'113, mlcod=41'110)", 2024-09-17T07:29:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 27.746034129000002, 2024-09-17T07:29:30.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064392440000000002, 2024-09-17T07:29:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:29:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225418+0000", 2024-09-17T07:29:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8360000000000003e-06 2024-09-17T07:29:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225426+0000", 2024-09-17T07:29:31.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2670000000000006e-06 2024-09-17T07:29:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225434+0000", 2024-09-17T07:29:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0560000000000005e-06 2024-09-17T07:29:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225514+0000", 2024-09-17T07:29:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9979000000000001e-05 2024-09-17T07:29:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225535+0000", 2024-09-17T07:29:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0965e-05 2024-09-17T07:29:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231797+0000", 2024-09-17T07:29:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062621129999999997 2024-09-17T07:29:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231850+0000", 2024-09-17T07:29:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2611999999999997e-05 2024-09-17T07:29:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20054 3.9 e44/23 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:head v 44'131, mlcod=44'129)", 2024-09-17T07:29:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 25.703883012999999, 2024-09-17T07:29:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028510570000000002, 2024-09-17T07:29:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267563+0000", 2024-09-17T07:29:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.584e-06 2024-09-17T07:29:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268269+0000", 2024-09-17T07:29:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00070567399999999999 2024-09-17T07:29:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268287+0000", 2024-09-17T07:29:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.84e-05 2024-09-17T07:29:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268297+0000", 2024-09-17T07:29:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9059999999999997e-06 2024-09-17T07:29:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268368+0000", 2024-09-17T07:29:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0562999999999995e-05 2024-09-17T07:29:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268393+0000", 2024-09-17T07:29:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4958999999999999e-05 2024-09-17T07:29:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270330+0000", 2024-09-17T07:29:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001936644 2024-09-17T07:29:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270413+0000", 2024-09-17T07:29:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3326999999999997e-05 2024-09-17T07:29:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.008 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20055 3.9 e44/23 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:head v 44'133, mlcod=44'129)", 2024-09-17T07:29:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.268886+0000", 2024-09-17T07:29:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 25.702558942, 2024-09-17T07:29:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019232920000000001, 2024-09-17T07:29:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268886+0000", 2024-09-17T07:29:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268886+0000", 2024-09-17T07:29:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268887+0000", 2024-09-17T07:29:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.85e-07 2024-09-17T07:29:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268905+0000", 2024-09-17T07:29:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8263000000000001e-05 2024-09-17T07:29:31.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268916+0000", 2024-09-17T07:29:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0876000000000001e-05 2024-09-17T07:29:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268922+0000", 2024-09-17T07:29:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7579999999999998e-06 2024-09-17T07:29:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:31.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.269201+0000", 2024-09-17T07:29:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027979200000000001 2024-09-17T07:29:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.269224+0000", 2024-09-17T07:29:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2838999999999999e-05 2024-09-17T07:29:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270774+0000", 2024-09-17T07:29:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015495629999999999 2024-09-17T07:29:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270809+0000", 2024-09-17T07:29:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5416000000000001e-05 2024-09-17T07:29:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20163 3.9 e44/23 3:9772d386:::rbd_data.1183ce4061a0.0000000000000025:head v 44'134, mlcod=44'131)", 2024-09-17T07:29:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.294194+0000", 2024-09-17T07:29:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 25.677250591, 2024-09-17T07:29:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001874824, 2024-09-17T07:29:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:31.013 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:31.014 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:31.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:31.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294194+0000", 2024-09-17T07:29:31.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:31.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294194+0000", 2024-09-17T07:29:31.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:31.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294196+0000", 2024-09-17T07:29:31.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3599999999999999e-06 2024-09-17T07:29:31.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.015 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:31.015 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294663+0000", 2024-09-17T07:29:31.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00046696200000000002 2024-09-17T07:29:31.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.015 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:31.015 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294683+0000", 2024-09-17T07:29:31.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0829999999999999e-05 2024-09-17T07:29:31.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.015 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294697+0000", 2024-09-17T07:29:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3524e-05 2024-09-17T07:29:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294758+0000", 2024-09-17T07:29:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0618999999999997e-05 2024-09-17T07:29:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.294776+0000", 2024-09-17T07:29:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8431e-05 2024-09-17T07:29:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.296042+0000", 2024-09-17T07:29:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012656329999999999 2024-09-17T07:29:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.296069+0000", 2024-09-17T07:29:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7464999999999999e-05 2024-09-17T07:29:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.1f 51/50 [PushOp(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head, version: 40'83, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head@40'83, size: 19583394, copy_subset: [(0, 19583394)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:29:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:13.339735+0000", 2024-09-17T07:29:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.631710244000001, 2024-09-17T07:29:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0018873760000000001, 2024-09-17T07:29:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.339735+0000", 2024-09-17T07:29:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.339735+0000", 2024-09-17T07:29:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.339736+0000", 2024-09-17T07:29:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.392e-06 2024-09-17T07:29:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340637+0000", 2024-09-17T07:29:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00090133599999999998 2024-09-17T07:29:31.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340668+0000", 2024-09-17T07:29:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0488e-05 2024-09-17T07:29:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340683+0000", 2024-09-17T07:29:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5315000000000002e-05 2024-09-17T07:29:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:31.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340777+0000", 2024-09-17T07:29:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3456999999999996e-05 2024-09-17T07:29:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340786+0000", 2024-09-17T07:29:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2220000000000008e-06 2024-09-17T07:29:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.341622+0000", 2024-09-17T07:29:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083616599999999995 2024-09-17T07:29:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.021 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.1f 51/50 [PushOp(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head, version: 40'83, data_included: [3407872~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head@40'83, size: 19583394, copy_subset: [(0, 19583394)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4456448, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3407872, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:29:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:13.350296+0000", 2024-09-17T07:29:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.621149298999999, 2024-09-17T07:29:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00191949, 2024-09-17T07:29:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:31.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.350296+0000", 2024-09-17T07:29:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.350296+0000", 2024-09-17T07:29:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.350297+0000", 2024-09-17T07:29:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4440000000000001e-06 2024-09-17T07:29:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351304+0000", 2024-09-17T07:29:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001007319 2024-09-17T07:29:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351331+0000", 2024-09-17T07:29:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6701000000000001e-05 2024-09-17T07:29:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351346+0000", 2024-09-17T07:29:31.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4911e-05 2024-09-17T07:29:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351425+0000", 2024-09-17T07:29:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9328999999999999e-05 2024-09-17T07:29:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351436+0000", 2024-09-17T07:29:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0467e-05 2024-09-17T07:29:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:31.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.352215+0000", 2024-09-17T07:29:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000779319 2024-09-17T07:29:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:32543 3.0 e55/23 3:03bfd97d:::rbd_data.11839a97b753.0000000000000007:head v 55'150, mlcod=54'148)", 2024-09-17T07:29:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.516807876, 2024-09-17T07:29:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031813480000000001, 2024-09-17T07:29:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:31.026 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5199999999999998e-07 2024-09-17T07:29:31.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455083+0000", 2024-09-17T07:29:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044629199999999999 2024-09-17T07:29:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455098+0000", 2024-09-17T07:29:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4183e-05 2024-09-17T07:29:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455106+0000", 2024-09-17T07:29:31.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:29:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455173+0000", 2024-09-17T07:29:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6346000000000005e-05 2024-09-17T07:29:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455200+0000", 2024-09-17T07:29:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7319000000000001e-05 2024-09-17T07:29:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:31.029 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457774+0000", 2024-09-17T07:29:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025742650000000001 2024-09-17T07:29:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457818+0000", 2024-09-17T07:29:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3891999999999999e-05 2024-09-17T07:29:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33735 3.1f e55/52 3:fea54097:::rbd_data.11839a97b753.0000000000000000:head v 55'260, mlcod=55'258)", 2024-09-17T07:29:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:29:31.030 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.146875248000001, 2024-09-17T07:29:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035215200000000002, 2024-09-17T07:29:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:29:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.031 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:29:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824571+0000", 2024-09-17T07:29:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.164e-06 2024-09-17T07:29:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825118+0000", 2024-09-17T07:29:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054696400000000004 2024-09-17T07:29:31.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825134+0000", 2024-09-17T07:29:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6342999999999999e-05 2024-09-17T07:29:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825142+0000", 2024-09-17T07:29:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5270000000000001e-06 2024-09-17T07:29:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825200+0000", 2024-09-17T07:29:31.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.893e-05 2024-09-17T07:29:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825218+0000", 2024-09-17T07:29:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7935999999999999e-05 2024-09-17T07:29:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828037+0000", 2024-09-17T07:29:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002818961 2024-09-17T07:29:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:31.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828091+0000", 2024-09-17T07:29:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3695e-05 2024-09-17T07:29:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33869 3.9 e55/23 3:907b81b5:::rbd_data.11839a97b753.0000000000000001:head v 55'171, mlcod=55'169)", 2024-09-17T07:29:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:29:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.129819425000001, 2024-09-17T07:29:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033990859999999999, 2024-09-17T07:29:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:31.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:29:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:29:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841626+0000", 2024-09-17T07:29:31.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.023e-06 2024-09-17T07:29:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842067+0000", 2024-09-17T07:29:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000440612 2024-09-17T07:29:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842080+0000", 2024-09-17T07:29:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.29e-05 2024-09-17T07:29:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842086+0000", 2024-09-17T07:29:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2090000000000002e-06 2024-09-17T07:29:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842144+0000", 2024-09-17T07:29:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7627999999999998e-05 2024-09-17T07:29:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842171+0000", 2024-09-17T07:29:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7214000000000001e-05 2024-09-17T07:29:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.844973+0000", 2024-09-17T07:29:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002802232 2024-09-17T07:29:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.845024+0000", 2024-09-17T07:29:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1267999999999998e-05 2024-09-17T07:29:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51432 3.0 e59/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 59'177, mlcod=58'174)", 2024-09-17T07:29:31.039 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:29:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 8.1502224079999994, 2024-09-17T07:29:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010790086000000001, 2024-09-17T07:29:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:29:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:31.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:29:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821224+0000", 2024-09-17T07:29:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.511e-06 2024-09-17T07:29:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825174+0000", 2024-09-17T07:29:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0039497949999999999 2024-09-17T07:29:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:31.041 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825196+0000", 2024-09-17T07:29:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2643999999999999e-05 2024-09-17T07:29:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825206+0000", 2024-09-17T07:29:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4320000000000005e-06 2024-09-17T07:29:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825272+0000", 2024-09-17T07:29:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5829999999999998e-05 2024-09-17T07:29:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825299+0000", 2024-09-17T07:29:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7107e-05 2024-09-17T07:29:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.831939+0000", 2024-09-17T07:29:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0066398510000000004 2024-09-17T07:29:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.832012+0000", 2024-09-17T07:29:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3916000000000002e-05 2024-09-17T07:29:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.043 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51969 3.0 e59/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 59'179, mlcod=59'177)", 2024-09-17T07:29:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:29:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 8.0413540300000008, 2024-09-17T07:29:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060297249999999997, 2024-09-17T07:29:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:29:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:29:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930093+0000", 2024-09-17T07:29:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.764e-06 2024-09-17T07:29:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931779+0000", 2024-09-17T07:29:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001686379 2024-09-17T07:29:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931802+0000", 2024-09-17T07:29:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2682000000000001e-05 2024-09-17T07:29:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931811+0000", 2024-09-17T07:29:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5640000000000002e-06 2024-09-17T07:29:31.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931877+0000", 2024-09-17T07:29:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6155999999999995e-05 2024-09-17T07:29:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931896+0000", 2024-09-17T07:29:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9111e-05 2024-09-17T07:29:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936063+0000", 2024-09-17T07:29:31.047 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004166711 2024-09-17T07:29:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936121+0000", 2024-09-17T07:29:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7358000000000002e-05 2024-09-17T07:29:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54417 3.0 e60/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 60'182, mlcod=59'179)", 2024-09-17T07:29:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:29:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 7.5338370389999998, 2024-09-17T07:29:31.048 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0059885219999999996, 2024-09-17T07:29:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:29:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:29:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437609+0000", 2024-09-17T07:29:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5829999999999999e-06 2024-09-17T07:29:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437623+0000", 2024-09-17T07:29:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3348e-05 2024-09-17T07:29:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437643+0000", 2024-09-17T07:29:31.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0463999999999999e-05 2024-09-17T07:29:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437653+0000", 2024-09-17T07:29:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6290000000000005e-06 2024-09-17T07:29:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437719+0000", 2024-09-17T07:29:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6428999999999997e-05 2024-09-17T07:29:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:31.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437745+0000", 2024-09-17T07:29:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5690999999999998e-05 2024-09-17T07:29:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443549+0000", 2024-09-17T07:29:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0058038259999999998 2024-09-17T07:29:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443596+0000", 2024-09-17T07:29:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7552000000000001e-05 2024-09-17T07:29:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.052 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54422 3.1f e60/52 3:fef8599a:::rbd_data.1183a58293a3.0000000000000007:head v 60'280, mlcod=58'277)", 2024-09-17T07:29:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.441086+0000", 2024-09-17T07:29:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 7.5303586639999995, 2024-09-17T07:29:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001916933, 2024-09-17T07:29:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441086+0000", 2024-09-17T07:29:31.053 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441086+0000", 2024-09-17T07:29:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441088+0000", 2024-09-17T07:29:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.517e-06 2024-09-17T07:29:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:31.054 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441100+0000", 2024-09-17T07:29:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2532e-05 2024-09-17T07:29:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441118+0000", 2024-09-17T07:29:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7798000000000001e-05 2024-09-17T07:29:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441127+0000", 2024-09-17T07:29:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4749999999999999e-06 2024-09-17T07:29:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.055 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441675+0000", 2024-09-17T07:29:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054747699999999999 2024-09-17T07:29:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441693+0000", 2024-09-17T07:29:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7621e-05 2024-09-17T07:29:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.442956+0000", 2024-09-17T07:29:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012633900000000001 2024-09-17T07:29:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443003+0000", 2024-09-17T07:29:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7123000000000003e-05 2024-09-17T07:29:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54419 3.0 e60/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 60'184, mlcod=59'179)", 2024-09-17T07:29:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:29:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 7.5274846819999999, 2024-09-17T07:29:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004706844, 2024-09-17T07:29:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:31.057 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:29:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:29:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:31.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443961+0000", 2024-09-17T07:29:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:29:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443967+0000", 2024-09-17T07:29:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.268e-06 2024-09-17T07:29:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443983+0000", 2024-09-17T07:29:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5248999999999999e-05 2024-09-17T07:29:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443990+0000", 2024-09-17T07:29:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5120000000000002e-06 2024-09-17T07:29:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444056+0000", 2024-09-17T07:29:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6197000000000003e-05 2024-09-17T07:29:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444081+0000", 2024-09-17T07:29:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4791e-05 2024-09-17T07:29:31.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448616+0000", 2024-09-17T07:29:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045346129999999998 2024-09-17T07:29:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448667+0000", 2024-09-17T07:29:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-17T07:29:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.061 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54895 3.1f e60/52 3:fa1c1177:::rbd_data.1183ad87379e.0000000000000001:head v 60'283, mlcod=60'281)", 2024-09-17T07:29:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:29:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 7.1800036860000001, 2024-09-17T07:29:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0029495210000000001, 2024-09-17T07:29:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:29:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:29:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791443+0000", 2024-09-17T07:29:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.153e-06 2024-09-17T07:29:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792619+0000", 2024-09-17T07:29:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001176011 2024-09-17T07:29:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792637+0000", 2024-09-17T07:29:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-05 2024-09-17T07:29:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792649+0000", 2024-09-17T07:29:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1712000000000001e-05 2024-09-17T07:29:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792722+0000", 2024-09-17T07:29:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3545000000000001e-05 2024-09-17T07:29:31.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792738+0000", 2024-09-17T07:29:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5223e-05 2024-09-17T07:29:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.794330+0000", 2024-09-17T07:29:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015922740000000001 2024-09-17T07:29:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.794391+0000", 2024-09-17T07:29:31.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0803e-05 2024-09-17T07:29:31.066 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.066 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.066 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.066 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:31.066 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:31.066 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:31.067 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:31.068 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:31.067+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:31.169 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:31.168+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:31.172 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:31.172 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:31.172 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:31.172 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:31.180 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:31.269 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:31.269+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:31.287 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-17T07:29:31.329 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:31.329 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:31.329 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:31.329 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:31.329 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:31.338 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:31.351 INFO:teuthology.orchestra.run.smithi045.stdout:flattening image image_client.0-clone26 2024-09-17T07:29:31.380 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:31.379+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:31.472 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:31.471+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:31.484 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-17T07:29:31.487 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:31.487 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:31.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:31.488 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:31.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.488 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:30525 3.1b 3:def7c8f0:::rbd_data.118381ffbbab.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T07:29:31.488 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:31.488 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 17.964164783000001, 2024-09-17T07:29:31.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55515850300000003, 2024-09-17T07:29:31.488 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.488 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.488 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:31.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.489 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30525 2024-09-17T07:29:31.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.489 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:31.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:31.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522300+0000", 2024-09-17T07:29:31.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:29:31.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522305+0000", 2024-09-17T07:29:31.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-17T07:29:31.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522318+0000", 2024-09-17T07:29:31.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3030999999999999e-05 2024-09-17T07:29:31.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522326+0000", 2024-09-17T07:29:31.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4200000000000001e-06 2024-09-17T07:29:31.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522385+0000", 2024-09-17T07:29:31.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9005000000000002e-05 2024-09-17T07:29:31.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:29:31.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522393+0000", 2024-09-17T07:29:31.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-06 2024-09-17T07:29:31.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075347+0000", 2024-09-17T07:29:31.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55295417800000002 2024-09-17T07:29:31.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075502+0000", 2024-09-17T07:29:31.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154272 2024-09-17T07:29:31.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:31.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075630+0000", 2024-09-17T07:29:31.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012816300000000001 2024-09-17T07:29:31.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:31.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077221+0000", 2024-09-17T07:29:31.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015912929999999999 2024-09-17T07:29:31.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:31.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077353+0000", 2024-09-17T07:29:31.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013215699999999999 2024-09-17T07:29:31.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:31.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077413+0000", 2024-09-17T07:29:31.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0034999999999997e-05 2024-09-17T07:29:31.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077457+0000", 2024-09-17T07:29:31.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4017999999999998e-05 2024-09-17T07:29:31.495 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.495 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.495 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.495 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44181 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13922304~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.495 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:29:31.495 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8489861350000005, 2024-09-17T07:29:31.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046010502000000002, 2024-09-17T07:29:31.495 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.496 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.496 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.496 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.496 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.496 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.496 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44181 2024-09-17T07:29:31.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.496 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:29:31.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:29:31.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637478+0000", 2024-09-17T07:29:31.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:29:31.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637480+0000", 2024-09-17T07:29:31.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.843e-06 2024-09-17T07:29:31.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637487+0000", 2024-09-17T07:29:31.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6159999999999996e-06 2024-09-17T07:29:31.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637492+0000", 2024-09-17T07:29:31.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5360000000000003e-06 2024-09-17T07:29:31.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683351+0000", 2024-09-17T07:29:31.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045858758999999999 2024-09-17T07:29:31.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683422+0000", 2024-09-17T07:29:31.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1705999999999994e-05 2024-09-17T07:29:31.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683488+0000", 2024-09-17T07:29:31.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5612999999999997e-05 2024-09-17T07:29:31.500 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.500 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.500 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.500 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44182 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13938688~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.500 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:31.500 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8489654639999991, 2024-09-17T07:29:31.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046153118999999999, 2024-09-17T07:29:31.501 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.501 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.501 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.501 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44182 2024-09-17T07:29:31.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.501 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:31.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:31.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:31.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:29:31.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637506+0000", 2024-09-17T07:29:31.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1520000000000003e-06 2024-09-17T07:29:31.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637512+0000", 2024-09-17T07:29:31.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-09-17T07:29:31.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637517+0000", 2024-09-17T07:29:31.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-09-17T07:29:31.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683514+0000", 2024-09-17T07:29:31.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045997616999999998 2024-09-17T07:29:31.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683584+0000", 2024-09-17T07:29:31.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9942000000000002e-05 2024-09-17T07:29:31.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683651+0000", 2024-09-17T07:29:31.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6951000000000003e-05 2024-09-17T07:29:31.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.505 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44183 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13955072~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.505 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:29:31.505 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8489402320000003, 2024-09-17T07:29:31.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046339793999999997, 2024-09-17T07:29:31.505 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.506 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.506 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.506 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44183 2024-09-17T07:29:31.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.506 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:29:31.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:29:31.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639583+0000", 2024-09-17T07:29:31.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002059412 2024-09-17T07:29:31.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639597+0000", 2024-09-17T07:29:31.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3832000000000001e-05 2024-09-17T07:29:31.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639607+0000", 2024-09-17T07:29:31.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485e-05 2024-09-17T07:29:31.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639614+0000", 2024-09-17T07:29:31.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0849999999999999e-06 2024-09-17T07:29:31.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683705+0000", 2024-09-17T07:29:31.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044091261 2024-09-17T07:29:31.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683792+0000", 2024-09-17T07:29:31.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6197000000000002e-05 2024-09-17T07:29:31.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683863+0000", 2024-09-17T07:29:31.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1521999999999997e-05 2024-09-17T07:29:31.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.510 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.510 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44199 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14217216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.511 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:31.511 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8454226800000004, 2024-09-17T07:29:31.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045995623999999999, 2024-09-17T07:29:31.511 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.511 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.511 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.512 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44199 2024-09-17T07:29:31.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.512 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:31.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:31.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:31.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:29:31.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641043+0000", 2024-09-17T07:29:31.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.624e-06 2024-09-17T07:29:31.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641048+0000", 2024-09-17T07:29:31.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4029999999999998e-06 2024-09-17T07:29:31.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641053+0000", 2024-09-17T07:29:31.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6670000000000004e-06 2024-09-17T07:29:31.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686918+0000", 2024-09-17T07:29:31.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045865131000000003 2024-09-17T07:29:31.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686986+0000", 2024-09-17T07:29:31.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8299000000000005e-05 2024-09-17T07:29:31.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687036+0000", 2024-09-17T07:29:31.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0164999999999998e-05 2024-09-17T07:29:31.515 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.515 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.515 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44200 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14233600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8454048509999996, 2024-09-17T07:29:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046183984999999997, 2024-09-17T07:29:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44200 2024-09-17T07:29:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.517 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:31.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:31.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:31.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:31.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641061+0000", 2024-09-17T07:29:31.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.767e-06 2024-09-17T07:29:31.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641067+0000", 2024-09-17T07:29:31.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7379999999999999e-06 2024-09-17T07:29:31.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641071+0000", 2024-09-17T07:29:31.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-09-17T07:29:31.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687078+0000", 2024-09-17T07:29:31.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046007095999999997 2024-09-17T07:29:31.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687158+0000", 2024-09-17T07:29:31.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0374e-05 2024-09-17T07:29:31.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687243+0000", 2024-09-17T07:29:31.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4221999999999994e-05 2024-09-17T07:29:31.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.520 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.521 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44201 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14249984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.521 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:29:31.521 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8453872049999998, 2024-09-17T07:29:31.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046303182999999998, 2024-09-17T07:29:31.521 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.521 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.521 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.521 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44201 2024-09-17T07:29:31.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.521 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:29:31.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:29:31.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641120+0000", 2024-09-17T07:29:31.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3933000000000001e-05 2024-09-17T07:29:31.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641123+0000", 2024-09-17T07:29:31.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.191e-06 2024-09-17T07:29:31.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641133+0000", 2024-09-17T07:29:31.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4020000000000008e-06 2024-09-17T07:29:31.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641138+0000", 2024-09-17T07:29:31.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6559999999999997e-06 2024-09-17T07:29:31.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687279+0000", 2024-09-17T07:29:31.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046140278999999999 2024-09-17T07:29:31.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687338+0000", 2024-09-17T07:29:31.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9268999999999998e-05 2024-09-17T07:29:31.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687379+0000", 2024-09-17T07:29:31.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1452999999999998e-05 2024-09-17T07:29:31.525 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.525 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.525 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.525 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44202 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14266368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.525 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:29:31.525 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8453164389999994, 2024-09-17T07:29:31.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046484299, 2024-09-17T07:29:31.526 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.526 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.526 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.526 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44202 2024-09-17T07:29:31.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.526 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:29:31.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:29:31.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642810+0000", 2024-09-17T07:29:31.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016627980000000001 2024-09-17T07:29:31.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642815+0000", 2024-09-17T07:29:31.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.091e-06 2024-09-17T07:29:31.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642829+0000", 2024-09-17T07:29:31.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4047e-05 2024-09-17T07:29:31.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642844+0000", 2024-09-17T07:29:31.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4515000000000001e-05 2024-09-17T07:29:31.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687426+0000", 2024-09-17T07:29:31.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044582201000000002 2024-09-17T07:29:31.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687529+0000", 2024-09-17T07:29:31.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103078 2024-09-17T07:29:31.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687631+0000", 2024-09-17T07:29:31.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102569 2024-09-17T07:29:31.530 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.530 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.530 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.530 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44209 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14381056~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.530 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:29:31.530 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8434182870000004, 2024-09-17T07:29:31.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045944444000000001, 2024-09-17T07:29:31.530 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.530 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.530 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.531 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.531 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.531 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44209 2024-09-17T07:29:31.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.531 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:29:31.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:29:31.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643046+0000", 2024-09-17T07:29:31.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:29:31.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643048+0000", 2024-09-17T07:29:31.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:29:31.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643054+0000", 2024-09-17T07:29:31.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.032e-06 2024-09-17T07:29:31.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643065+0000", 2024-09-17T07:29:31.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1861999999999999e-05 2024-09-17T07:29:31.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688806+0000", 2024-09-17T07:29:31.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045740617999999997 2024-09-17T07:29:31.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688927+0000", 2024-09-17T07:29:31.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000121327 2024-09-17T07:29:31.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688990+0000", 2024-09-17T07:29:31.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2293999999999997e-05 2024-09-17T07:29:31.534 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.534 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.534 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.535 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44210 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14397440~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.535 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:29:31.535 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8433913610000001, 2024-09-17T07:29:31.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046104635999999997, 2024-09-17T07:29:31.535 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.535 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.535 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.535 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44210 2024-09-17T07:29:31.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.536 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:29:31.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:29:31.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643073+0000", 2024-09-17T07:29:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.77e-07 2024-09-17T07:29:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643075+0000", 2024-09-17T07:29:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-09-17T07:29:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643082+0000", 2024-09-17T07:29:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7789999999999996e-06 2024-09-17T07:29:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643087+0000", 2024-09-17T07:29:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.967e-06 2024-09-17T07:29:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689022+0000", 2024-09-17T07:29:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045934349999999999 2024-09-17T07:29:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689112+0000", 2024-09-17T07:29:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0039999999999999e-05 2024-09-17T07:29:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689177+0000", 2024-09-17T07:29:31.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5041000000000003e-05 2024-09-17T07:29:31.539 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.539 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.539 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.539 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44211 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14413824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.539 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:29:31.539 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8433702279999995, 2024-09-17T07:29:31.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046309003000000001, 2024-09-17T07:29:31.540 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.540 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.540 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.540 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.540 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.540 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.540 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44211 2024-09-17T07:29:31.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.540 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:29:31.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:29:31.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643094+0000", 2024-09-17T07:29:31.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:29:31.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643095+0000", 2024-09-17T07:29:31.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-17T07:29:31.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643101+0000", 2024-09-17T07:29:31.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8730000000000004e-06 2024-09-17T07:29:31.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643106+0000", 2024-09-17T07:29:31.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6859999999999997e-06 2024-09-17T07:29:31.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689220+0000", 2024-09-17T07:29:31.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046113945000000003 2024-09-17T07:29:31.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689309+0000", 2024-09-17T07:29:31.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9560000000000003e-05 2024-09-17T07:29:31.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689402+0000", 2024-09-17T07:29:31.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2786000000000002e-05 2024-09-17T07:29:31.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.544 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.544 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44212 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14430208~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.544 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:31.544 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8433256660000001, 2024-09-17T07:29:31.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046448912000000002, 2024-09-17T07:29:31.544 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.544 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.545 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.545 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44212 2024-09-17T07:29:31.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.545 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:31.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:31.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:31.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2200000000000004e-07 2024-09-17T07:29:31.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643141+0000", 2024-09-17T07:29:31.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5210000000000001e-06 2024-09-17T07:29:31.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643149+0000", 2024-09-17T07:29:31.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6490000000000008e-06 2024-09-17T07:29:31.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643161+0000", 2024-09-17T07:29:31.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2241e-05 2024-09-17T07:29:31.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689423+0000", 2024-09-17T07:29:31.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046262323000000001 2024-09-17T07:29:31.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689505+0000", 2024-09-17T07:29:31.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2103999999999999e-05 2024-09-17T07:29:31.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689587+0000", 2024-09-17T07:29:31.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1452000000000006e-05 2024-09-17T07:29:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44213 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14446592~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:29:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8432941669999998, 2024-09-17T07:29:31.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046601780000000002, 2024-09-17T07:29:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44213 2024-09-17T07:29:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:29:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:29:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643170+0000", 2024-09-17T07:29:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-17T07:29:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643172+0000", 2024-09-17T07:29:31.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1660000000000001e-06 2024-09-17T07:29:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643180+0000", 2024-09-17T07:29:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-09-17T07:29:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643185+0000", 2024-09-17T07:29:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-09-17T07:29:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689622+0000", 2024-09-17T07:29:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046436742000000003 2024-09-17T07:29:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689697+0000", 2024-09-17T07:29:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5179999999999995e-05 2024-09-17T07:29:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689771+0000", 2024-09-17T07:29:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3928000000000001e-05 2024-09-17T07:29:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.553 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44214 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14462976~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:29:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8432711069999996, 2024-09-17T07:29:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046736038000000001, 2024-09-17T07:29:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.554 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44214 2024-09-17T07:29:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:29:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:29:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643193+0000", 2024-09-17T07:29:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:29:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643195+0000", 2024-09-17T07:29:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0640000000000001e-06 2024-09-17T07:29:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643201+0000", 2024-09-17T07:29:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2859999999999996e-06 2024-09-17T07:29:31.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643206+0000", 2024-09-17T07:29:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-17T07:29:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689814+0000", 2024-09-17T07:29:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046607742000000001 2024-09-17T07:29:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689886+0000", 2024-09-17T07:29:31.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2020999999999994e-05 2024-09-17T07:29:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689928+0000", 2024-09-17T07:29:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2771000000000001e-05 2024-09-17T07:29:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44215 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14479360~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:29:31.558 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8432272100000002, 2024-09-17T07:29:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046868543999999998, 2024-09-17T07:29:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44215 2024-09-17T07:29:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:29:31.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:29:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643237+0000", 2024-09-17T07:29:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:29:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643245+0000", 2024-09-17T07:29:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3270000000000001e-06 2024-09-17T07:29:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643254+0000", 2024-09-17T07:29:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6049999999999998e-06 2024-09-17T07:29:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643260+0000", 2024-09-17T07:29:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7849999999999997e-06 2024-09-17T07:29:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689961+0000", 2024-09-17T07:29:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046701196 2024-09-17T07:29:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690036+0000", 2024-09-17T07:29:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5153000000000004e-05 2024-09-17T07:29:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690105+0000", 2024-09-17T07:29:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8874000000000005e-05 2024-09-17T07:29:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.562 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44216 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14495744~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:29:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.843195497, 2024-09-17T07:29:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047003047999999999, 2024-09-17T07:29:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44216 2024-09-17T07:29:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:29:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:29:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643269+0000", 2024-09-17T07:29:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:29:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643271+0000", 2024-09-17T07:29:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-09-17T07:29:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643278+0000", 2024-09-17T07:29:31.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8850000000000001e-06 2024-09-17T07:29:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643282+0000", 2024-09-17T07:29:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5979999999999999e-06 2024-09-17T07:29:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690133+0000", 2024-09-17T07:29:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850470999999998 2024-09-17T07:29:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690205+0000", 2024-09-17T07:29:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2442999999999997e-05 2024-09-17T07:29:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690271+0000", 2024-09-17T07:29:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5874e-05 2024-09-17T07:29:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44217 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14512128~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.567 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8431744850000005, 2024-09-17T07:29:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047166485000000001, 2024-09-17T07:29:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44217 2024-09-17T07:29:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-17T07:29:31.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643291+0000", 2024-09-17T07:29:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.962e-06 2024-09-17T07:29:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643297+0000", 2024-09-17T07:29:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0700000000000003e-06 2024-09-17T07:29:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643302+0000", 2024-09-17T07:29:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:29:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690308+0000", 2024-09-17T07:29:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047005354999999999 2024-09-17T07:29:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690385+0000", 2024-09-17T07:29:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7541999999999997e-05 2024-09-17T07:29:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690455+0000", 2024-09-17T07:29:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0328999999999997e-05 2024-09-17T07:29:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44218 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14528512~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8431548079999995, 2024-09-17T07:29:31.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047321015000000001, 2024-09-17T07:29:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44218 2024-09-17T07:29:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:31.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-17T07:29:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643311+0000", 2024-09-17T07:29:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.933e-06 2024-09-17T07:29:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643318+0000", 2024-09-17T07:29:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3980000000000004e-06 2024-09-17T07:29:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643322+0000", 2024-09-17T07:29:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7609999999999998e-06 2024-09-17T07:29:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690484+0000", 2024-09-17T07:29:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047161428999999998 2024-09-17T07:29:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690564+0000", 2024-09-17T07:29:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9845000000000005e-05 2024-09-17T07:29:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690630+0000", 2024-09-17T07:29:31.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6021999999999999e-05 2024-09-17T07:29:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44219 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14544896~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8431344410000001, 2024-09-17T07:29:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048128708999999999, 2024-09-17T07:29:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.577 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44219 2024-09-17T07:29:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:29:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643337+0000", 2024-09-17T07:29:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.109e-06 2024-09-17T07:29:31.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643350+0000", 2024-09-17T07:29:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3046e-05 2024-09-17T07:29:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643356+0000", 2024-09-17T07:29:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8930000000000002e-06 2024-09-17T07:29:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690660+0000", 2024-09-17T07:29:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047304061000000001 2024-09-17T07:29:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691372+0000", 2024-09-17T07:29:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00071281299999999997 2024-09-17T07:29:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691458+0000", 2024-09-17T07:29:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5352000000000005e-05 2024-09-17T07:29:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44220 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14561280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:29:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 9.8431003930000003, 2024-09-17T07:29:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048309849000000002, 2024-09-17T07:29:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:31.582 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44220 2024-09-17T07:29:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:29:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:29:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:31.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646869+0000", 2024-09-17T07:29:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003506267 2024-09-17T07:29:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646874+0000", 2024-09-17T07:29:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7500000000000003e-06 2024-09-17T07:29:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:31.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646887+0000", 2024-09-17T07:29:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3115999999999999e-05 2024-09-17T07:29:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646894+0000", 2024-09-17T07:29:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5289999999999997e-06 2024-09-17T07:29:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691547+0000", 2024-09-17T07:29:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044652743000000002 2024-09-17T07:29:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691622+0000", 2024-09-17T07:29:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5037999999999995e-05 2024-09-17T07:29:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691673+0000", 2024-09-17T07:29:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-17T07:29:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:31.586 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:31.586 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:31.588 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:31.588 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:31.587+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:31.689 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:31.688+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:31.689 INFO:teuthology.orchestra.run.smithi045.stdout:2603 write 0x91650 thru 0x9d6df (0xc090 bytes) 2024-09-17T07:29:31.698 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:31.698 INFO:teuthology.orchestra.run.smithi045.stdout:2605 write 0xc018098 thru 0xc018b04 (0xa6d bytes) 2024-09-17T07:29:31.701 INFO:teuthology.orchestra.run.smithi045.stdout:2606 write 0x4be36ae thru 0x4bf2df3 (0xf746 bytes) 2024-09-17T07:29:31.713 INFO:teuthology.orchestra.run.smithi045.stdout:2607 trunc from 0xc022733 to 0x3e57b60 2024-09-17T07:29:31.737 INFO:teuthology.orchestra.run.smithi045.stdout:2608 punch from 0x298ca4d to 0x29939ce, (0x6f81 bytes) 2024-09-17T07:29:31.737 INFO:teuthology.orchestra.run.smithi045.stdout:2609 read 0x117ba9f thru 0x11823ff (0x6961 bytes) 2024-09-17T07:29:31.738 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:31.739 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:31.740 INFO:teuthology.orchestra.run.smithi045.stdout:2610 trunc from 0x3e57b60 to 0x173c5b2 2024-09-17T07:29:31.748 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:31.753 INFO:teuthology.orchestra.run.smithi045.stdout:2611 write 0x2db57d thru 0x2e5d31 (0xa7b5 bytes) 2024-09-17T07:29:31.755 INFO:teuthology.orchestra.run.smithi045.stdout:2612 read 0x431e4 thru 0x472fd (0x411a bytes) 2024-09-17T07:29:31.755 INFO:teuthology.orchestra.run.smithi045.stdout:2613 write 0x669b7 thru 0x6aa73 (0x40bd bytes) 2024-09-17T07:29:31.756 INFO:teuthology.orchestra.run.smithi045.stdout:2614 read 0x573fca thru 0x579d5b (0x5d92 bytes) 2024-09-17T07:29:31.773 INFO:teuthology.orchestra.run.smithi045.stdout:2615 write 0xb3e4d5f thru 0xb3f1805 (0xcaa7 bytes) 2024-09-17T07:29:31.776 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:31.786 INFO:teuthology.orchestra.run.smithi045.stdout:2619 trunc from 0xb3f1806 to 0x1a43757 2024-09-17T07:29:31.789 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:31.789+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:31.804 INFO:teuthology.orchestra.run.smithi045.stdout:2620 punch from 0x138c438 to 0x13904ed, (0x40b5 bytes) 2024-09-17T07:29:31.804 INFO:teuthology.orchestra.run.smithi045.stdout:2622 read 0x13593df thru 0x136392d (0xa54f bytes) 2024-09-17T07:29:31.823 INFO:teuthology.orchestra.run.smithi045.stdout:2623 write 0xc90f899 thru 0xc91c5a6 (0xcd0e bytes) 2024-09-17T07:29:31.826 INFO:teuthology.orchestra.run.smithi045.stdout:2624 write 0xa1af656 thru 0xa1b3013 (0x39be bytes) 2024-09-17T07:29:31.828 INFO:teuthology.orchestra.run.smithi045.stdout:2625 trunc from 0xc91c5a7 to 0xca73aaf 2024-09-17T07:29:31.833 INFO:teuthology.orchestra.run.smithi045.stdout:2627 punch from 0x87d5b6c to 0x87dd455, (0x78e9 bytes) 2024-09-17T07:29:31.833 INFO:teuthology.orchestra.run.smithi045.stdout:2628 write 0x50b27a4 thru 0x50bfecd (0xd72a bytes) 2024-09-17T07:29:31.835 INFO:teuthology.orchestra.run.smithi045.stdout:2629 writesame 0x377bddc thru 0x3782bb9 data_size 0x52(0x6dde bytes) 2024-09-17T07:29:31.837 INFO:teuthology.orchestra.run.smithi045.stdout:2630 writesame 0xabf684a thru 0xac04af1 data_size 0x4(0xe2a8 bytes) 2024-09-17T07:29:31.846 INFO:teuthology.orchestra.run.smithi045.stdout:2632 read 0xbc33a6e thru 0xbc40556 (0xcae9 bytes) 2024-09-17T07:29:31.846 INFO:teuthology.orchestra.run.smithi045.stdout:2635 write 0xc2a4101 thru 0xc2a53c7 (0x12c7 bytes) 2024-09-17T07:29:31.849 INFO:teuthology.orchestra.run.smithi045.stdout:2637 punch from 0xc94d423 to 0xc9564b3, (0x9090 bytes) 2024-09-17T07:29:31.849 INFO:teuthology.orchestra.run.smithi045.stdout:2639 punch from 0x3951715 to 0x3961204, (0xfaef bytes) 2024-09-17T07:29:31.849 INFO:teuthology.orchestra.run.smithi045.stdout:2640 write 0x630dc60 thru 0x6316113 (0x84b4 bytes) 2024-09-17T07:29:31.852 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:31.852 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:31.852 INFO:teuthology.orchestra.run.smithi045.stdout:2645 writesame 0x49c73bb thru 0x49cfcfd data_size 0x35(0x8943 bytes) 2024-09-17T07:29:31.855 INFO:teuthology.orchestra.run.smithi045.stdout:2647 punch from 0x5cd29e1 to 0x5cd8a5b, (0x607a bytes) 2024-09-17T07:29:31.855 INFO:teuthology.orchestra.run.smithi045.stdout:2648 punch from 0x30086eb to 0x3011296, (0x8bab bytes) 2024-09-17T07:29:31.855 INFO:teuthology.orchestra.run.smithi045.stdout:2649 punch from 0x8369ed4 to 0x8375539, (0xb665 bytes) 2024-09-17T07:29:31.855 INFO:teuthology.orchestra.run.smithi045.stdout:2650 read 0xabeba86 thru 0xabf2ba5 (0x7120 bytes) 2024-09-17T07:29:31.855 INFO:teuthology.orchestra.run.smithi045.stdout:2651 punch from 0x25dc66a to 0x25e834e, (0xbce4 bytes) 2024-09-17T07:29:31.855 INFO:teuthology.orchestra.run.smithi045.stdout:2652 write 0x2293e7a thru 0x22a3bbf (0xfd46 bytes) 2024-09-17T07:29:31.857 INFO:teuthology.orchestra.run.smithi045.stdout:2654 writesame 0x78ff36f thru 0x7903762 data_size 0x2(0x43f4 bytes) 2024-09-17T07:29:31.862 INFO:teuthology.orchestra.run.smithi045.stdout:2655 read 0x4d6a0cb thru 0x4d792e2 (0xf218 bytes) 2024-09-17T07:29:31.862 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:31.873 INFO:teuthology.orchestra.run.smithi045.stdout:2657 trunc from 0xca73aaf to 0x2c062cc 2024-09-17T07:29:31.885 INFO:teuthology.orchestra.run.smithi045.stdout:2659 punch from 0xe175eb to 0xe21aa3, (0xa4b8 bytes) 2024-09-17T07:29:31.886 INFO:teuthology.orchestra.run.smithi045.stdout:2660 read 0x3f1cf0 thru 0x3f6ae2 (0x4df3 bytes) 2024-09-17T07:29:31.886 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:31.890 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:31.890+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:31.896 INFO:teuthology.orchestra.run.smithi045.stdout:2665 trunc from 0x2c062cc to 0xbfc1f48 2024-09-17T07:29:31.897 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:31.897 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:31.897 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:31.897 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:31.897 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:31.902 INFO:teuthology.orchestra.run.smithi045.stdout:2666 write 0x96b4a6e thru 0x96c37eb (0xed7e bytes) 2024-09-17T07:29:31.905 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:31.906 INFO:teuthology.orchestra.run.smithi045.stdout:2668 writesame 0x1e9abdb thru 0x1eaab7c data_size 0x27(0xffa2 bytes) 2024-09-17T07:29:31.907 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:31.908 INFO:teuthology.orchestra.run.smithi045.stdout:2669 read 0x85b0c85 thru 0x85b3442 (0x27be bytes) 2024-09-17T07:29:31.908 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:31.908 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:31.908 INFO:teuthology.orchestra.run.smithi045.stdout:2674 writesame 0xaebbbc6 thru 0xaec3d51 data_size 0x2(0x818c bytes) 2024-09-17T07:29:31.921 INFO:teuthology.orchestra.run.smithi045.stdout:2675 writesame 0xc8cc8f0 thru 0xc8d14a6 data_size 0x47(0x4bb7 bytes) 2024-09-17T07:29:31.928 INFO:teuthology.orchestra.run.smithi045.stdout:2676 write 0xe5705f0 thru 0xe57b735 (0xb146 bytes) 2024-09-17T07:29:31.931 INFO:teuthology.orchestra.run.smithi045.stdout:2677 punch from 0xca7bb3d to 0xca87362, (0xb825 bytes) 2024-09-17T07:29:31.931 INFO:teuthology.orchestra.run.smithi045.stdout:2678 write 0xd159699 thru 0xd15ec3e (0x55a6 bytes) 2024-09-17T07:29:31.933 INFO:teuthology.orchestra.run.smithi045.stdout:2679 punch from 0xda2b468 to 0xda3065a, (0x51f2 bytes) 2024-09-17T07:29:31.933 INFO:teuthology.orchestra.run.smithi045.stdout:2680 write 0x375b07b thru 0x375b86a (0x7f0 bytes) 2024-09-17T07:29:31.936 INFO:teuthology.orchestra.run.smithi045.stdout:2682 punch from 0x6e3fed8 to 0x6e42d11, (0x2e39 bytes) 2024-09-17T07:29:31.936 INFO:teuthology.orchestra.run.smithi045.stdout:2683 read 0xa847f71 thru 0xa84e3e3 (0x6473 bytes) 2024-09-17T07:29:31.936 INFO:teuthology.orchestra.run.smithi045.stdout:2684 write 0xdebfe54 thru 0xdec5707 (0x58b4 bytes) 2024-09-17T07:29:31.939 INFO:teuthology.orchestra.run.smithi045.stdout:2685 read 0x4cce458 thru 0x4cd2faa (0x4b53 bytes) 2024-09-17T07:29:31.939 INFO:teuthology.orchestra.run.smithi045.stdout:2686 writesame 0xa935663 thru 0xa9435c5 data_size 0x35(0xdf63 bytes) 2024-09-17T07:29:31.942 INFO:teuthology.orchestra.run.smithi045.stdout:2687 read 0xb3a78c0 thru 0xb3b36cb (0xbe0c bytes) 2024-09-17T07:29:31.942 INFO:teuthology.orchestra.run.smithi045.stdout:2688 writesame 0x54cdc0c thru 0x54d0a16 data_size 0x1(0x2e0b bytes) 2024-09-17T07:29:31.949 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:31.949 INFO:teuthology.orchestra.run.smithi045.stdout:2690 writesame 0xb5459c0 thru 0xb553bfa data_size 0x75(0xe23b bytes) 2024-09-17T07:29:31.951 INFO:teuthology.orchestra.run.smithi045.stdout:2691 writesame 0x9119608 thru 0x911adca data_size 0xb(0x17c3 bytes) 2024-09-17T07:29:31.954 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:31.954 INFO:teuthology.orchestra.run.smithi045.stdout:2694 read 0x783be7a thru 0x7841791 (0x5918 bytes) 2024-09-17T07:29:31.954 INFO:teuthology.orchestra.run.smithi045.stdout:2695 write 0x757b357 thru 0x7587065 (0xbd0f bytes) 2024-09-17T07:29:31.957 INFO:teuthology.orchestra.run.smithi045.stdout:2698 writesame 0x27c1ec thru 0x27e504 data_size 0xf(0x2319 bytes) 2024-09-17T07:29:31.959 INFO:teuthology.orchestra.run.smithi045.stdout:2700 write 0x53d092f thru 0x53d9081 (0x8753 bytes) 2024-09-17T07:29:31.961 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:31.962 INFO:teuthology.orchestra.run.smithi045.stdout:2704 writesame 0x24ee2e1 thru 0x24f5353 data_size 0x1(0x7073 bytes) 2024-09-17T07:29:31.976 INFO:teuthology.orchestra.run.smithi045.stdout:2705 trunc from 0xe57b736 to 0xa3e420b 2024-09-17T07:29:31.987 INFO:teuthology.orchestra.run.smithi045.stdout:2706 punch from 0x4fbb720 to 0x4fca680, (0xef60 bytes) 2024-09-17T07:29:31.987 INFO:teuthology.orchestra.run.smithi045.stdout:2707 writesame 0x22be2b8 thru 0x22c3bc7 data_size 0x3c(0x5910 bytes) 2024-09-17T07:29:31.988 INFO:teuthology.orchestra.run.smithi045.stdout:2710 read 0x6713618 thru 0x671fd3c (0xc725 bytes) 2024-09-17T07:29:31.988 INFO:teuthology.orchestra.run.smithi045.stdout:2711 writesame 0x29def9c thru 0x29ec34d data_size 0x31(0xd3b2 bytes) 2024-09-17T07:29:31.990 INFO:teuthology.orchestra.run.smithi045.stdout:2713 write 0x574225c thru 0x57469e8 (0x478d bytes) 2024-09-17T07:29:31.991 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:31.991+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:31.992 INFO:teuthology.orchestra.run.smithi045.stdout:2714 read 0x218ebb1 thru 0x2197825 (0x8c75 bytes) 2024-09-17T07:29:31.995 INFO:teuthology.orchestra.run.smithi045.stdout:2715 trunc from 0xa3e420b to 0x8779670 2024-09-17T07:29:32.006 INFO:teuthology.orchestra.run.smithi045.stdout:2716 write 0x295342 thru 0x2972e1 (0x1fa0 bytes) 2024-09-17T07:29:32.008 INFO:teuthology.orchestra.run.smithi045.stdout:2717 punch from 0x19c3a84 to 0x19c6226, (0x27a2 bytes) 2024-09-17T07:29:32.008 INFO:teuthology.orchestra.run.smithi045.stdout:2719 write 0x679b280 thru 0x679ccf1 (0x1a72 bytes) 2024-09-17T07:29:32.010 INFO:teuthology.orchestra.run.smithi045.stdout:2722 write 0x5b1bf52 thru 0x5b27e9c (0xbf4b bytes) 2024-09-17T07:29:32.017 INFO:teuthology.orchestra.run.smithi045.stdout:2723 trunc from 0x8779670 to 0x45b9236 2024-09-17T07:29:32.043 INFO:teuthology.orchestra.run.smithi045.stdout:2724 write 0xc86c479 thru 0xc86f744 (0x32cc bytes) 2024-09-17T07:29:32.046 INFO:teuthology.orchestra.run.smithi045.stdout:2725 write 0xc80e29d thru 0xc81ce05 (0xeb69 bytes) 2024-09-17T07:29:32.048 INFO:teuthology.orchestra.run.smithi045.stdout:2726 punch from 0x4ecd30b to 0x4edcb2d, (0xf822 bytes) 2024-09-17T07:29:32.048 INFO:teuthology.orchestra.run.smithi045.stdout:2727 clone 28 order 25 su 8388608 sc 10 2024-09-17T07:29:32.060 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:32.060 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19985 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5275648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.806819814000001, 2024-09-17T07:29:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071819998999999995, 2024-09-17T07:29:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19985 2024-09-17T07:29:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252493+0000", 2024-09-17T07:29:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6300000000000004e-07 2024-09-17T07:29:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252496+0000", 2024-09-17T07:29:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-17T07:29:32.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252499+0000", 2024-09-17T07:29:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:29:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324036+0000", 2024-09-17T07:29:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071536587999999998 2024-09-17T07:29:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324118+0000", 2024-09-17T07:29:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2336999999999995e-05 2024-09-17T07:29:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324312+0000", 2024-09-17T07:29:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000193709 2024-09-17T07:29:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.065 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19986 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5505024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.806808662000002, 2024-09-17T07:29:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072083761999999996, 2024-09-17T07:29:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19986 2024-09-17T07:29:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:29:32.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252508+0000", 2024-09-17T07:29:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:29:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252511+0000", 2024-09-17T07:29:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.145e-06 2024-09-17T07:29:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252514+0000", 2024-09-17T07:29:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5569999999999998e-06 2024-09-17T07:29:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324326+0000", 2024-09-17T07:29:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071812681000000003 2024-09-17T07:29:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324372+0000", 2024-09-17T07:29:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5893000000000003e-05 2024-09-17T07:29:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324587+0000", 2024-09-17T07:29:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214608 2024-09-17T07:29:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19987 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5734400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.806795044000001, 2024-09-17T07:29:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072276676999999998, 2024-09-17T07:29:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19987 2024-09-17T07:29:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252518+0000", 2024-09-17T07:29:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-17T07:29:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252522+0000", 2024-09-17T07:29:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2440000000000004e-06 2024-09-17T07:29:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252525+0000", 2024-09-17T07:29:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5919999999999999e-06 2024-09-17T07:29:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324621+0000", 2024-09-17T07:29:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072096442999999996 2024-09-17T07:29:32.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324699+0000", 2024-09-17T07:29:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8061999999999999e-05 2024-09-17T07:29:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324793+0000", 2024-09-17T07:29:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4207000000000001e-05 2024-09-17T07:29:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19988 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.806784072999999, 2024-09-17T07:29:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072585798000000007, 2024-09-17T07:29:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19988 2024-09-17T07:29:32.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252529+0000", 2024-09-17T07:29:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:29:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252532+0000", 2024-09-17T07:29:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9579999999999999e-06 2024-09-17T07:29:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252534+0000", 2024-09-17T07:29:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5730000000000002e-06 2024-09-17T07:29:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324825+0000", 2024-09-17T07:29:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072290757999999997 2024-09-17T07:29:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324874+0000", 2024-09-17T07:29:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9360000000000002e-05 2024-09-17T07:29:32.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325113+0000", 2024-09-17T07:29:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000239025 2024-09-17T07:29:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19989 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:32.079 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.806774451999999, 2024-09-17T07:29:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072963494000000004, 2024-09-17T07:29:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19989 2024-09-17T07:29:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:29:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:29:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:29:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2900000000000002e-07 2024-09-17T07:29:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252542+0000", 2024-09-17T07:29:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8600000000000003e-06 2024-09-17T07:29:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252545+0000", 2024-09-17T07:29:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.605e-06 2024-09-17T07:29:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325144+0000", 2024-09-17T07:29:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072599460000000005 2024-09-17T07:29:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325223+0000", 2024-09-17T07:29:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8473999999999994e-05 2024-09-17T07:29:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325501+0000", 2024-09-17T07:29:32.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000278063 2024-09-17T07:29:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19990 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6422528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.806763981, 2024-09-17T07:29:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073241127000000003, 2024-09-17T07:29:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.084 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19990 2024-09-17T07:29:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:32.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252552+0000", 2024-09-17T07:29:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4719999999999997e-06 2024-09-17T07:29:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252557+0000", 2024-09-17T07:29:32.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3039999999999998e-06 2024-09-17T07:29:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252560+0000", 2024-09-17T07:29:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8830000000000002e-06 2024-09-17T07:29:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325529+0000", 2024-09-17T07:29:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072968880999999999 2024-09-17T07:29:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325574+0000", 2024-09-17T07:29:32.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466000000000001e-05 2024-09-17T07:29:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325789+0000", 2024-09-17T07:29:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021490999999999999 2024-09-17T07:29:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19991 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6651904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.806744679000001, 2024-09-17T07:29:32.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073720552999999994, 2024-09-17T07:29:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19991 2024-09-17T07:29:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4699999999999998e-07 2024-09-17T07:29:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252568+0000", 2024-09-17T07:29:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5300000000000002e-07 2024-09-17T07:29:32.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252572+0000", 2024-09-17T07:29:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9160000000000003e-06 2024-09-17T07:29:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252575+0000", 2024-09-17T07:29:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:29:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325820+0000", 2024-09-17T07:29:32.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073245597999999995 2024-09-17T07:29:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325901+0000", 2024-09-17T07:29:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0913000000000003e-05 2024-09-17T07:29:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326288+0000", 2024-09-17T07:29:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038646900000000002 2024-09-17T07:29:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.092 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19992 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6881280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.806734326000001, 2024-09-17T07:29:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074054683999999996, 2024-09-17T07:29:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19992 2024-09-17T07:29:32.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252578+0000", 2024-09-17T07:29:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:29:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252579+0000", 2024-09-17T07:29:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2500000000000004e-07 2024-09-17T07:29:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252582+0000", 2024-09-17T07:29:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-17T07:29:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252585+0000", 2024-09-17T07:29:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.542e-06 2024-09-17T07:29:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326315+0000", 2024-09-17T07:29:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073729921000000004 2024-09-17T07:29:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326359+0000", 2024-09-17T07:29:32.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4095999999999999e-05 2024-09-17T07:29:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326632+0000", 2024-09-17T07:29:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027314499999999997 2024-09-17T07:29:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19993 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7110656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.806724022000001, 2024-09-17T07:29:32.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074425142999999999, 2024-09-17T07:29:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19993 2024-09-17T07:29:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:32.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:29:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252589+0000", 2024-09-17T07:29:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.33e-07 2024-09-17T07:29:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252593+0000", 2024-09-17T07:29:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:29:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252595+0000", 2024-09-17T07:29:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.621e-06 2024-09-17T07:29:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326666+0000", 2024-09-17T07:29:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074070787999999999 2024-09-17T07:29:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326752+0000", 2024-09-17T07:29:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5748000000000006e-05 2024-09-17T07:29:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327013+0000", 2024-09-17T07:29:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026098400000000002 2024-09-17T07:29:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19994 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7340032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.806713562999999, 2024-09-17T07:29:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074765920999999999, 2024-09-17T07:29:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19994 2024-09-17T07:29:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:29:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252599+0000", 2024-09-17T07:29:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2099999999999995e-07 2024-09-17T07:29:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252603+0000", 2024-09-17T07:29:32.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:29:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252606+0000", 2024-09-17T07:29:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-09-17T07:29:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327047+0000", 2024-09-17T07:29:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074441050999999994 2024-09-17T07:29:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327121+0000", 2024-09-17T07:29:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4283999999999996e-05 2024-09-17T07:29:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327364+0000", 2024-09-17T07:29:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000243155 2024-09-17T07:29:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19995 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7569408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.806702865999998, 2024-09-17T07:29:32.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075172694999999998, 2024-09-17T07:29:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19995 2024-09-17T07:29:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252610+0000", 2024-09-17T07:29:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6799999999999999e-07 2024-09-17T07:29:32.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252614+0000", 2024-09-17T07:29:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000002e-06 2024-09-17T07:29:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252616+0000", 2024-09-17T07:29:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.419e-06 2024-09-17T07:29:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327407+0000", 2024-09-17T07:29:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074790617000000004 2024-09-17T07:29:32.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327498+0000", 2024-09-17T07:29:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0758000000000001e-05 2024-09-17T07:29:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327782+0000", 2024-09-17T07:29:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028395599999999998 2024-09-17T07:29:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.110 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7798784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.806692550000001, 2024-09-17T07:29:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075605832999999997, 2024-09-17T07:29:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:29:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:29:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:29:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:29:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:29:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:29:32.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:29:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327844+0000", 2024-09-17T07:29:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075217709999999993 2024-09-17T07:29:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327927+0000", 2024-09-17T07:29:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2628999999999998e-05 2024-09-17T07:29:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328225+0000", 2024-09-17T07:29:32.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029800600000000002 2024-09-17T07:29:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8028160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.806682197000001, 2024-09-17T07:29:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076061237000000004, 2024-09-17T07:29:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:29:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:29:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:29:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:29:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:29:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:29:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:29:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328268+0000", 2024-09-17T07:29:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075626323999999995 2024-09-17T07:29:32.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328352+0000", 2024-09-17T07:29:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3578999999999994e-05 2024-09-17T07:29:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328691+0000", 2024-09-17T07:29:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033915099999999999 2024-09-17T07:29:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8257536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.806666746000001, 2024-09-17T07:29:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076441903000000005, 2024-09-17T07:29:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:29:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.120 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:29:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:29:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:29:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:29:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:29:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:29:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:29:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328747+0000", 2024-09-17T07:29:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076094192000000005 2024-09-17T07:29:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328824+0000", 2024-09-17T07:29:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7096999999999997e-05 2024-09-17T07:29:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.329087+0000", 2024-09-17T07:29:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026249999999999998 2024-09-17T07:29:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20029 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.796581757999999, 2024-09-17T07:29:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071783696999999994, 2024-09-17T07:29:32.124 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20029 2024-09-17T07:29:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262735+0000", 2024-09-17T07:29:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.955e-06 2024-09-17T07:29:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262737+0000", 2024-09-17T07:29:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0090000000000002e-06 2024-09-17T07:29:32.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262743+0000", 2024-09-17T07:29:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2890000000000003e-06 2024-09-17T07:29:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262748+0000", 2024-09-17T07:29:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334163+0000", 2024-09-17T07:29:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071415336999999995 2024-09-17T07:29:32.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334243+0000", 2024-09-17T07:29:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9548000000000005e-05 2024-09-17T07:29:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334514+0000", 2024-09-17T07:29:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027086800000000001 2024-09-17T07:29:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7356416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.796550586999999, 2024-09-17T07:29:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072172616999999994, 2024-09-17T07:29:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:29:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:29:32.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:29:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:29:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:29:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:29:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:29:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:29:32.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:29:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334558+0000", 2024-09-17T07:29:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071780790999999997 2024-09-17T07:29:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334647+0000", 2024-09-17T07:29:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9448000000000001e-05 2024-09-17T07:29:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334934+0000", 2024-09-17T07:29:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028654599999999999 2024-09-17T07:29:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7585792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.796528875, 2024-09-17T07:29:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072606135000000002, 2024-09-17T07:29:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:29:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:29:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:29:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:29:32.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:29:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:29:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:29:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:29:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335116+0000", 2024-09-17T07:29:32.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072323897999999998 2024-09-17T07:29:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335166+0000", 2024-09-17T07:29:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9882000000000001e-05 2024-09-17T07:29:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335389+0000", 2024-09-17T07:29:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022334300000000001 2024-09-17T07:29:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.137 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7815168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.796516676, 2024-09-17T07:29:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073051230999999994, 2024-09-17T07:29:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:29:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.138 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:32.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:29:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:29:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:29:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:29:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:29:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:29:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335431+0000", 2024-09-17T07:29:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072627852000000007 2024-09-17T07:29:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335525+0000", 2024-09-17T07:29:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3708000000000005e-05 2024-09-17T07:29:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335846+0000", 2024-09-17T07:29:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032182000000000001 2024-09-17T07:29:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.796505818, 2024-09-17T07:29:32.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073284199999999994, 2024-09-17T07:29:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:29:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:29:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:29:32.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:29:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:29:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:29:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:29:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:29:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335863+0000", 2024-09-17T07:29:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073048921000000003 2024-09-17T07:29:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335904+0000", 2024-09-17T07:29:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1545000000000003e-05 2024-09-17T07:29:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336090+0000", 2024-09-17T07:29:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185787 2024-09-17T07:29:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8273920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.796491558, 2024-09-17T07:29:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073670478999999997, 2024-09-17T07:29:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:29:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:29:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:29:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:29:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:29:32.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:29:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:29:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:29:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336126+0000", 2024-09-17T07:29:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073297254000000006 2024-09-17T07:29:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336211+0000", 2024-09-17T07:29:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4728000000000006e-05 2024-09-17T07:29:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336491+0000", 2024-09-17T07:29:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027979399999999998 2024-09-17T07:29:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.151 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.151 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:32.153 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:32.153 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:32.152+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:32.254 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:32.254+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:32.300 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:32.300 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:32.300 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:32.300 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:32.310 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:32.353 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone27 from 0xc86f745 (overlap 0x173c5b2) to 0x0 2024-09-17T07:29:32.354 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:32.354+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:32.409 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-17T07:29:32.422 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:32.422 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:32.422 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:32.422 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:32.422 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:32.431 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:32.455 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:32.455+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:32.556 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:32.556+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:32.576 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:32.576 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:32.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:32.577 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:32.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.577 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:32.577 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:32.577 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.669839966000001, 2024-09-17T07:29:32.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:29:32.577 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.577 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.577 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.577 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:29:32.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.578 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:32.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:32.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:32.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:29:32.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:29:32.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:29:32.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:29:32.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:29:32.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:29:32.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:29:32.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:29:32.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:29:32.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:29:32.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:29:32.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:29:32.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:29:32.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.581 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.581 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:32.581 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:32.581 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.669821289000001, 2024-09-17T07:29:32.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:29:32.582 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.582 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.582 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.582 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:29:32.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.582 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:32.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:32.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:29:32.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:32.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:29:32.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:29:32.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:29:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:29:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:29:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:29:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:29:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:29:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:29:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:29:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:29:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:29:32.585 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.669802936, 2024-09-17T07:29:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:29:32.586 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:29:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:32.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:29:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:29:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:29:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:29:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:29:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:29:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:29:32.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:29:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:29:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:29:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:29:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:29:32.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:29:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.669783337999998, 2024-09-17T07:29:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:29:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:29:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:29:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:29:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:29:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:29:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:29:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:29:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:29:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:29:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:29:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:29:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:29:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:29:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:29:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:29:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.595 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.669763382999999, 2024-09-17T07:29:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:29:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.596 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:29:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:29:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:29:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:29:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:29:32.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:29:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:29:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:29:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:29:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:29:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:29:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:29:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:29:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:29:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:32.600 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.669744728000001, 2024-09-17T07:29:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:29:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:29:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:29:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:29:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:29:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:29:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:29:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:29:32.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:29:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:29:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:29:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:29:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:29:32.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:29:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:29:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:32.605 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.669726252, 2024-09-17T07:29:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:29:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:29:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:29:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:29:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:29:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:29:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:29:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:29:32.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:29:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:29:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:29:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:29:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:29:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:29:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:29:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:29:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:32.610 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.669706652999999, 2024-09-17T07:29:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:29:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:29:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:32.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:29:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:29:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:29:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:29:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:29:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:29:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:29:32.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:29:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:29:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:29:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:29:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:29:32.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:29:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.669688078, 2024-09-17T07:29:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:29:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.615 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:29:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:29:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:29:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:29:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:29:32.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:29:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:29:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:29:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:29:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:29:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:29:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:29:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:29:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:29:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.619 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.669665650999999, 2024-09-17T07:29:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:29:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:29:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:32.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:29:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:29:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:29:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:29:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:29:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:29:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:29:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:29:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:29:32.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:29:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:29:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:29:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.669637834, 2024-09-17T07:29:32.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:29:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:29:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:32.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:29:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:29:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:29:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:29:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:29:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:29:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:29:32.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:29:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:29:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:29:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:29:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:29:32.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:29:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.669615889999999, 2024-09-17T07:29:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:29:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:29:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:29:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:29:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:29:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:29:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:29:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:29:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:29:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:29:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:29:32.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:29:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:29:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:29:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:29:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.669594786000001, 2024-09-17T07:29:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:29:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:29:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.634 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:29:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:29:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:29:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:29:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:29:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:29:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:29:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:29:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:29:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:29:32.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:29:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:29:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.669575182999999, 2024-09-17T07:29:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:29:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.639 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:29:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:29:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:29:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:29:32.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:29:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:29:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:29:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:29:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:29:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:29:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:29:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:29:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:29:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:29:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:29:32.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.71663407, 2024-09-17T07:29:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:29:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:29:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:32.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:29:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:29:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:29:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:29:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:29:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:29:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:29:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:29:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:29:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:29:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:29:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:29:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:29:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:29:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.716620410000001, 2024-09-17T07:29:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:29:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.649 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:29:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:29:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:29:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:29:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:29:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:29:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:29:32.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:29:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:29:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:29:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:29:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:29:32.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:29:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:29:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.716605628, 2024-09-17T07:29:32.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:29:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:29:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:32.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:29:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:29:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:29:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:29:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:29:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:29:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:29:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:29:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:29:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:29:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:29:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:29:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:29:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.716594736999999, 2024-09-17T07:29:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:29:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.659 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:29:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:29:32.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:29:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:29:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:29:32.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:29:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:29:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:29:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:29:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:29:32.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:29:32.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:29:32.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:29:32.668 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.668 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.668 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.668 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:32.669 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:32.669 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.716585177000001, 2024-09-17T07:29:32.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:29:32.669 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.669 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.669 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.670 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:29:32.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.670 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:32.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:32.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:29:32.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:32.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:29:32.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:29:32.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:29:32.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:29:32.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:29:32.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:29:32.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:29:32.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:29:32.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:29:32.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:29:32.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:29:32.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:29:32.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.677 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.677 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.678 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:32.678 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:32.678 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.716574616999999, 2024-09-17T07:29:32.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:29:32.679 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:32.679 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:32.679 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:32.679 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:32.679 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:32.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:32.680 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:29:32.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.680 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:32.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:32.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:32.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:32.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:32.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:32.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:32.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:29:32.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:29:32.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:32.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:29:32.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:29:32.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:32.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:29:32.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:29:32.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:32.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:29:32.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:29:32.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:32.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:29:32.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:29:32.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:32.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:29:32.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:29:32.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:32.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:32.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:32.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:29:32.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:29:32.687 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.687 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.687 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.687 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:32.687 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:32.687 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:32.690 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:32.691 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:32.688+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:32.692 INFO:teuthology.orchestra.run.smithi045.stdout:2728 read 0x48e31d7 thru 0x48e7a66 (0x4890 bytes) 2024-09-17T07:29:32.692 INFO:teuthology.orchestra.run.smithi045.stdout:2731 writesame 0xeb71e83 thru 0xeb7e6da data_size 0xc(0xc858 bytes) 2024-09-17T07:29:32.692 INFO:teuthology.orchestra.run.smithi045.stdout:2733 write 0xbfde2af thru 0xbfeb0fd (0xce4f bytes) 2024-09-17T07:29:32.693 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:32.693 INFO:teuthology.orchestra.run.smithi045.stdout:2735 punch from 0x9d5681b to 0x9d61837, (0xb01c bytes) 2024-09-17T07:29:32.693 INFO:teuthology.orchestra.run.smithi045.stdout:2737 write 0x3b06d6b thru 0x3b1382c (0xcac2 bytes) 2024-09-17T07:29:32.693 INFO:teuthology.orchestra.run.smithi045.stdout:2739 punch from 0xb224054 to 0xb22f034, (0xafe0 bytes) 2024-09-17T07:29:32.693 INFO:teuthology.orchestra.run.smithi045.stdout:2740 read 0x5ae3345 thru 0x5ae9d8c (0x6a48 bytes) 2024-09-17T07:29:32.693 INFO:teuthology.orchestra.run.smithi045.stdout:2741 write 0xbc2b191 thru 0xbc3a66b (0xf4db bytes) 2024-09-17T07:29:32.693 INFO:teuthology.orchestra.run.smithi045.stdout:2742 read 0xbff32a6 thru 0xbffde1b (0xab76 bytes) 2024-09-17T07:29:32.693 INFO:teuthology.orchestra.run.smithi045.stdout:2744 write 0x23fe408 thru 0x240ac60 (0xc859 bytes) 2024-09-17T07:29:32.693 INFO:teuthology.orchestra.run.smithi045.stdout:2745 punch from 0xc968e6f to 0xc96a091, (0x1222 bytes) 2024-09-17T07:29:32.693 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:32.693 INFO:teuthology.orchestra.run.smithi045.stdout:2749 trunc from 0xeb7e6db to 0xdea9608 2024-09-17T07:29:32.693 INFO:teuthology.orchestra.run.smithi045.stdout:2750 writesame 0xb2d9de0 thru 0xb2e1688 data_size 0x17(0x78a9 bytes) 2024-09-17T07:29:32.693 INFO:teuthology.orchestra.run.smithi045.stdout:2752 writesame 0xb6b9e6f thru 0xb6c1738 data_size 0x1(0x78ca bytes) 2024-09-17T07:29:32.695 INFO:teuthology.orchestra.run.smithi045.stdout:2753 trunc from 0xdea9608 to 0x633ad4e 2024-09-17T07:29:32.713 INFO:teuthology.orchestra.run.smithi045.stdout:2754 read 0x3ec4282 thru 0x3ec6454 (0x21d3 bytes) 2024-09-17T07:29:32.714 INFO:teuthology.orchestra.run.smithi045.stdout:2755 read 0x3090510 thru 0x309a031 (0x9b22 bytes) 2024-09-17T07:29:32.714 INFO:teuthology.orchestra.run.smithi045.stdout:2756 read 0x5adedf7 thru 0x5ae2318 (0x3522 bytes) 2024-09-17T07:29:32.722 INFO:teuthology.orchestra.run.smithi045.stdout:2757 write 0x65bf975 thru 0x65c20a1 (0x272d bytes) 2024-09-17T07:29:32.724 INFO:teuthology.orchestra.run.smithi045.stdout:2758 write 0x13ec495 thru 0x13efa05 (0x3571 bytes) 2024-09-17T07:29:32.725 INFO:teuthology.orchestra.run.smithi045.stdout:2760 punch from 0x2c96775 to 0x2ca346f, (0xccfa bytes) 2024-09-17T07:29:32.725 INFO:teuthology.orchestra.run.smithi045.stdout:2761 punch from 0x34cc5d0 to 0x34da3e3, (0xde13 bytes) 2024-09-17T07:29:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:2763 punch from 0x1920732 to 0x192354a, (0x2e18 bytes) 2024-09-17T07:29:32.735 INFO:teuthology.orchestra.run.smithi045.stdout:2765 write 0x8c7edc6 thru 0x8c84333 (0x556e bytes) 2024-09-17T07:29:32.737 INFO:teuthology.orchestra.run.smithi045.stdout:2767 read 0x4b78cad thru 0x4b883a0 (0xf6f4 bytes) 2024-09-17T07:29:32.737 INFO:teuthology.orchestra.run.smithi045.stdout:2768 write 0x4b3a91e thru 0x4b3ff13 (0x55f6 bytes) 2024-09-17T07:29:32.739 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:29:32.739 INFO:teuthology.orchestra.run.smithi045.stdout:2769 punch from 0x81cbbcd to 0x81d65bc, (0xa9ef bytes) 2024-09-17T07:29:32.743 INFO:teuthology.orchestra.run.smithi045.stdout:2770 trunc from 0x8c84334 to 0x5624a2d 2024-09-17T07:29:32.762 INFO:teuthology.orchestra.run.smithi045.stdout:2771 writesame 0xb14338c thru 0xb14cec2 data_size 0xf(0x9b37 bytes) 2024-09-17T07:29:32.764 INFO:teuthology.orchestra.run.smithi045.stdout:2772 punch from 0x64f83cb to 0x64fac35, (0x286a bytes) 2024-09-17T07:29:32.773 INFO:teuthology.orchestra.run.smithi045.stdout:2773 write 0xdb8c05a thru 0xdb8daf8 (0x1a9f bytes) 2024-09-17T07:29:32.775 INFO:teuthology.orchestra.run.smithi045.stdout:2774 clone 29 order 20 su 262144 sc 5 2024-09-17T07:29:32.790 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:32.789+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:32.837 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:32.837 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:32.837 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:32.837 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:32.845 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:32.867 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:32.867 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:32.884 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:32.891 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:32.890+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:32.900 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:32.900 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:32.901 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:32.917 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:32.918 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:32.934 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:32.934 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:32.951 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:32.951 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:32.989 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:32.990 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:32.992 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:32.991+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:32.998 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:33.093 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:33.092+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:33.142 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:29:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 66.806654248000001, 2024-09-17T07:29:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:29:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.143 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:29:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:29:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:29:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:29:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:29:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:29:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:29:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:29:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:29:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:29:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:29:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:29:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:29:33.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:29:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:29:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:29:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:29:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:29:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:29:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:29:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:29:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:29:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:29:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:29:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:29:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:29:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:29:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:29:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:29:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:29:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:29:33.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7378 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7176192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.217612213999999, 2024-09-17T07:29:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073003980999999996, 2024-09-17T07:29:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7378 2024-09-17T07:29:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1799999999999999e-07 2024-09-17T07:29:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923713+0000", 2024-09-17T07:29:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.057e-06 2024-09-17T07:29:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923716+0000", 2024-09-17T07:29:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0599999999999999e-06 2024-09-17T07:29:33.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923719+0000", 2024-09-17T07:29:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8219999999999998e-06 2024-09-17T07:29:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996401+0000", 2024-09-17T07:29:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072681467999999999 2024-09-17T07:29:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996452+0000", 2024-09-17T07:29:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.134e-05 2024-09-17T07:29:33.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996716+0000", 2024-09-17T07:29:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000264016 2024-09-17T07:29:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.217602098, 2024-09-17T07:29:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:29:33.154 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:29:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:29:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:29:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:29:33.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:29:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:29:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:29:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:29:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:29:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:29:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:29:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:29:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:29:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:29:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:29:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.158 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7398 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6389760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.212333563999998, 2024-09-17T07:29:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072891580999999997, 2024-09-17T07:29:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7398 2024-09-17T07:29:33.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928995+0000", 2024-09-17T07:29:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0759999999999996e-06 2024-09-17T07:29:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928999+0000", 2024-09-17T07:29:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1609999999999996e-06 2024-09-17T07:29:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929002+0000", 2024-09-17T07:29:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8449999999999999e-06 2024-09-17T07:29:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001598+0000", 2024-09-17T07:29:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072596318000000007 2024-09-17T07:29:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001663+0000", 2024-09-17T07:29:33.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4995999999999999e-05 2024-09-17T07:29:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001882+0000", 2024-09-17T07:29:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021897299999999999 2024-09-17T07:29:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.212318981999999, 2024-09-17T07:29:33.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:29:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:29:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:33.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:29:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:29:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:29:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:29:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:29:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:29:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:29:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:29:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:29:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:29:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:29:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:29:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:29:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.167 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.212288002000001, 2024-09-17T07:29:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:29:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.168 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:29:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:29:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:29:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:29:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:29:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:29:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:29:33.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:29:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:29:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:29:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:29:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:29:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:29:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:29:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:29:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:33.172 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.212257952999998, 2024-09-17T07:29:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:29:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:29:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:29:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:29:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:29:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:29:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:29:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:29:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:29:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:29:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:29:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:29:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:29:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:29:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:29:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:29:33.176 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.212229546000003, 2024-09-17T07:29:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:29:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.177 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:29:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:29:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:29:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:29:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:29:33.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:29:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:29:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:29:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:29:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:29:33.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:29:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:29:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:29:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.212209725000001, 2024-09-17T07:29:33.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:29:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:29:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:29:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:29:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:29:33.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:29:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:29:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:29:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:29:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:29:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:29:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:29:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:29:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:29:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:29:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:29:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.185 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.212185472999998, 2024-09-17T07:29:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:29:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.186 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:29:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:29:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:29:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:29:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:29:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:29:33.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:29:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:29:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:29:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:29:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:29:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:29:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:29:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:29:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:29:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7418 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7700480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.190 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.206779449999999, 2024-09-17T07:29:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072752965000000003, 2024-09-17T07:29:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7418 2024-09-17T07:29:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3599999999999999e-07 2024-09-17T07:29:33.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934547+0000", 2024-09-17T07:29:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3350000000000001e-06 2024-09-17T07:29:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934554+0000", 2024-09-17T07:29:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9859999999999997e-06 2024-09-17T07:29:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934557+0000", 2024-09-17T07:29:33.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.818e-06 2024-09-17T07:29:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007104+0000", 2024-09-17T07:29:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072546779000000006 2024-09-17T07:29:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007147+0000", 2024-09-17T07:29:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2904000000000001e-05 2024-09-17T07:29:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007298+0000", 2024-09-17T07:29:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015080700000000001 2024-09-17T07:29:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.206762885000003, 2024-09-17T07:29:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:29:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.195 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:29:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:29:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:29:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:29:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:29:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:29:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:29:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:29:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:29:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:29:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:29:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:29:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:29:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:29:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:29:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.206716010000001, 2024-09-17T07:29:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:29:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.200 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:29:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:33.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:29:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:29:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:29:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:29:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:29:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:29:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:29:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:29:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:29:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:29:33.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:29:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:29:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.206701305999999, 2024-09-17T07:29:33.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:29:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:29:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:29:33.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:29:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:29:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:29:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:29:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:29:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:29:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:29:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:29:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:29:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:29:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:29:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.212 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.206678211000003, 2024-09-17T07:29:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:29:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:29:33.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:29:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:29:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:29:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:29:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:29:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:29:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:29:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:29:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:29:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:29:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:29:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:29:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:29:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.206646835999997, 2024-09-17T07:29:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:29:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.217 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:29:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:29:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:29:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:29:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:29:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:29:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:29:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:29:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:29:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:29:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:29:33.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:29:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:29:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:29:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:29:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.206631575999999, 2024-09-17T07:29:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:29:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:29:33.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:29:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:29:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:29:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:29:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:29:33.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:29:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:29:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:29:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:29:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:29:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:29:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:29:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:29:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:33.226 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.206611506000002, 2024-09-17T07:29:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:29:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:29:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:29:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:29:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:29:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:29:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:29:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:29:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:29:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:29:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:29:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:29:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:29:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:29:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:29:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.230 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.206599566000001, 2024-09-17T07:29:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:29:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.231 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:29:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:29:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:29:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:29:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:29:33.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:29:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:29:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:29:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:29:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:29:33.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:29:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:29:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:29:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.206576407999997, 2024-09-17T07:29:33.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:29:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:29:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:29:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:29:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:29:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:29:33.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:29:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:29:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:29:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:29:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:29:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:29:33.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:29:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:29:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:29:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:29:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:33.239 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:33.240 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:33.240 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:33.241 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:33.242 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:33.243 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:33.243 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:33.240+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:33.303 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T07:29:33.341 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-17T07:29:33.341 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:29:33.342 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-17T07:29:33.342 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T07:29:33.342 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T07:29:33.342 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-17T07:29:33.360 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone28 from 0xdb8daf9 (overlap 0x5624a2d) to 0x2b12516 2024-09-17T07:29:33.361 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:33.360+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:33.403 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:33.403 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:33.413 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:33.443 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-17T07:29:33.443 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:33.442+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:33.445 INFO:teuthology.orchestra.run.smithi045.stdout:2775 write 0x85d171b thru 0x85d3453 (0x1d39 bytes) 2024-09-17T07:29:33.451 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:33.452 INFO:teuthology.orchestra.run.smithi045.stdout:2777 read 0x1776b54 thru 0x177f56e (0x8a1b bytes) 2024-09-17T07:29:33.452 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:33.452 INFO:teuthology.orchestra.run.smithi045.stdout:2780 clone 30 order 25 su 2097152 sc 9 2024-09-17T07:29:33.477 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:33.477 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T07:29:33.477 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:33.543 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:33.543+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:33.552 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:33.552 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:33.561 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:33.645 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:33.644+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:33.708 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:33.708 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:33.718 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:33.746 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:33.745+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:33.825 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:33.825 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:33.833 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:33.846 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:33.846+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:33.941 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:33.941 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:33.947 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:33.947+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:33.949 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:34.049 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:34.048+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:34.097 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:34.097 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:34.105 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:34.150 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:34.149+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:34.212 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:34.212 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:34.221 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:34.264 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:34.264+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:34.297 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-17T07:29:34.333 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:34.333 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:34.342 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:34.352 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:34.351+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone29 from 0xdb8daf9 (overlap 0xdb8daf9) to 0x0 2024-09-17T07:29:34.453 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:34.452+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:34.512 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:34.512 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:34.524 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:34.554 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:34.553+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:34.642 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-17T07:29:34.644 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:34.645 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:34.654 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:34.655 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:34.655+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:34.693 INFO:teuthology.orchestra.run.smithi045.stdout:2781 read 0x2ab84bc thru 0x2abfb89 (0x76ce bytes) 2024-09-17T07:29:34.695 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:34.695 INFO:teuthology.orchestra.run.smithi045.stdout:2784 write 0xb309330 thru 0xb312278 (0x8f49 bytes) 2024-09-17T07:29:34.752 INFO:teuthology.orchestra.run.smithi045.stdout:2786 write 0x7990a12 thru 0x7998947 (0x7f36 bytes) 2024-09-17T07:29:34.756 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:34.755+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:34.762 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:34.762 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:34.771 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:34.832 INFO:teuthology.orchestra.run.smithi045.stdout:2787 writesame 0x829f1a thru 0x838d9d data_size 0x56(0xee84 bytes) 2024-09-17T07:29:34.841 INFO:teuthology.orchestra.run.smithi045.stdout:2788 write 0xd88524f thru 0xd88f047 (0x9df9 bytes) 2024-09-17T07:29:34.857 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:34.856+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:34.886 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:34.886 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:34.898 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:34.905 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:34.905 INFO:teuthology.orchestra.run.smithi045.stdout:2790 writesame 0xa04ffdf thru 0xa05b440 data_size 0xb(0xb462 bytes) 2024-09-17T07:29:34.908 INFO:teuthology.orchestra.run.smithi045.stdout:2791 writesame 0x3daf596 thru 0x3db59ff data_size 0x1(0x646a bytes) 2024-09-17T07:29:34.957 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:34.957+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:34.994 INFO:teuthology.orchestra.run.smithi045.stdout:2792 read 0x283f62d thru 0x284683a (0x720e bytes) 2024-09-17T07:29:34.995 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:34.995 INFO:teuthology.orchestra.run.smithi045.stdout:2795 read 0xb6ae05a thru 0xb6ae58b (0x532 bytes) 2024-09-17T07:29:34.995 INFO:teuthology.orchestra.run.smithi045.stdout:2797 trunc from 0xdb8daf9 to 0xd4d5b19 2024-09-17T07:29:35.048 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.049 INFO:teuthology.orchestra.run.smithi045.stdout:2799 read 0x8abbed thru 0x8b85a4 (0xc9b8 bytes) 2024-09-17T07:29:35.049 INFO:teuthology.orchestra.run.smithi045.stdout:2800 punch from 0xbf3ee8 to 0xbf5e51, (0x1f69 bytes) 2024-09-17T07:29:35.049 INFO:teuthology.orchestra.run.smithi045.stdout:2802 writesame 0x88e16d4 thru 0x88ea542 data_size 0x1(0x8e6f bytes) 2024-09-17T07:29:35.051 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:35.051 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:35.058 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:35.058+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:35.060 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:35.085 INFO:teuthology.orchestra.run.smithi045.stdout:2804 trunc from 0xd4d5b19 to 0x256d2e4 2024-09-17T07:29:35.111 INFO:teuthology.orchestra.run.smithi045.stdout:2805 writesame 0x5fe2768 thru 0x5fe9c5c data_size 0x1(0x74f5 bytes) 2024-09-17T07:29:35.117 INFO:teuthology.orchestra.run.smithi045.stdout:2806 writesame 0x6508eee thru 0x6512a3f data_size 0x9(0x9b52 bytes) 2024-09-17T07:29:35.118 INFO:teuthology.orchestra.run.smithi045.stdout:2807 punch from 0x7dac17 to 0x7e28eb, (0x7cd4 bytes) 2024-09-17T07:29:35.123 INFO:teuthology.orchestra.run.smithi045.stdout:2808 writesame 0x6ab1136 thru 0x6ac016b data_size 0x6f(0xf036 bytes) 2024-09-17T07:29:35.125 INFO:teuthology.orchestra.run.smithi045.stdout:2809 read 0x1ddaea1 thru 0x1dea139 (0xf299 bytes) 2024-09-17T07:29:35.125 INFO:teuthology.orchestra.run.smithi045.stdout:2810 punch from 0x288105b to 0x28818d2, (0x877 bytes) 2024-09-17T07:29:35.125 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.126 INFO:teuthology.orchestra.run.smithi045.stdout:2812 write 0x4db4b6d thru 0x4db6422 (0x18b6 bytes) 2024-09-17T07:29:35.127 INFO:teuthology.orchestra.run.smithi045.stdout:2813 punch from 0x284bf6c to 0x285433e, (0x83d2 bytes) 2024-09-17T07:29:35.127 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.138 INFO:teuthology.orchestra.run.smithi045.stdout:2816 write 0xba2d994 thru 0xba36773 (0x8de0 bytes) 2024-09-17T07:29:35.140 INFO:teuthology.orchestra.run.smithi045.stdout:2817 trunc from 0xba36774 to 0xc741154 2024-09-17T07:29:35.146 INFO:teuthology.orchestra.run.smithi045.stdout:2818 write 0x97e7b5c thru 0x97f0cf7 (0x919c bytes) 2024-09-17T07:29:35.147 INFO:teuthology.orchestra.run.smithi045.stdout:2819 punch from 0xc6c2331 to 0xc6c70ba, (0x4d89 bytes) 2024-09-17T07:29:35.148 INFO:teuthology.orchestra.run.smithi045.stdout:2822 write 0x3a499 thru 0x3c3e6 (0x1f4e bytes) 2024-09-17T07:29:35.149 INFO:teuthology.orchestra.run.smithi045.stdout:2823 write 0x451995d thru 0x45251f5 (0xb899 bytes) 2024-09-17T07:29:35.150 INFO:teuthology.orchestra.run.smithi045.stdout:2824 read 0xc3658a2 thru 0xc371d1f (0xc47e bytes) 2024-09-17T07:29:35.151 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.151 INFO:teuthology.orchestra.run.smithi045.stdout:2826 read 0xa536ea thru 0xa57e84 (0x479b bytes) 2024-09-17T07:29:35.158 INFO:teuthology.orchestra.run.smithi045.stdout:2828 write 0xd692c35 thru 0xd69c23a (0x9606 bytes) 2024-09-17T07:29:35.158 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:35.158+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:35.165 INFO:teuthology.orchestra.run.smithi045.stdout:2829 writesame 0xdb78821 thru 0xdb7f0c3 data_size 0x1(0x68a3 bytes) 2024-09-17T07:29:35.166 INFO:teuthology.orchestra.run.smithi045.stdout:2830 read 0x729a64f thru 0x729d489 (0x2e3b bytes) 2024-09-17T07:29:35.166 INFO:teuthology.orchestra.run.smithi045.stdout:2831 writesame 0x69d579b thru 0x69db1c2 data_size 0x1(0x5a28 bytes) 2024-09-17T07:29:35.174 INFO:teuthology.orchestra.run.smithi045.stdout:2832 write 0xbcd549a thru 0xbcda0cd (0x4c34 bytes) 2024-09-17T07:29:35.176 INFO:teuthology.orchestra.run.smithi045.stdout:2833 punch from 0x5e63296 to 0x5e6381c, (0x586 bytes) 2024-09-17T07:29:35.176 INFO:teuthology.orchestra.run.smithi045.stdout:2834 read 0x3218476 thru 0x3220919 (0x84a4 bytes) 2024-09-17T07:29:35.177 INFO:teuthology.orchestra.run.smithi045.stdout:2835 read 0x3fc5a82 thru 0x3fc5d7c (0x2fb bytes) 2024-09-17T07:29:35.177 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.177 INFO:teuthology.orchestra.run.smithi045.stdout:2837 writesame 0xca1ad4 thru 0xca29c6 data_size 0x1(0xef3 bytes) 2024-09-17T07:29:35.179 INFO:teuthology.orchestra.run.smithi045.stdout:2840 write 0x603bf51 thru 0x6043ca8 (0x7d58 bytes) 2024-09-17T07:29:35.183 INFO:teuthology.orchestra.run.smithi045.stdout:2841 trunc from 0xdb7f0c4 to 0xaef813c 2024-09-17T07:29:35.190 INFO:teuthology.orchestra.run.smithi045.stdout:2842 punch from 0x4f298dd to 0x4f36e15, (0xd538 bytes) 2024-09-17T07:29:35.203 INFO:teuthology.orchestra.run.smithi045.stdout:2845 trunc from 0xaef813c to 0x391fe7 2024-09-17T07:29:35.212 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:35.212 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:35.221 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:35.229 INFO:teuthology.orchestra.run.smithi045.stdout:2846 write 0x45d8650 thru 0x45dd695 (0x5046 bytes) 2024-09-17T07:29:35.231 INFO:teuthology.orchestra.run.smithi045.stdout:2847 write 0x1a7caa8 thru 0x1a8b702 (0xec5b bytes) 2024-09-17T07:29:35.234 INFO:teuthology.orchestra.run.smithi045.stdout:2848 read 0x4436cfd thru 0x4441483 (0xa787 bytes) 2024-09-17T07:29:35.235 INFO:teuthology.orchestra.run.smithi045.stdout:2849 trunc from 0x45dd696 to 0x4bbb441 2024-09-17T07:29:35.240 INFO:teuthology.orchestra.run.smithi045.stdout:2850 read 0x3647380 thru 0x3648dbc (0x1a3d bytes) 2024-09-17T07:29:35.240 INFO:teuthology.orchestra.run.smithi045.stdout:2851 read 0x44faec7 thru 0x44ff957 (0x4a91 bytes) 2024-09-17T07:29:35.243 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:29:35.245 INFO:teuthology.orchestra.run.smithi045.stdout:2852 trunc from 0x4bbb441 to 0x92a127e 2024-09-17T07:29:35.253 INFO:teuthology.orchestra.run.smithi045.stdout:2853 read 0x299ed08 thru 0x29a78f1 (0x8bea bytes) 2024-09-17T07:29:35.253 INFO:teuthology.orchestra.run.smithi045.stdout:2854 read 0x602d1f2 thru 0x6030c08 (0x3a17 bytes) 2024-09-17T07:29:35.253 INFO:teuthology.orchestra.run.smithi045.stdout:2855 punch from 0x6484bd0 to 0x648a8ac, (0x5cdc bytes) 2024-09-17T07:29:35.253 INFO:teuthology.orchestra.run.smithi045.stdout:2857 punch from 0x72823c3 to 0x7291e40, (0xfa7d bytes) 2024-09-17T07:29:35.254 INFO:teuthology.orchestra.run.smithi045.stdout:2858 trunc from 0x92a127e to 0x6714051 2024-09-17T07:29:35.265 INFO:teuthology.orchestra.run.smithi045.stdout:2859 trunc from 0x6714051 to 0x9c15cda 2024-09-17T07:29:35.266 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:35.266+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:35.280 INFO:teuthology.orchestra.run.smithi045.stdout:2860 trunc from 0x9c15cda to 0x824829 2024-09-17T07:29:35.290 INFO:teuthology.orchestra.run.smithi045.stdout:2862 read 0x17ec62 thru 0x18df4a (0xf2e9 bytes) 2024-09-17T07:29:35.290 INFO:teuthology.orchestra.run.smithi045.stdout:2863 read 0x1d7585 thru 0x1dfd97 (0x8813 bytes) 2024-09-17T07:29:35.304 INFO:teuthology.orchestra.run.smithi045.stdout:2864 write 0x64073b8 thru 0x640d619 (0x6262 bytes) 2024-09-17T07:29:35.307 INFO:teuthology.orchestra.run.smithi045.stdout:2865 punch from 0x425ba65 to 0x4261d34, (0x62cf bytes) 2024-09-17T07:29:35.307 INFO:teuthology.orchestra.run.smithi045.stdout:2866 read 0x54985a9 thru 0x54a718a (0xebe2 bytes) 2024-09-17T07:29:35.320 INFO:teuthology.orchestra.run.smithi045.stdout:2867 writesame 0xc15dcf4 thru 0xc16a3db data_size 0x26(0xc6e8 bytes) 2024-09-17T07:29:35.321 INFO:teuthology.orchestra.run.smithi045.stdout:2868 writesame 0x8ab0bcf thru 0x8abee3b data_size 0x1(0xe26d bytes) 2024-09-17T07:29:35.329 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:35.329 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:35.339 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:35.343 INFO:teuthology.orchestra.run.smithi045.stdout:2869 read 0x798184a thru 0x798ea38 (0xd1ef bytes) 2024-09-17T07:29:35.344 INFO:teuthology.orchestra.run.smithi045.stdout:2870 punch from 0x13ea8b0 to 0x13f9309, (0xea59 bytes) 2024-09-17T07:29:35.344 INFO:teuthology.orchestra.run.smithi045.stdout:2872 write 0x2244fb5 thru 0x22472bb (0x2307 bytes) 2024-09-17T07:29:35.347 INFO:teuthology.orchestra.run.smithi045.stdout:2876 read 0xb73e4be thru 0xb748a47 (0xa58a bytes) 2024-09-17T07:29:35.347 INFO:teuthology.orchestra.run.smithi045.stdout:2878 read 0x26c3b9f thru 0x26cf595 (0xb9f7 bytes) 2024-09-17T07:29:35.348 INFO:teuthology.orchestra.run.smithi045.stdout:2879 writesame 0x8c6e82e thru 0x8c71645 data_size 0x3b(0x2e18 bytes) 2024-09-17T07:29:35.352 INFO:teuthology.orchestra.run.smithi045.stdout:2880 writesame 0x3b3c1e9 thru 0x3b4b4b7 data_size 0x1(0xf2cf bytes) 2024-09-17T07:29:35.381 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:35.380+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:35.382 INFO:teuthology.orchestra.run.smithi045.stdout:2881 writesame 0xbf0dc0b thru 0xbf11208 data_size 0x1(0x35fe bytes) 2024-09-17T07:29:35.384 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:35.384 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:35.390 INFO:teuthology.orchestra.run.smithi045.stdout:2882 trunc from 0xc16a3dc to 0xad01f1e 2024-09-17T07:29:35.398 INFO:teuthology.orchestra.run.smithi045.stdout:2883 write 0x8df0683 thru 0x8df9dcd (0x974b bytes) 2024-09-17T07:29:35.399 INFO:teuthology.orchestra.run.smithi045.stdout:2885 write 0x5133a44 thru 0x5139e3f (0x63fc bytes) 2024-09-17T07:29:35.401 INFO:teuthology.orchestra.run.smithi045.stdout:2886 write 0x846864c thru 0x846a400 (0x1db5 bytes) 2024-09-17T07:29:35.404 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:35.404 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:35.405 INFO:teuthology.orchestra.run.smithi045.stdout:2891 read 0xa017577 thru 0xa025f64 (0xe9ee bytes) 2024-09-17T07:29:35.405 INFO:teuthology.orchestra.run.smithi045.stdout:2892 write 0x7b39ebc thru 0x7b41482 (0x75c7 bytes) 2024-09-17T07:29:35.406 INFO:teuthology.orchestra.run.smithi045.stdout:2893 write 0x26f331c thru 0x26fb140 (0x7e25 bytes) 2024-09-17T07:29:35.412 INFO:teuthology.orchestra.run.smithi045.stdout:2894 trunc from 0xad01f1e to 0x6484d6c 2024-09-17T07:29:35.423 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:35.423 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:35.424 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:35.425 INFO:teuthology.orchestra.run.smithi045.stdout:2895 write 0x75c78ca thru 0x75d52b7 (0xd9ee bytes) 2024-09-17T07:29:35.426 INFO:teuthology.orchestra.run.smithi045.stdout:2896 punch from 0x68a3368 to 0x68a4c12, (0x18aa bytes) 2024-09-17T07:29:35.426 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.434 INFO:teuthology.orchestra.run.smithi045.stdout:2900 trunc from 0x75d52b8 to 0xe0fa6a7 2024-09-17T07:29:35.443 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:35.443 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:35.444 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.444 INFO:teuthology.orchestra.run.smithi045.stdout:2902 write 0x4039a0f thru 0x4044748 (0xad3a bytes) 2024-09-17T07:29:35.445 INFO:teuthology.orchestra.run.smithi045.stdout:2903 writesame 0xd98b5a2 thru 0xd9944f2 data_size 0x1(0x8f51 bytes) 2024-09-17T07:29:35.460 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:35.460 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:35.460 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:35.459+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:35.462 INFO:teuthology.orchestra.run.smithi045.stdout:2904 write 0x930aa01 thru 0x930e899 (0x3e99 bytes) 2024-09-17T07:29:35.463 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.463 INFO:teuthology.orchestra.run.smithi045.stdout:2906 read 0x817943 thru 0x81cc0e (0x52cc bytes) 2024-09-17T07:29:35.464 INFO:teuthology.orchestra.run.smithi045.stdout:2907 read 0x2bcf6cd thru 0x2bd7e56 (0x878a bytes) 2024-09-17T07:29:35.464 INFO:teuthology.orchestra.run.smithi045.stdout:2909 writesame 0xa28eb7c thru 0xa290d51 data_size 0x2(0x21d6 bytes) 2024-09-17T07:29:35.466 INFO:teuthology.orchestra.run.smithi045.stdout:2912 read 0x24f2cac thru 0x24fb3d2 (0x8727 bytes) 2024-09-17T07:29:35.467 INFO:teuthology.orchestra.run.smithi045.stdout:2913 read 0xd1b542 thru 0xd228da (0x7399 bytes) 2024-09-17T07:29:35.467 INFO:teuthology.orchestra.run.smithi045.stdout:2915 write 0x25871a6 thru 0x258bfc8 (0x4e23 bytes) 2024-09-17T07:29:35.469 INFO:teuthology.orchestra.run.smithi045.stdout:2916 writesame 0x6f92091 thru 0x6f9e9e0 data_size 0x8(0xc950 bytes) 2024-09-17T07:29:35.472 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.472 INFO:teuthology.orchestra.run.smithi045.stdout:2918 writesame 0x18b87ad thru 0x18bfd4e data_size 0x3f(0x75a2 bytes) 2024-09-17T07:29:35.473 INFO:teuthology.orchestra.run.smithi045.stdout:2919 read 0x2989928 thru 0x2995d4f (0xc428 bytes) 2024-09-17T07:29:35.474 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.474 INFO:teuthology.orchestra.run.smithi045.stdout:2922 read 0xa6ca577 thru 0xa6cc6d1 (0x215b bytes) 2024-09-17T07:29:35.474 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.474 INFO:teuthology.orchestra.run.smithi045.stdout:2924 write 0x663b74a thru 0x663c3af (0xc66 bytes) 2024-09-17T07:29:35.475 INFO:teuthology.orchestra.run.smithi045.stdout:2925 read 0xfcf636 thru 0xfd6ddb (0x77a6 bytes) 2024-09-17T07:29:35.476 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.476 INFO:teuthology.orchestra.run.smithi045.stdout:2927 read 0x7167d29 thru 0x71722d7 (0xa5af bytes) 2024-09-17T07:29:35.476 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.477 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:35.478 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:35.482 INFO:teuthology.orchestra.run.smithi045.stdout:2929 trunc from 0xe0fa6a7 to 0x8e7495e 2024-09-17T07:29:35.489 INFO:teuthology.orchestra.run.smithi045.stdout:2930 read 0x498ec5f thru 0x49923d8 (0x377a bytes) 2024-09-17T07:29:35.492 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:35.493 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:35.493 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:35.493 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:35.494 INFO:teuthology.orchestra.run.smithi045.stdout:2931 write 0x935b35b thru 0x9361ebb (0x6b61 bytes) 2024-09-17T07:29:35.496 INFO:teuthology.orchestra.run.smithi045.stdout:2932 read 0x58d38d9 thru 0x58d61f3 (0x291b bytes) 2024-09-17T07:29:35.496 INFO:teuthology.orchestra.run.smithi045.stdout:2935 read 0x32b9997 thru 0x32c8b7c (0xf1e6 bytes) 2024-09-17T07:29:35.497 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.497 INFO:teuthology.orchestra.run.smithi045.stdout:2939 write 0x533a8e9 thru 0x5343f66 (0x967e bytes) 2024-09-17T07:29:35.497 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:35.498 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:35.500 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:35.501 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:35.503 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:35.505 INFO:teuthology.orchestra.run.smithi045.stdout:2940 write 0xabd3015 thru 0xabd4829 (0x1815 bytes) 2024-09-17T07:29:35.506 INFO:teuthology.orchestra.run.smithi045.stdout:2941 read 0x790996e thru 0x79156c0 (0xbd53 bytes) 2024-09-17T07:29:35.507 INFO:teuthology.orchestra.run.smithi045.stdout:2942 punch from 0x69aeb71 to 0x69b8359, (0x97e8 bytes) 2024-09-17T07:29:35.513 INFO:teuthology.orchestra.run.smithi045.stdout:2944 writesame 0xbd4a665 thru 0xbd4e3e4 data_size 0x40(0x3d80 bytes) 2024-09-17T07:29:35.514 INFO:teuthology.orchestra.run.smithi045.stdout:2945 writesame 0x2fb5d59 thru 0x2fba855 data_size 0x4f(0x4afd bytes) 2024-09-17T07:29:35.516 INFO:teuthology.orchestra.run.smithi045.stdout:2946 write 0x7ae26cb thru 0x7ae47da (0x2110 bytes) 2024-09-17T07:29:35.517 INFO:teuthology.orchestra.run.smithi045.stdout:2947 write 0x574843c thru 0x574fbbd (0x7782 bytes) 2024-09-17T07:29:35.518 INFO:teuthology.orchestra.run.smithi045.stdout:2949 writesame 0x5e16eb5 thru 0x5e249bb data_size 0x1(0xdb07 bytes) 2024-09-17T07:29:35.545 INFO:teuthology.orchestra.run.smithi045.stdout:2950 punch from 0x59c2588 to 0x59d093a, (0xe3b2 bytes) 2024-09-17T07:29:35.545 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.551 INFO:teuthology.orchestra.run.smithi045.stdout:2952 trunc from 0xbd4e3e5 to 0x5c9f284 2024-09-17T07:29:35.558 INFO:teuthology.orchestra.run.smithi045.stdout:2953 read 0x19cf87c thru 0x19ddec9 (0xe64e bytes) 2024-09-17T07:29:35.559 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.559 INFO:teuthology.orchestra.run.smithi045.stdout:2956 write 0x2d2da0b thru 0x2d3bf91 (0xe587 bytes) 2024-09-17T07:29:35.561 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:35.561+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:35.569 INFO:teuthology.orchestra.run.smithi045.stdout:2957 write 0x9a6104f thru 0x9a6f9e3 (0xe995 bytes) 2024-09-17T07:29:35.570 INFO:teuthology.orchestra.run.smithi045.stdout:2959 write 0x25ae9a0 thru 0x25b9aef (0xb150 bytes) 2024-09-17T07:29:35.572 INFO:teuthology.orchestra.run.smithi045.stdout:2960 write 0x4919b44 thru 0x4928c45 (0xf102 bytes) 2024-09-17T07:29:35.574 INFO:teuthology.orchestra.run.smithi045.stdout:2962 read 0x24fad84 thru 0x24fdf0d (0x318a bytes) 2024-09-17T07:29:35.574 INFO:teuthology.orchestra.run.smithi045.stdout:2963 punch from 0x80b0a46 to 0x80bbdb5, (0xb36f bytes) 2024-09-17T07:29:35.574 INFO:teuthology.orchestra.run.smithi045.stdout:2964 read 0x5c81c7a thru 0x5c85c48 (0x3fcf bytes) 2024-09-17T07:29:35.575 INFO:teuthology.orchestra.run.smithi045.stdout:2966 read 0x1100cfc thru 0x110a41f (0x9724 bytes) 2024-09-17T07:29:35.575 INFO:teuthology.orchestra.run.smithi045.stdout:2970 trunc from 0x9a6f9e4 to 0x933c7be 2024-09-17T07:29:35.588 INFO:teuthology.orchestra.run.smithi045.stdout:2972 write 0xa06e5bd thru 0xa07b1ab (0xcbef bytes) 2024-09-17T07:29:35.599 INFO:teuthology.orchestra.run.smithi045.stdout:2973 trunc from 0xa07b1ac to 0x135ff76 2024-09-17T07:29:35.614 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:35.615 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:35.620 INFO:teuthology.orchestra.run.smithi045.stdout:2975 write 0x8de465a thru 0x8de950f (0x4eb6 bytes) 2024-09-17T07:29:35.621 INFO:teuthology.orchestra.run.smithi045.stdout:2976 writesame 0x608f921 thru 0x6096ee2 data_size 0x1(0x75c2 bytes) 2024-09-17T07:29:35.624 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:35.635 INFO:teuthology.orchestra.run.smithi045.stdout:2977 read 0x6d2b3f5 thru 0x6d2c2d4 (0xee0 bytes) 2024-09-17T07:29:35.635 INFO:teuthology.orchestra.run.smithi045.stdout:2978 write 0x3ee85fd thru 0x3ef65c5 (0xdfc9 bytes) 2024-09-17T07:29:35.641 INFO:teuthology.orchestra.run.smithi045.stdout:2980 trunc from 0x8de9510 to 0xbe788d2 2024-09-17T07:29:35.646 INFO:teuthology.orchestra.run.smithi045.stdout:2981 write 0x89213c3 thru 0x892283b (0x1479 bytes) 2024-09-17T07:29:35.647 INFO:teuthology.orchestra.run.smithi045.stdout:2982 read 0x63a5206 thru 0x63aa9fc (0x57f7 bytes) 2024-09-17T07:29:35.647 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.647 INFO:teuthology.orchestra.run.smithi045.stdout:2984 read 0x3d6a6e3 thru 0x3d6ebaf (0x44cd bytes) 2024-09-17T07:29:35.648 INFO:teuthology.orchestra.run.smithi045.stdout:2985 read 0x5f9f03b thru 0x5fae574 (0xf53a bytes) 2024-09-17T07:29:35.648 INFO:teuthology.orchestra.run.smithi045.stdout:2986 write 0x71497b8 thru 0x714b92b (0x2174 bytes) 2024-09-17T07:29:35.650 INFO:teuthology.orchestra.run.smithi045.stdout:2988 write 0x6861c46 thru 0x686d1e3 (0xb59e bytes) 2024-09-17T07:29:35.651 INFO:teuthology.orchestra.run.smithi045.stdout:2989 write 0x25846b4 thru 0x25928e5 (0xe232 bytes) 2024-09-17T07:29:35.653 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.653 INFO:teuthology.orchestra.run.smithi045.stdout:2992 read 0xb45421f thru 0xb462617 (0xe3f9 bytes) 2024-09-17T07:29:35.653 INFO:teuthology.orchestra.run.smithi045.stdout:2994 writesame 0x40ec790 thru 0x40f05ad data_size 0x1(0x3e1e bytes) 2024-09-17T07:29:35.662 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.662 INFO:teuthology.orchestra.run.smithi045.stdout:2997 write 0x95ed38f thru 0x95f2e45 (0x5ab7 bytes) 2024-09-17T07:29:35.662 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:35.661+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:35.663 INFO:teuthology.orchestra.run.smithi045.stdout:2998 write 0x93d1ca0 thru 0x93e17d0 (0xfb31 bytes) 2024-09-17T07:29:35.664 INFO:teuthology.orchestra.run.smithi045.stdout:2999 writesame 0x30fc235 thru 0x31086f1 data_size 0x7(0xc4bd bytes) 2024-09-17T07:29:35.668 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.669 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.669 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.675 INFO:teuthology.orchestra.run.smithi045.stdout:3006 write 0xd35ede1 thru 0xd367cdd (0x8efd bytes) 2024-09-17T07:29:35.682 INFO:teuthology.orchestra.run.smithi045.stdout:3007 trunc from 0xd367cde to 0x874d2bc 2024-09-17T07:29:35.693 INFO:teuthology.orchestra.run.smithi045.stdout:3008 trunc from 0x874d2bc to 0xbd431ac 2024-09-17T07:29:35.698 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.698 INFO:teuthology.orchestra.run.smithi045.stdout:3010 read 0x4fa2c8f thru 0x4fb29b1 (0xfd23 bytes) 2024-09-17T07:29:35.704 INFO:teuthology.orchestra.run.smithi045.stdout:3011 trunc from 0xbd431ac to 0x6f70970 2024-09-17T07:29:35.721 INFO:teuthology.orchestra.run.smithi045.stdout:3012 write 0xc2dc541 thru 0xc2dcf6b (0xa2b bytes) 2024-09-17T07:29:35.722 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.729 INFO:teuthology.orchestra.run.smithi045.stdout:3014 writesame 0xd42f12c thru 0xd435c9e data_size 0x35(0x6b73 bytes) 2024-09-17T07:29:35.730 INFO:teuthology.orchestra.run.smithi045.stdout:3015 write 0x4b1a5d7 thru 0x4b1c2bd (0x1ce7 bytes) 2024-09-17T07:29:35.731 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.745 INFO:teuthology.orchestra.run.smithi045.stdout:3018 trunc from 0xd435c9f to 0x12f9e4f 2024-09-17T07:29:35.759 INFO:teuthology.orchestra.run.smithi045.stdout:3019 writesame 0x28fee16 thru 0x290e467 data_size 0x29(0xf652 bytes) 2024-09-17T07:29:35.762 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:35.762+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:35.772 INFO:teuthology.orchestra.run.smithi045.stdout:3020 write 0x802f171 thru 0x803530d (0x619d bytes) 2024-09-17T07:29:35.773 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.866879976, 2024-09-17T07:29:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:29:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:29:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:29:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:29:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:29:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:29:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:29:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:29:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:29:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:29:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:29:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:29:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:29:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:29:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:29:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.866861299, 2024-09-17T07:29:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:29:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:29:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:29:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:29:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:29:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:29:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:29:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:29:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:29:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:29:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:29:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:29:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:29:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:29:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:29:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.866842945999998, 2024-09-17T07:29:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:29:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:29:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:29:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:29:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:29:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:29:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:29:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:29:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:29:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:29:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:29:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:29:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:29:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:29:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:29:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.866823348, 2024-09-17T07:29:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:29:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:29:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:29:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:29:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:29:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:29:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:29:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:29:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:29:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:29:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:29:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:29:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:29:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:29:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:29:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:29:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.866803393000001, 2024-09-17T07:29:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:29:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:29:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:29:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:29:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:29:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:29:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:29:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:29:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:29:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:29:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:29:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:29:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:29:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:29:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:29:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.866784738, 2024-09-17T07:29:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:29:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:29:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:29:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:29:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:29:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:29:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:29:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:29:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:29:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:29:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:29:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:29:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:29:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:29:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:29:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.866766261999999, 2024-09-17T07:29:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:29:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:29:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:29:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:29:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:29:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:29:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:29:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:29:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:29:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:29:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:29:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:29:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:29:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:29:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:29:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:29:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.866746663000001, 2024-09-17T07:29:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:29:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:29:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:29:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:29:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:29:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:29:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:29:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:29:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:29:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:29:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:29:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:29:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:29:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:29:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:29:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.866728087999999, 2024-09-17T07:29:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:29:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:29:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:29:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:29:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:29:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:29:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:29:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:29:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:29:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:29:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:29:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:29:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:29:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:29:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:29:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.866705661000001, 2024-09-17T07:29:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:29:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:29:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:29:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:29:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:29:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:29:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:29:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:29:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:29:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:29:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:29:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:29:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:29:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:29:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.866677844000002, 2024-09-17T07:29:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:29:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:29:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:29:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:29:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:29:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:29:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:29:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:29:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:29:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:29:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:29:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:29:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:29:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:29:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:29:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.866655900000001, 2024-09-17T07:29:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:29:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:29:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:29:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:29:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:29:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:29:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:29:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:29:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:29:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:29:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:29:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:29:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:29:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:29:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:29:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.866634796, 2024-09-17T07:29:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:29:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:29:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:29:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:29:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:29:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:29:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:29:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:29:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:29:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:29:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:29:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:29:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:29:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:29:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.866615193000001, 2024-09-17T07:29:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:29:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:29:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:29:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:29:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:29:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:29:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:29:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:29:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:29:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:29:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:29:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:29:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:29:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:29:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:29:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:29:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.91367408, 2024-09-17T07:29:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:29:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:29:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:29:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:29:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:29:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:29:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:29:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:29:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:29:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:29:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.913660419999999, 2024-09-17T07:29:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:29:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:29:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:29:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:29:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:29:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:29:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:29:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:29:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:29:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:29:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:29:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:29:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:29:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:29:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:29:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.913645638, 2024-09-17T07:29:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:29:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:29:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:29:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:29:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:29:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:29:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:29:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:29:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:29:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:29:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:29:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:29:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:29:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:29:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:29:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.913634747, 2024-09-17T07:29:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:29:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:29:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:29:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:29:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:29:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:29:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:29:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:29:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:29:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:29:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:29:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:29:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:29:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:29:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.913625186999999, 2024-09-17T07:29:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:29:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:29:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:29:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:29:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:29:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:29:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:29:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:29:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:29:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:29:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:29:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:29:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:29:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:29:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:29:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:35.863 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.913614626999999, 2024-09-17T07:29:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:29:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:29:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:29:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:29:35.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:29:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:29:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:35.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:29:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:29:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:29:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:29:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:29:35.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:29:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:29:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:29:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:29:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:29:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:35.870 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:35.870 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:35.872 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:35.872 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:35.870+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:35.873 INFO:teuthology.orchestra.run.smithi045.stdout:3021 read 0x1ea3070 thru 0x1eac404 (0x9395 bytes) 2024-09-17T07:29:35.873 INFO:teuthology.orchestra.run.smithi045.stdout:3022 write 0xe3917 thru 0xf32f2 (0xf9dc bytes) 2024-09-17T07:29:35.874 INFO:teuthology.orchestra.run.smithi045.stdout:3025 read 0x5492f79 thru 0x54963aa (0x3432 bytes) 2024-09-17T07:29:35.874 INFO:teuthology.orchestra.run.smithi045.stdout:3026 punch from 0x648605e to 0x648dbce, (0x7b70 bytes) 2024-09-17T07:29:35.874 INFO:teuthology.orchestra.run.smithi045.stdout:3027 write 0x92e2b8 thru 0x930b74 (0x28bd bytes) 2024-09-17T07:29:35.874 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.874 INFO:teuthology.orchestra.run.smithi045.stdout:3030 punch from 0x6120c69 to 0x61249b3, (0x3d4a bytes) 2024-09-17T07:29:35.874 INFO:teuthology.orchestra.run.smithi045.stdout:3031 write 0xac75042 thru 0xac76848 (0x1807 bytes) 2024-09-17T07:29:35.874 INFO:teuthology.orchestra.run.smithi045.stdout:3032 read 0x395bd37 thru 0x395dd50 (0x201a bytes) 2024-09-17T07:29:35.874 INFO:teuthology.orchestra.run.smithi045.stdout:3033 write 0x4aa7427 thru 0x4aaee1a (0x79f4 bytes) 2024-09-17T07:29:35.874 INFO:teuthology.orchestra.run.smithi045.stdout:3034 trunc from 0xac76849 to 0x85ba343 2024-09-17T07:29:35.874 INFO:teuthology.orchestra.run.smithi045.stdout:3036 write 0x6cf4482 thru 0x6cf6688 (0x2207 bytes) 2024-09-17T07:29:35.874 INFO:teuthology.orchestra.run.smithi045.stdout:3039 write 0x26e8024 thru 0x26ee428 (0x6405 bytes) 2024-09-17T07:29:35.874 INFO:teuthology.orchestra.run.smithi045.stdout:3041 read 0x6a299b6 thru 0x6a37f63 (0xe5ae bytes) 2024-09-17T07:29:35.874 INFO:teuthology.orchestra.run.smithi045.stdout:3042 read 0x6984333 thru 0x69919bb (0xd689 bytes) 2024-09-17T07:29:35.875 INFO:teuthology.orchestra.run.smithi045.stdout:3043 read 0x46001ca thru 0x460663a (0x6471 bytes) 2024-09-17T07:29:35.875 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:35.875 INFO:teuthology.orchestra.run.smithi045.stdout:3047 write 0x484b727 thru 0x4852dc1 (0x769b bytes) 2024-09-17T07:29:35.875 INFO:teuthology.orchestra.run.smithi045.stdout:3048 clone 31 order 21 su 8192 sc 8 2024-09-17T07:29:35.973 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:35.972+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:36.016 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:36.016 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:36.024 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:36.108 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:36.108+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:36.167 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:36.168 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:36.168 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:36.168 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:36.168 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:36.174 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:36.174+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:36.176 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:36.278 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:36.277+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:36.312 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T07:29:36.324 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:36.324 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:36.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:36.324 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:36.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.325 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:29:36.325 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:36.325 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.988365723000001, 2024-09-17T07:29:36.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:29:36.325 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.325 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.325 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:36.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:29:36.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:29:36.325 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:29:36.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.326 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:36.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:36.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:29:36.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:29:36.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:29:36.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:29:36.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:29:36.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:29:36.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:29:36.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:29:36.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:29:36.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:29:36.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:29:36.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:29:36.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:29:36.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:29:36.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:29:36.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:29:36.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:29:36.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:29:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:29:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:29:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:29:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:29:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:29:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:29:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:36.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:29:36.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:29:36.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:29:36.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:29:36.331 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.331 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.331 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7378 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7176192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.399323688999999, 2024-09-17T07:29:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073003980999999996, 2024-09-17T07:29:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7378 2024-09-17T07:29:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:36.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:36.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:36.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1799999999999999e-07 2024-09-17T07:29:36.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923713+0000", 2024-09-17T07:29:36.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.057e-06 2024-09-17T07:29:36.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923716+0000", 2024-09-17T07:29:36.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0599999999999999e-06 2024-09-17T07:29:36.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923719+0000", 2024-09-17T07:29:36.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8219999999999998e-06 2024-09-17T07:29:36.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996401+0000", 2024-09-17T07:29:36.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072681467999999999 2024-09-17T07:29:36.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996452+0000", 2024-09-17T07:29:36.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.134e-05 2024-09-17T07:29:36.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996716+0000", 2024-09-17T07:29:36.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000264016 2024-09-17T07:29:36.336 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.336 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.336 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.336 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.336 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:36.336 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.399313573000001, 2024-09-17T07:29:36.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:29:36.337 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.337 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.337 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.337 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:29:36.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.337 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:36.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:36.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:29:36.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:29:36.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:29:36.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:29:36.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:29:36.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:29:36.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:29:36.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:29:36.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:29:36.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:29:36.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:29:36.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:29:36.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:29:36.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:29:36.342 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.342 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.342 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.342 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7398 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6389760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.342 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:36.342 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.394045038999998, 2024-09-17T07:29:36.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072891580999999997, 2024-09-17T07:29:36.342 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.343 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.343 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.343 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.343 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.343 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.343 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7398 2024-09-17T07:29:36.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.343 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:36.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:36.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:36.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:36.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928995+0000", 2024-09-17T07:29:36.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0759999999999996e-06 2024-09-17T07:29:36.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928999+0000", 2024-09-17T07:29:36.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1609999999999996e-06 2024-09-17T07:29:36.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929002+0000", 2024-09-17T07:29:36.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8449999999999999e-06 2024-09-17T07:29:36.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001598+0000", 2024-09-17T07:29:36.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072596318000000007 2024-09-17T07:29:36.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001663+0000", 2024-09-17T07:29:36.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4995999999999999e-05 2024-09-17T07:29:36.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001882+0000", 2024-09-17T07:29:36.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021897299999999999 2024-09-17T07:29:36.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.347 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.347 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.347 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:36.347 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.394030457, 2024-09-17T07:29:36.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:29:36.347 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.347 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.347 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.348 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:29:36.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.348 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:36.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:36.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:36.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:29:36.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:29:36.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:29:36.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:29:36.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:29:36.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:29:36.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:29:36.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:29:36.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:29:36.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:29:36.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:29:36.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:29:36.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:29:36.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.351 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.352 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.352 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:36.352 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.393999477000001, 2024-09-17T07:29:36.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:29:36.352 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.352 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.352 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.353 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:29:36.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.353 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:36.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:36.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:29:36.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:29:36.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:29:36.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:29:36.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:29:36.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:29:36.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:29:36.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:29:36.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:29:36.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:29:36.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:29:36.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:29:36.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:29:36.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:29:36.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.357 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.357 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.357 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:36.357 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.393969427999998, 2024-09-17T07:29:36.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:29:36.357 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.357 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.357 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.358 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:29:36.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.358 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:36.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:36.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:29:36.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:29:36.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:29:36.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:29:36.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:29:36.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:29:36.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:29:36.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:29:36.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:29:36.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:29:36.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:29:36.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:29:36.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:29:36.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:29:36.361 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.361 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.361 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.361 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.361 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:36.362 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.393941021000003, 2024-09-17T07:29:36.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:29:36.362 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.362 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.362 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.362 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:29:36.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.362 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:36.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:36.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:36.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:36.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:29:36.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:29:36.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:29:36.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:29:36.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:29:36.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:29:36.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:29:36.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:29:36.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:29:36.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:29:36.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:29:36.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:29:36.366 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.366 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.366 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.366 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.366 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:36.366 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.393921200000001, 2024-09-17T07:29:36.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:29:36.366 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.366 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.366 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.367 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:29:36.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.367 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:36.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:36.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:29:36.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:29:36.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:29:36.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:29:36.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:29:36.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:29:36.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:29:36.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:29:36.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:29:36.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:29:36.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:29:36.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:29:36.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:29:36.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:29:36.370 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.370 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.370 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.371 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.371 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:36.371 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.393896947999998, 2024-09-17T07:29:36.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:29:36.371 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.371 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.371 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.371 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:29:36.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.372 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:36.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:36.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:29:36.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:29:36.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:29:36.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:29:36.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:29:36.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:29:36.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:29:36.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:29:36.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:29:36.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:29:36.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:29:36.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:29:36.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7418 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7700480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.388490924999999, 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072752965000000003, 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.375 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7418 2024-09-17T07:29:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3599999999999999e-07 2024-09-17T07:29:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934547+0000", 2024-09-17T07:29:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3350000000000001e-06 2024-09-17T07:29:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934554+0000", 2024-09-17T07:29:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9859999999999997e-06 2024-09-17T07:29:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934557+0000", 2024-09-17T07:29:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.818e-06 2024-09-17T07:29:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007104+0000", 2024-09-17T07:29:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072546779000000006 2024-09-17T07:29:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007147+0000", 2024-09-17T07:29:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2904000000000001e-05 2024-09-17T07:29:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007298+0000", 2024-09-17T07:29:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015080700000000001 2024-09-17T07:29:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.388474360000004, 2024-09-17T07:29:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:29:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.380 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:29:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:29:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:29:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:29:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:29:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:29:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:29:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:29:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:29:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:29:36.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:29:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:29:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:29:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:29:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:29:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.388427485000001, 2024-09-17T07:29:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:29:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:29:36.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:29:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:29:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:29:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:29:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:29:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:29:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:29:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:29:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:29:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:29:36.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:29:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:29:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.388412781, 2024-09-17T07:29:36.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:29:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:29:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:29:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:29:36.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:29:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:29:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:29:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:29:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:29:36.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:29:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:29:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:29:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:29:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:29:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.388389685999996, 2024-09-17T07:29:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:29:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:29:36.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:29:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:29:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:29:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:29:36.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:29:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:29:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:29:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:29:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:29:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:29:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:29:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:29:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:29:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.398 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.388358310999998, 2024-09-17T07:29:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:29:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:29:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:36.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:29:36.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:29:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:29:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:29:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:29:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:29:36.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:29:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:29:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:29:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:29:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:29:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.388343051, 2024-09-17T07:29:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:29:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:29:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:29:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:29:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:29:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:29:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:29:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:29:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:29:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:29:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:29:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:29:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:29:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:29:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:29:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.388322981000002, 2024-09-17T07:29:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:29:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.407 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:29:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:36.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:29:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:29:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:29:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:29:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:29:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:29:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:29:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:29:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:29:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:29:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:29:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:29:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:29:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:36.411 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.388311041000001, 2024-09-17T07:29:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:29:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:29:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:36.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:29:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:29:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:29:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:29:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:29:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:29:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:29:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:29:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:29:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:29:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:29:36.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:29:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.388287882999997, 2024-09-17T07:29:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:29:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:36.416 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:29:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:29:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:29:36.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:29:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:29:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:29:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:29:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:29:36.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:29:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:29:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:29:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:29:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:29:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:36.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:29:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:29:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:36.421 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:36.421 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:36.422 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone30 from 0x85ba343 (overlap 0x391fe7) to 0x16d98f 2024-09-17T07:29:36.422 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:36.422+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:36.423 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:36.464 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-17T07:29:36.465 INFO:teuthology.orchestra.run.smithi045.stdout:3049 write 0x4ebcca4 thru 0x4ec592c (0x8c89 bytes) 2024-09-17T07:29:36.523 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:36.523+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:36.539 INFO:teuthology.orchestra.run.smithi045.stdout:3050 read 0x81096e4 thru 0x8110e9c (0x77b9 bytes) 2024-09-17T07:29:36.540 INFO:teuthology.orchestra.run.smithi045.stdout:3051 read 0x6cbe490 thru 0x6ccbdb3 (0xd924 bytes) 2024-09-17T07:29:36.541 INFO:teuthology.orchestra.run.smithi045.stdout:3052 read 0x30e385 thru 0x316981 (0x85fd bytes) 2024-09-17T07:29:36.543 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:36.543 INFO:teuthology.orchestra.run.smithi045.stdout:3054 write 0x1b1be4e thru 0x1b27c9c (0xbe4f bytes) 2024-09-17T07:29:36.574 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:36.574 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:36.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.574 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:68849 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.574 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.554634+0000", 2024-09-17T07:29:36.575 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012887378, 2024-09-17T07:29:36.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.012911857000000001, 2024-09-17T07:29:36.575 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.575 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.575 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.575 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.575 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.575 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.575 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 68849 2024-09-17T07:29:36.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.575 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.575 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.576 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554634+0000", 2024-09-17T07:29:36.576 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.576 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.576 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554634+0000", 2024-09-17T07:29:36.576 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.576 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554635+0000", 2024-09-17T07:29:36.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:29:36.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.577 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554637+0000", 2024-09-17T07:29:36.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.013e-06 2024-09-17T07:29:36.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.577 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554643+0000", 2024-09-17T07:29:36.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0419999999999999e-06 2024-09-17T07:29:36.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.578 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.578 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554647+0000", 2024-09-17T07:29:36.578 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5800000000000002e-06 2024-09-17T07:29:36.578 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.578 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.578 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.578 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69105 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.578 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555651+0000", 2024-09-17T07:29:36.578 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011870772999999999, 2024-09-17T07:29:36.578 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011982374, 2024-09-17T07:29:36.579 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.579 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.579 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.579 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.579 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.579 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.579 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69105 2024-09-17T07:29:36.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.579 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.579 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.579 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555651+0000", 2024-09-17T07:29:36.579 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.580 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.580 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555651+0000", 2024-09-17T07:29:36.580 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.580 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.580 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555651+0000", 2024-09-17T07:29:36.580 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0400000000000002e-07 2024-09-17T07:29:36.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.581 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.581 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555653+0000", 2024-09-17T07:29:36.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.03e-06 2024-09-17T07:29:36.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.581 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.581 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555660+0000", 2024-09-17T07:29:36.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6279999999999999e-06 2024-09-17T07:29:36.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.582 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.582 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555665+0000", 2024-09-17T07:29:36.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6639999999999997e-06 2024-09-17T07:29:36.582 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.582 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.582 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.582 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:68850 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.582 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.554654+0000", 2024-09-17T07:29:36.582 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012867754, 2024-09-17T07:29:36.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013032625000000001, 2024-09-17T07:29:36.582 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.583 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.583 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.583 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.583 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.583 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.583 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 68850 2024-09-17T07:29:36.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.583 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554654+0000", 2024-09-17T07:29:36.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.584 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.584 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554654+0000", 2024-09-17T07:29:36.584 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.584 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.584 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554655+0000", 2024-09-17T07:29:36.584 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0999999999999999e-07 2024-09-17T07:29:36.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.584 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.584 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554657+0000", 2024-09-17T07:29:36.584 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0700000000000001e-06 2024-09-17T07:29:36.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554663+0000", 2024-09-17T07:29:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3289999999999999e-06 2024-09-17T07:29:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554668+0000", 2024-09-17T07:29:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6460000000000001e-06 2024-09-17T07:29:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69106 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555671+0000", 2024-09-17T07:29:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011850698, 2024-09-17T07:29:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.01208324, 2024-09-17T07:29:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69106 2024-09-17T07:29:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555671+0000", 2024-09-17T07:29:36.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555671+0000", 2024-09-17T07:29:36.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.588 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.588 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555671+0000", 2024-09-17T07:29:36.588 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:29:36.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.588 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.588 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555673+0000", 2024-09-17T07:29:36.588 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.869e-06 2024-09-17T07:29:36.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.588 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.588 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555680+0000", 2024-09-17T07:29:36.589 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4219999999999997e-06 2024-09-17T07:29:36.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.589 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.589 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555685+0000", 2024-09-17T07:29:36.589 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.758e-06 2024-09-17T07:29:36.589 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.589 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.589 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.589 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:68851 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.589 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.554675+0000", 2024-09-17T07:29:36.589 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012846396, 2024-09-17T07:29:36.590 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013138370999999999, 2024-09-17T07:29:36.590 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.590 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.590 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.590 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.590 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.590 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 68851 2024-09-17T07:29:36.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.590 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.590 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.590 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554675+0000", 2024-09-17T07:29:36.590 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.591 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.591 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554675+0000", 2024-09-17T07:29:36.591 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.591 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.591 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554676+0000", 2024-09-17T07:29:36.591 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4400000000000001e-07 2024-09-17T07:29:36.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.592 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.592 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554682+0000", 2024-09-17T07:29:36.592 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9059999999999999e-06 2024-09-17T07:29:36.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.592 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.592 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554689+0000", 2024-09-17T07:29:36.592 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9770000000000003e-06 2024-09-17T07:29:36.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.593 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.593 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554694+0000", 2024-09-17T07:29:36.593 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2209999999999996e-06 2024-09-17T07:29:36.593 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.593 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.593 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.593 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69107 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.593 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555691+0000", 2024-09-17T07:29:36.593 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011831072, 2024-09-17T07:29:36.593 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.012172717, 2024-09-17T07:29:36.594 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.594 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.594 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.594 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.594 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.594 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.594 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69107 2024-09-17T07:29:36.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.594 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.594 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.594 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555691+0000", 2024-09-17T07:29:36.594 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.595 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.595 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.595 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555691+0000", 2024-09-17T07:29:36.595 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.595 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.595 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.595 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555691+0000", 2024-09-17T07:29:36.595 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9400000000000001e-07 2024-09-17T07:29:36.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.595 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.595 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.595 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555693+0000", 2024-09-17T07:29:36.596 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7460000000000001e-06 2024-09-17T07:29:36.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.596 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.596 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.596 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555699+0000", 2024-09-17T07:29:36.596 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0039999999999996e-06 2024-09-17T07:29:36.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.596 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.596 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.596 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555709+0000", 2024-09-17T07:29:36.596 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9499999999999996e-06 2024-09-17T07:29:36.596 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.596 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.597 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.597 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:68852 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.597 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.554706+0000", 2024-09-17T07:29:36.597 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012815889, 2024-09-17T07:29:36.597 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013218994, 2024-09-17T07:29:36.597 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.597 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.598 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.598 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.598 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.598 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.598 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 68852 2024-09-17T07:29:36.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.598 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.598 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.598 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554706+0000", 2024-09-17T07:29:36.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.598 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554706+0000", 2024-09-17T07:29:36.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.599 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554706+0000", 2024-09-17T07:29:36.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6199999999999998e-07 2024-09-17T07:29:36.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.599 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554709+0000", 2024-09-17T07:29:36.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.244e-06 2024-09-17T07:29:36.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.599 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.600 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554716+0000", 2024-09-17T07:29:36.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.063e-06 2024-09-17T07:29:36.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.600 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.600 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554721+0000", 2024-09-17T07:29:36.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3650000000000003e-06 2024-09-17T07:29:36.600 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.600 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.600 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.600 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69108 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.600 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555716+0000", 2024-09-17T07:29:36.600 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011806283000000001, 2024-09-17T07:29:36.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.012259961999999999, 2024-09-17T07:29:36.601 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.601 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.601 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.601 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.601 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.601 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.601 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69108 2024-09-17T07:29:36.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.601 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555716+0000", 2024-09-17T07:29:36.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555716+0000", 2024-09-17T07:29:36.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555716+0000", 2024-09-17T07:29:36.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:29:36.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555718+0000", 2024-09-17T07:29:36.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9759999999999998e-06 2024-09-17T07:29:36.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.603 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555724+0000", 2024-09-17T07:29:36.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.195e-06 2024-09-17T07:29:36.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.603 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555729+0000", 2024-09-17T07:29:36.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:29:36.603 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.603 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.603 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:68853 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.554729+0000", 2024-09-17T07:29:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012793212999999999, 2024-09-17T07:29:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013302266, 2024-09-17T07:29:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 68853 2024-09-17T07:29:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554729+0000", 2024-09-17T07:29:36.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554729+0000", 2024-09-17T07:29:36.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554729+0000", 2024-09-17T07:29:36.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8599999999999998e-07 2024-09-17T07:29:36.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554731+0000", 2024-09-17T07:29:36.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.041e-06 2024-09-17T07:29:36.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554737+0000", 2024-09-17T07:29:36.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3470000000000004e-06 2024-09-17T07:29:36.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554744+0000", 2024-09-17T07:29:36.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7440000000000001e-06 2024-09-17T07:29:36.607 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.607 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.607 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.607 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69109 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.607 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555734+0000", 2024-09-17T07:29:36.608 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011787631, 2024-09-17T07:29:36.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.012347505, 2024-09-17T07:29:36.608 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.608 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.608 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.608 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.608 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.608 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.608 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69109 2024-09-17T07:29:36.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.608 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555734+0000", 2024-09-17T07:29:36.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.609 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555734+0000", 2024-09-17T07:29:36.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.609 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555735+0000", 2024-09-17T07:29:36.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8099999999999998e-07 2024-09-17T07:29:36.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555736+0000", 2024-09-17T07:29:36.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.609e-06 2024-09-17T07:29:36.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555742+0000", 2024-09-17T07:29:36.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6609999999999997e-06 2024-09-17T07:29:36.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555746+0000", 2024-09-17T07:29:36.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3970000000000004e-06 2024-09-17T07:29:36.610 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.611 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.611 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.611 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:68854 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.611 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.554751+0000", 2024-09-17T07:29:36.611 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012770597, 2024-09-17T07:29:36.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013378091, 2024-09-17T07:29:36.611 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.611 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.611 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.611 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.612 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.612 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.612 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 68854 2024-09-17T07:29:36.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.612 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554751+0000", 2024-09-17T07:29:36.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554751+0000", 2024-09-17T07:29:36.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554752+0000", 2024-09-17T07:29:36.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.08e-07 2024-09-17T07:29:36.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554754+0000", 2024-09-17T07:29:36.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.119e-06 2024-09-17T07:29:36.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554760+0000", 2024-09-17T07:29:36.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2210000000000002e-06 2024-09-17T07:29:36.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554765+0000", 2024-09-17T07:29:36.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8359999999999999e-06 2024-09-17T07:29:36.614 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.614 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.614 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.614 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69110 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.615 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555752+0000", 2024-09-17T07:29:36.615 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011770101999999999, 2024-09-17T07:29:36.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.012424456, 2024-09-17T07:29:36.615 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.615 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.615 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.615 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.615 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69110 2024-09-17T07:29:36.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.615 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555752+0000", 2024-09-17T07:29:36.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555752+0000", 2024-09-17T07:29:36.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555752+0000", 2024-09-17T07:29:36.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:29:36.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555754+0000", 2024-09-17T07:29:36.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7260000000000001e-06 2024-09-17T07:29:36.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555759+0000", 2024-09-17T07:29:36.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3750000000000002e-06 2024-09-17T07:29:36.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555764+0000", 2024-09-17T07:29:36.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3370000000000001e-06 2024-09-17T07:29:36.618 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.618 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.618 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.619 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:68855 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.619 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.554772+0000", 2024-09-17T07:29:36.619 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.01275008, 2024-09-17T07:29:36.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013471093999999999, 2024-09-17T07:29:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 68855 2024-09-17T07:29:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554772+0000", 2024-09-17T07:29:36.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554772+0000", 2024-09-17T07:29:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554772+0000", 2024-09-17T07:29:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2599999999999998e-07 2024-09-17T07:29:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.621 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554774+0000", 2024-09-17T07:29:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2089999999999999e-06 2024-09-17T07:29:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554781+0000", 2024-09-17T07:29:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6279999999999999e-06 2024-09-17T07:29:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.554786+0000", 2024-09-17T07:29:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9830000000000004e-06 2024-09-17T07:29:36.622 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.623 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.623 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.623 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69111 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.623 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555774+0000", 2024-09-17T07:29:36.623 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011747458000000001, 2024-09-17T07:29:36.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0125349, 2024-09-17T07:29:36.623 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69111 2024-09-17T07:29:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555774+0000", 2024-09-17T07:29:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555774+0000", 2024-09-17T07:29:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555775+0000", 2024-09-17T07:29:36.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.58e-07 2024-09-17T07:29:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555777+0000", 2024-09-17T07:29:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1459999999999999e-06 2024-09-17T07:29:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555783+0000", 2024-09-17T07:29:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2990000000000002e-06 2024-09-17T07:29:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555788+0000", 2024-09-17T07:29:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.792e-06 2024-09-17T07:29:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.627 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:68856 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555101+0000", 2024-09-17T07:29:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012420799, 2024-09-17T07:29:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013259969999999999, 2024-09-17T07:29:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.628 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 68856 2024-09-17T07:29:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555101+0000", 2024-09-17T07:29:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555101+0000", 2024-09-17T07:29:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555102+0000", 2024-09-17T07:29:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1600000000000001e-07 2024-09-17T07:29:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555105+0000", 2024-09-17T07:29:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.112e-06 2024-09-17T07:29:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555117+0000", 2024-09-17T07:29:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1959e-05 2024-09-17T07:29:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555123+0000", 2024-09-17T07:29:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4080000000000003e-06 2024-09-17T07:29:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69112 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555796+0000", 2024-09-17T07:29:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011725691999999999, 2024-09-17T07:29:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.012626185, 2024-09-17T07:29:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.632 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69112 2024-09-17T07:29:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555796+0000", 2024-09-17T07:29:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555796+0000", 2024-09-17T07:29:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555797+0000", 2024-09-17T07:29:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7899999999999999e-07 2024-09-17T07:29:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555799+0000", 2024-09-17T07:29:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1789999999999998e-06 2024-09-17T07:29:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555806+0000", 2024-09-17T07:29:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7959999999999998e-06 2024-09-17T07:29:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555810+0000", 2024-09-17T07:29:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9389999999999996e-06 2024-09-17T07:29:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:68857 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555133+0000", 2024-09-17T07:29:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012388362999999999, 2024-09-17T07:29:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013337231999999999, 2024-09-17T07:29:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 68857 2024-09-17T07:29:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555133+0000", 2024-09-17T07:29:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555133+0000", 2024-09-17T07:29:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555134+0000", 2024-09-17T07:29:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3900000000000005e-07 2024-09-17T07:29:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555136+0000", 2024-09-17T07:29:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3640000000000002e-06 2024-09-17T07:29:36.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555143+0000", 2024-09-17T07:29:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1509999999999998e-06 2024-09-17T07:29:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555149+0000", 2024-09-17T07:29:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.288e-06 2024-09-17T07:29:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.638 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69113 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555817+0000", 2024-09-17T07:29:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011704973, 2024-09-17T07:29:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.012710968, 2024-09-17T07:29:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69113 2024-09-17T07:29:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555817+0000", 2024-09-17T07:29:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555817+0000", 2024-09-17T07:29:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555817+0000", 2024-09-17T07:29:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:29:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555819+0000", 2024-09-17T07:29:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:29:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555825+0000", 2024-09-17T07:29:36.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1279999999999996e-06 2024-09-17T07:29:36.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555830+0000", 2024-09-17T07:29:36.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8069999999999999e-06 2024-09-17T07:29:36.642 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.642 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.642 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.642 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:68858 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.642 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555156+0000", 2024-09-17T07:29:36.643 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012365849, 2024-09-17T07:29:36.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013420425, 2024-09-17T07:29:36.643 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.643 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.643 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.643 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.643 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.643 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.644 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 68858 2024-09-17T07:29:36.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.644 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.644 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555156+0000", 2024-09-17T07:29:36.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.644 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555156+0000", 2024-09-17T07:29:36.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555156+0000", 2024-09-17T07:29:36.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5600000000000001e-07 2024-09-17T07:29:36.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555158+0000", 2024-09-17T07:29:36.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9520000000000001e-06 2024-09-17T07:29:36.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555165+0000", 2024-09-17T07:29:36.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6980000000000001e-06 2024-09-17T07:29:36.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555170+0000", 2024-09-17T07:29:36.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0340000000000004e-06 2024-09-17T07:29:36.646 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.646 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.646 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.646 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69114 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.646 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555837+0000", 2024-09-17T07:29:36.646 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011685081999999999, 2024-09-17T07:29:36.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.012785675, 2024-09-17T07:29:36.646 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.647 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.647 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.647 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.647 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.647 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69114 2024-09-17T07:29:36.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.647 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.647 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555837+0000", 2024-09-17T07:29:36.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.648 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555837+0000", 2024-09-17T07:29:36.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.648 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555837+0000", 2024-09-17T07:29:36.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4700000000000002e-07 2024-09-17T07:29:36.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555839+0000", 2024-09-17T07:29:36.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8950000000000001e-06 2024-09-17T07:29:36.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555845+0000", 2024-09-17T07:29:36.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2009999999999996e-06 2024-09-17T07:29:36.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555850+0000", 2024-09-17T07:29:36.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8099999999999997e-06 2024-09-17T07:29:36.650 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.650 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.650 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.650 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:68859 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.650 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555177+0000", 2024-09-17T07:29:36.650 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012344773999999999, 2024-09-17T07:29:36.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013491668, 2024-09-17T07:29:36.650 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.651 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.651 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.651 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.651 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.651 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.651 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 68859 2024-09-17T07:29:36.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.651 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555177+0000", 2024-09-17T07:29:36.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555177+0000", 2024-09-17T07:29:36.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555177+0000", 2024-09-17T07:29:36.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:29:36.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555180+0000", 2024-09-17T07:29:36.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0810000000000001e-06 2024-09-17T07:29:36.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555186+0000", 2024-09-17T07:29:36.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3380000000000001e-06 2024-09-17T07:29:36.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555191+0000", 2024-09-17T07:29:36.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7339999999999999e-06 2024-09-17T07:29:36.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.653 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.654 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69115 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.654 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555856+0000", 2024-09-17T07:29:36.654 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011665549000000001, 2024-09-17T07:29:36.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.01288154, 2024-09-17T07:29:36.654 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.654 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.654 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.654 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.654 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.654 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.654 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69115 2024-09-17T07:29:36.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.655 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555856+0000", 2024-09-17T07:29:36.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555856+0000", 2024-09-17T07:29:36.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555857+0000", 2024-09-17T07:29:36.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1300000000000001e-07 2024-09-17T07:29:36.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555859+0000", 2024-09-17T07:29:36.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.855e-06 2024-09-17T07:29:36.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555864+0000", 2024-09-17T07:29:36.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6779999999999997e-06 2024-09-17T07:29:36.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555869+0000", 2024-09-17T07:29:36.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8899999999999998e-06 2024-09-17T07:29:36.657 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.657 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.657 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.657 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:68860 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.658 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555198+0000", 2024-09-17T07:29:36.658 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012324293, 2024-09-17T07:29:36.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013609085999999999, 2024-09-17T07:29:36.658 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.658 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.658 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.659 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.659 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.659 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 68860 2024-09-17T07:29:36.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.659 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555198+0000", 2024-09-17T07:29:36.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555198+0000", 2024-09-17T07:29:36.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555198+0000", 2024-09-17T07:29:36.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5699999999999998e-07 2024-09-17T07:29:36.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555200+0000", 2024-09-17T07:29:36.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9700000000000002e-06 2024-09-17T07:29:36.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555206+0000", 2024-09-17T07:29:36.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5089999999999999e-06 2024-09-17T07:29:36.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555211+0000", 2024-09-17T07:29:36.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8849999999999998e-06 2024-09-17T07:29:36.662 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.662 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.662 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.662 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69116 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.662 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555875+0000", 2024-09-17T07:29:36.662 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011646345000000001, 2024-09-17T07:29:36.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.012984839, 2024-09-17T07:29:36.662 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.662 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.663 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.663 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.663 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69116 2024-09-17T07:29:36.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.663 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555875+0000", 2024-09-17T07:29:36.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555875+0000", 2024-09-17T07:29:36.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555876+0000", 2024-09-17T07:29:36.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:29:36.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555878+0000", 2024-09-17T07:29:36.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8640000000000001e-06 2024-09-17T07:29:36.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555884+0000", 2024-09-17T07:29:36.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9649999999999996e-06 2024-09-17T07:29:36.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555888+0000", 2024-09-17T07:29:36.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5190000000000003e-06 2024-09-17T07:29:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:68861 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555218+0000", 2024-09-17T07:29:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012303922, 2024-09-17T07:29:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013696893999999999, 2024-09-17T07:29:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.667 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 68861 2024-09-17T07:29:36.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.667 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555218+0000", 2024-09-17T07:29:36.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555218+0000", 2024-09-17T07:29:36.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555218+0000", 2024-09-17T07:29:36.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6899999999999998e-07 2024-09-17T07:29:36.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555220+0000", 2024-09-17T07:29:36.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9010000000000001e-06 2024-09-17T07:29:36.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555226+0000", 2024-09-17T07:29:36.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:29:36.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555231+0000", 2024-09-17T07:29:36.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:36.669 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.669 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.669 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.670 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69117 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.670 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555893+0000", 2024-09-17T07:29:36.670 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011628311000000001, 2024-09-17T07:29:36.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013068831, 2024-09-17T07:29:36.670 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.670 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.670 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.670 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.670 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.670 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69117 2024-09-17T07:29:36.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.670 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.671 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555893+0000", 2024-09-17T07:29:36.671 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.671 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555893+0000", 2024-09-17T07:29:36.671 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.671 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555894+0000", 2024-09-17T07:29:36.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.72e-07 2024-09-17T07:29:36.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555896+0000", 2024-09-17T07:29:36.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6759999999999999e-06 2024-09-17T07:29:36.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555901+0000", 2024-09-17T07:29:36.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6620000000000002e-06 2024-09-17T07:29:36.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555906+0000", 2024-09-17T07:29:36.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.561e-06 2024-09-17T07:29:36.673 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.673 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.673 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.673 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:68862 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.673 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555238+0000", 2024-09-17T07:29:36.673 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012284232000000001, 2024-09-17T07:29:36.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013774859, 2024-09-17T07:29:36.673 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.673 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.674 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.674 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.674 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.674 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.674 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 68862 2024-09-17T07:29:36.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.674 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555238+0000", 2024-09-17T07:29:36.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.675 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555238+0000", 2024-09-17T07:29:36.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.675 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.675 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555238+0000", 2024-09-17T07:29:36.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3099999999999998e-07 2024-09-17T07:29:36.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.675 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.676 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555240+0000", 2024-09-17T07:29:36.676 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9870000000000002e-06 2024-09-17T07:29:36.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.676 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.676 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555246+0000", 2024-09-17T07:29:36.676 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1129999999999997e-06 2024-09-17T07:29:36.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.677 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.677 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555254+0000", 2024-09-17T07:29:36.677 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9859999999999994e-06 2024-09-17T07:29:36.677 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.677 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.678 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.678 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69118 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.678 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555911+0000", 2024-09-17T07:29:36.678 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011610641999999999, 2024-09-17T07:29:36.678 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.01314967, 2024-09-17T07:29:36.678 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.679 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.679 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.679 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.679 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.679 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69118 2024-09-17T07:29:36.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.680 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555911+0000", 2024-09-17T07:29:36.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.681 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.681 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555911+0000", 2024-09-17T07:29:36.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.681 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.681 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555911+0000", 2024-09-17T07:29:36.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.46e-07 2024-09-17T07:29:36.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555913+0000", 2024-09-17T07:29:36.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7150000000000001e-06 2024-09-17T07:29:36.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555919+0000", 2024-09-17T07:29:36.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7239999999999997e-06 2024-09-17T07:29:36.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.684 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555924+0000", 2024-09-17T07:29:36.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9819999999999999e-06 2024-09-17T07:29:36.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.684 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.685 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.685 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:68863 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.685 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555262+0000", 2024-09-17T07:29:36.685 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012260284, 2024-09-17T07:29:36.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013846901999999999, 2024-09-17T07:29:36.686 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.686 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.686 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.686 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.687 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.687 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 68863 2024-09-17T07:29:36.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.687 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555262+0000", 2024-09-17T07:29:36.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555262+0000", 2024-09-17T07:29:36.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.689 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555262+0000", 2024-09-17T07:29:36.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0200000000000002e-07 2024-09-17T07:29:36.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.690 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.690 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555269+0000", 2024-09-17T07:29:36.690 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8009999999999998e-06 2024-09-17T07:29:36.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.690 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.690 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555275+0000", 2024-09-17T07:29:36.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:29:36.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.691 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.691 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555280+0000", 2024-09-17T07:29:36.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0080000000000002e-06 2024-09-17T07:29:36.691 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.692 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.692 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.692 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69119 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.692 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555929+0000", 2024-09-17T07:29:36.693 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011592392999999999, 2024-09-17T07:29:36.693 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013226053, 2024-09-17T07:29:36.693 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.693 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.693 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.694 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.694 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.694 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69119 2024-09-17T07:29:36.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.694 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555929+0000", 2024-09-17T07:29:36.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555929+0000", 2024-09-17T07:29:36.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555930+0000", 2024-09-17T07:29:36.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:29:36.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.697 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.697 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555931+0000", 2024-09-17T07:29:36.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6700000000000001e-06 2024-09-17T07:29:36.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.698 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.698 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555937+0000", 2024-09-17T07:29:36.698 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7030000000000003e-06 2024-09-17T07:29:36.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.698 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.698 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.698 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555949+0000", 2024-09-17T07:29:36.698 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1575e-05 2024-09-17T07:29:36.698 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.698 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.698 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.699 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69088 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.699 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555288+0000", 2024-09-17T07:29:36.699 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012234161, 2024-09-17T07:29:36.699 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013915871999999999, 2024-09-17T07:29:36.699 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.699 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.699 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.699 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.699 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.699 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.699 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69088 2024-09-17T07:29:36.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.699 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555288+0000", 2024-09-17T07:29:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555288+0000", 2024-09-17T07:29:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.700 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555288+0000", 2024-09-17T07:29:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:29:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555290+0000", 2024-09-17T07:29:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.886e-06 2024-09-17T07:29:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555296+0000", 2024-09-17T07:29:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1909999999999997e-06 2024-09-17T07:29:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555301+0000", 2024-09-17T07:29:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.622e-06 2024-09-17T07:29:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69344 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.567585+0000", 2024-09-17T07:29:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.9999371, 2024-09-17T07:29:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001668182, 2024-09-17T07:29:36.702 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69344 2024-09-17T07:29:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567585+0000", 2024-09-17T07:29:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567585+0000", 2024-09-17T07:29:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567586+0000", 2024-09-17T07:29:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5960000000000001e-06 2024-09-17T07:29:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567593+0000", 2024-09-17T07:29:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2479999999999999e-06 2024-09-17T07:29:36.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567604+0000", 2024-09-17T07:29:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0657e-05 2024-09-17T07:29:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567612+0000", 2024-09-17T07:29:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2330000000000006e-06 2024-09-17T07:29:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69089 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555307+0000", 2024-09-17T07:29:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012214366000000001, 2024-09-17T07:29:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.013992156, 2024-09-17T07:29:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.706 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69089 2024-09-17T07:29:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555307+0000", 2024-09-17T07:29:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555307+0000", 2024-09-17T07:29:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555308+0000", 2024-09-17T07:29:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:29:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.708 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.709 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555310+0000", 2024-09-17T07:29:36.709 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9889999999999999e-06 2024-09-17T07:29:36.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.709 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.709 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555316+0000", 2024-09-17T07:29:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6720000000000001e-06 2024-09-17T07:29:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555320+0000", 2024-09-17T07:29:36.710 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4700000000000004e-06 2024-09-17T07:29:36.711 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.711 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.711 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.711 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69345 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.567704+0000", 2024-09-17T07:29:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.9998174, 2024-09-17T07:29:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016450220000000001, 2024-09-17T07:29:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69345 2024-09-17T07:29:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.713 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567704+0000", 2024-09-17T07:29:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.714 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567704+0000", 2024-09-17T07:29:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567705+0000", 2024-09-17T07:29:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4600000000000004e-07 2024-09-17T07:29:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567709+0000", 2024-09-17T07:29:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0430000000000001e-06 2024-09-17T07:29:36.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567718+0000", 2024-09-17T07:29:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:29:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.717 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567725+0000", 2024-09-17T07:29:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4300000000000003e-06 2024-09-17T07:29:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69090 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555327+0000", 2024-09-17T07:29:36.718 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012194517, 2024-09-17T07:29:36.719 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.014070479, 2024-09-17T07:29:36.719 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69090 2024-09-17T07:29:36.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555327+0000", 2024-09-17T07:29:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555327+0000", 2024-09-17T07:29:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555328+0000", 2024-09-17T07:29:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999996e-07 2024-09-17T07:29:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555330+0000", 2024-09-17T07:29:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.136e-06 2024-09-17T07:29:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555336+0000", 2024-09-17T07:29:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1340000000000001e-06 2024-09-17T07:29:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555341+0000", 2024-09-17T07:29:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8110000000000002e-06 2024-09-17T07:29:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69346 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.567735+0000", 2024-09-17T07:29:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.9997864, 2024-09-17T07:29:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017243440000000001, 2024-09-17T07:29:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69346 2024-09-17T07:29:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567735+0000", 2024-09-17T07:29:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567735+0000", 2024-09-17T07:29:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567736+0000", 2024-09-17T07:29:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9500000000000003e-07 2024-09-17T07:29:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567738+0000", 2024-09-17T07:29:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4030000000000001e-06 2024-09-17T07:29:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567745+0000", 2024-09-17T07:29:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.562e-06 2024-09-17T07:29:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567750+0000", 2024-09-17T07:29:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9989999999999999e-06 2024-09-17T07:29:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69091 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555349+0000", 2024-09-17T07:29:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012173183000000001, 2024-09-17T07:29:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.014158469, 2024-09-17T07:29:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69091 2024-09-17T07:29:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555349+0000", 2024-09-17T07:29:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555349+0000", 2024-09-17T07:29:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555349+0000", 2024-09-17T07:29:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5699999999999998e-07 2024-09-17T07:29:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555351+0000", 2024-09-17T07:29:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1440000000000001e-06 2024-09-17T07:29:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555358+0000", 2024-09-17T07:29:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4570000000000002e-06 2024-09-17T07:29:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555362+0000", 2024-09-17T07:29:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:29:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69347 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.567759+0000", 2024-09-17T07:29:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.999763, 2024-09-17T07:29:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001796242, 2024-09-17T07:29:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69347 2024-09-17T07:29:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567759+0000", 2024-09-17T07:29:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567759+0000", 2024-09-17T07:29:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567759+0000", 2024-09-17T07:29:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:29:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567761+0000", 2024-09-17T07:29:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0779999999999998e-06 2024-09-17T07:29:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567768+0000", 2024-09-17T07:29:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4099999999999996e-06 2024-09-17T07:29:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567772+0000", 2024-09-17T07:29:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5929999999999999e-06 2024-09-17T07:29:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69092 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555369+0000", 2024-09-17T07:29:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012152962999999999, 2024-09-17T07:29:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.014233858, 2024-09-17T07:29:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69092 2024-09-17T07:29:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555369+0000", 2024-09-17T07:29:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555369+0000", 2024-09-17T07:29:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555369+0000", 2024-09-17T07:29:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0900000000000002e-07 2024-09-17T07:29:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555371+0000", 2024-09-17T07:29:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9269999999999999e-06 2024-09-17T07:29:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555378+0000", 2024-09-17T07:29:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4119999999999998e-06 2024-09-17T07:29:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555386+0000", 2024-09-17T07:29:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4859999999999997e-06 2024-09-17T07:29:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69348 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.567779+0000", 2024-09-17T07:29:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.9997425, 2024-09-17T07:29:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0018698569999999999, 2024-09-17T07:29:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69348 2024-09-17T07:29:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567779+0000", 2024-09-17T07:29:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567779+0000", 2024-09-17T07:29:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567780+0000", 2024-09-17T07:29:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5299999999999999e-07 2024-09-17T07:29:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567782+0000", 2024-09-17T07:29:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2189999999999998e-06 2024-09-17T07:29:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567788+0000", 2024-09-17T07:29:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1369999999999999e-06 2024-09-17T07:29:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.567796+0000", 2024-09-17T07:29:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5259999999999996e-06 2024-09-17T07:29:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69093 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555393+0000", 2024-09-17T07:29:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012129035999999999, 2024-09-17T07:29:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.014304427999999999, 2024-09-17T07:29:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69093 2024-09-17T07:29:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555393+0000", 2024-09-17T07:29:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555393+0000", 2024-09-17T07:29:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555393+0000", 2024-09-17T07:29:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3099999999999998e-07 2024-09-17T07:29:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555395+0000", 2024-09-17T07:29:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1210000000000001e-06 2024-09-17T07:29:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555402+0000", 2024-09-17T07:29:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1979999999999998e-06 2024-09-17T07:29:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555406+0000", 2024-09-17T07:29:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8409999999999999e-06 2024-09-17T07:29:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69349 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.568074+0000", 2024-09-17T07:29:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.9994478, 2024-09-17T07:29:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016759590000000001, 2024-09-17T07:29:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69349 2024-09-17T07:29:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568074+0000", 2024-09-17T07:29:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568074+0000", 2024-09-17T07:29:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568075+0000", 2024-09-17T07:29:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6199999999999997e-07 2024-09-17T07:29:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568079+0000", 2024-09-17T07:29:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0419999999999996e-06 2024-09-17T07:29:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568089+0000", 2024-09-17T07:29:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0064999999999999e-05 2024-09-17T07:29:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568096+0000", 2024-09-17T07:29:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.041e-06 2024-09-17T07:29:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69094 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555413+0000", 2024-09-17T07:29:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012108569, 2024-09-17T07:29:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.01438512, 2024-09-17T07:29:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69094 2024-09-17T07:29:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555413+0000", 2024-09-17T07:29:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555413+0000", 2024-09-17T07:29:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555414+0000", 2024-09-17T07:29:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4400000000000001e-07 2024-09-17T07:29:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555416+0000", 2024-09-17T07:29:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1610000000000001e-06 2024-09-17T07:29:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555422+0000", 2024-09-17T07:29:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5409999999999997e-06 2024-09-17T07:29:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555427+0000", 2024-09-17T07:29:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8960000000000002e-06 2024-09-17T07:29:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69350 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.568108+0000", 2024-09-17T07:29:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.999414, 2024-09-17T07:29:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017382980000000001, 2024-09-17T07:29:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69350 2024-09-17T07:29:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568108+0000", 2024-09-17T07:29:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568108+0000", 2024-09-17T07:29:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568108+0000", 2024-09-17T07:29:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4000000000000002e-07 2024-09-17T07:29:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568111+0000", 2024-09-17T07:29:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7999999999999999e-06 2024-09-17T07:29:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568118+0000", 2024-09-17T07:29:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9789999999999996e-06 2024-09-17T07:29:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568124+0000", 2024-09-17T07:29:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5679999999999999e-06 2024-09-17T07:29:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69095 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555434+0000", 2024-09-17T07:29:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012087578, 2024-09-17T07:29:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.014459148, 2024-09-17T07:29:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69095 2024-09-17T07:29:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555434+0000", 2024-09-17T07:29:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555434+0000", 2024-09-17T07:29:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555435+0000", 2024-09-17T07:29:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3799999999999998e-07 2024-09-17T07:29:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555437+0000", 2024-09-17T07:29:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.176e-06 2024-09-17T07:29:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555443+0000", 2024-09-17T07:29:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1580000000000002e-06 2024-09-17T07:29:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555447+0000", 2024-09-17T07:29:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4530000000000004e-06 2024-09-17T07:29:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69351 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.568132+0000", 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.9993901, 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001820124, 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69351 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568132+0000", 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568132+0000", 2024-09-17T07:29:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568132+0000", 2024-09-17T07:29:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0299999999999999e-07 2024-09-17T07:29:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568135+0000", 2024-09-17T07:29:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.452e-06 2024-09-17T07:29:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568141+0000", 2024-09-17T07:29:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5860000000000001e-06 2024-09-17T07:29:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568146+0000", 2024-09-17T07:29:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0690000000000001e-06 2024-09-17T07:29:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69096 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555462+0000", 2024-09-17T07:29:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012059907999999999, 2024-09-17T07:29:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.014547431, 2024-09-17T07:29:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69096 2024-09-17T07:29:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555462+0000", 2024-09-17T07:29:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555462+0000", 2024-09-17T07:29:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555462+0000", 2024-09-17T07:29:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9999999999999998e-07 2024-09-17T07:29:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555465+0000", 2024-09-17T07:29:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3640000000000002e-06 2024-09-17T07:29:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555472+0000", 2024-09-17T07:29:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1269999999999997e-06 2024-09-17T07:29:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555477+0000", 2024-09-17T07:29:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2220000000000001e-06 2024-09-17T07:29:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69352 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.568154+0000", 2024-09-17T07:29:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.9993677, 2024-09-17T07:29:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019162059999999999, 2024-09-17T07:29:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69352 2024-09-17T07:29:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568154+0000", 2024-09-17T07:29:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568154+0000", 2024-09-17T07:29:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568155+0000", 2024-09-17T07:29:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7e-07 2024-09-17T07:29:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568157+0000", 2024-09-17T07:29:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0849999999999999e-06 2024-09-17T07:29:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568163+0000", 2024-09-17T07:29:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6800000000000004e-06 2024-09-17T07:29:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568168+0000", 2024-09-17T07:29:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0699999999999997e-06 2024-09-17T07:29:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69097 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555484+0000", 2024-09-17T07:29:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012037617, 2024-09-17T07:29:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.014641171999999999, 2024-09-17T07:29:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69097 2024-09-17T07:29:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555484+0000", 2024-09-17T07:29:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555484+0000", 2024-09-17T07:29:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555485+0000", 2024-09-17T07:29:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555487+0000", 2024-09-17T07:29:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0820000000000001e-06 2024-09-17T07:29:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555493+0000", 2024-09-17T07:29:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9059999999999999e-06 2024-09-17T07:29:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555498+0000", 2024-09-17T07:29:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9910000000000002e-06 2024-09-17T07:29:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69353 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.568229+0000", 2024-09-17T07:29:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.9992933, 2024-09-17T07:29:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001945455, 2024-09-17T07:29:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69353 2024-09-17T07:29:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568229+0000", 2024-09-17T07:29:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568229+0000", 2024-09-17T07:29:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568229+0000", 2024-09-17T07:29:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.61e-07 2024-09-17T07:29:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568232+0000", 2024-09-17T07:29:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0429999999999999e-06 2024-09-17T07:29:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568241+0000", 2024-09-17T07:29:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.935e-06 2024-09-17T07:29:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568247+0000", 2024-09-17T07:29:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0719999999999996e-06 2024-09-17T07:29:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69098 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555504+0000", 2024-09-17T07:29:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.012017398, 2024-09-17T07:29:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.014758373, 2024-09-17T07:29:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69098 2024-09-17T07:29:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555504+0000", 2024-09-17T07:29:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555504+0000", 2024-09-17T07:29:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555505+0000", 2024-09-17T07:29:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3099999999999998e-07 2024-09-17T07:29:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555514+0000", 2024-09-17T07:29:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1700000000000003e-06 2024-09-17T07:29:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555520+0000", 2024-09-17T07:29:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1190000000000002e-06 2024-09-17T07:29:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555525+0000", 2024-09-17T07:29:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6800000000000001e-06 2024-09-17T07:29:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69354 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.568257+0000", 2024-09-17T07:29:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.9992652, 2024-09-17T07:29:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020873300000000001, 2024-09-17T07:29:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69354 2024-09-17T07:29:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568257+0000", 2024-09-17T07:29:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568257+0000", 2024-09-17T07:29:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568257+0000", 2024-09-17T07:29:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0200000000000002e-07 2024-09-17T07:29:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568260+0000", 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0240000000000002e-06 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568265+0000", 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3170000000000001e-06 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568270+0000", 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0390000000000004e-06 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69099 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555531+0000", 2024-09-17T07:29:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011991229000000001, 2024-09-17T07:29:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.014861279, 2024-09-17T07:29:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69099 2024-09-17T07:29:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555531+0000", 2024-09-17T07:29:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555531+0000", 2024-09-17T07:29:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555531+0000", 2024-09-17T07:29:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7e-07 2024-09-17T07:29:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555537+0000", 2024-09-17T07:29:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6529999999999999e-06 2024-09-17T07:29:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555543+0000", 2024-09-17T07:29:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1279999999999996e-06 2024-09-17T07:29:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555547+0000", 2024-09-17T07:29:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6029999999999998e-06 2024-09-17T07:29:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69355 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.568278+0000", 2024-09-17T07:29:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.9992437, 2024-09-17T07:29:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002163781, 2024-09-17T07:29:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69355 2024-09-17T07:29:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568278+0000", 2024-09-17T07:29:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568278+0000", 2024-09-17T07:29:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568279+0000", 2024-09-17T07:29:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7800000000000002e-07 2024-09-17T07:29:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568281+0000", 2024-09-17T07:29:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.249e-06 2024-09-17T07:29:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568288+0000", 2024-09-17T07:29:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7619999999999998e-06 2024-09-17T07:29:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568294+0000", 2024-09-17T07:29:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.981e-06 2024-09-17T07:29:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69100 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555553+0000", 2024-09-17T07:29:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011968540999999999, 2024-09-17T07:29:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.014936059999999999, 2024-09-17T07:29:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69100 2024-09-17T07:29:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555553+0000", 2024-09-17T07:29:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555553+0000", 2024-09-17T07:29:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555554+0000", 2024-09-17T07:29:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8000000000000001e-07 2024-09-17T07:29:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555555+0000", 2024-09-17T07:29:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.705e-06 2024-09-17T07:29:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555561+0000", 2024-09-17T07:29:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8479999999999997e-06 2024-09-17T07:29:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555566+0000", 2024-09-17T07:29:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4309999999999996e-06 2024-09-17T07:29:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69356 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.568301+0000", 2024-09-17T07:29:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.9992204, 2024-09-17T07:29:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002234167, 2024-09-17T07:29:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69356 2024-09-17T07:29:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568301+0000", 2024-09-17T07:29:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568301+0000", 2024-09-17T07:29:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568302+0000", 2024-09-17T07:29:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1699999999999999e-07 2024-09-17T07:29:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568322+0000", 2024-09-17T07:29:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.012e-05 2024-09-17T07:29:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568330+0000", 2024-09-17T07:29:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8220000000000007e-06 2024-09-17T07:29:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568336+0000", 2024-09-17T07:29:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7740000000000001e-06 2024-09-17T07:29:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69101 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555571+0000", 2024-09-17T07:29:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011950649000000001, 2024-09-17T07:29:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.015010897, 2024-09-17T07:29:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69101 2024-09-17T07:29:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555571+0000", 2024-09-17T07:29:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555571+0000", 2024-09-17T07:29:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555572+0000", 2024-09-17T07:29:36.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:29:36.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555573+0000", 2024-09-17T07:29:36.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.72e-06 2024-09-17T07:29:36.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555579+0000", 2024-09-17T07:29:36.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7339999999999996e-06 2024-09-17T07:29:36.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555584+0000", 2024-09-17T07:29:36.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8380000000000001e-06 2024-09-17T07:29:36.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.811 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.811 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69357 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.811 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.568344+0000", 2024-09-17T07:29:36.811 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.9991779, 2024-09-17T07:29:36.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022852889999999998, 2024-09-17T07:29:36.811 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.811 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.811 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69357 2024-09-17T07:29:36.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.812 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.812 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568344+0000", 2024-09-17T07:29:36.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568344+0000", 2024-09-17T07:29:36.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568344+0000", 2024-09-17T07:29:36.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0399999999999996e-07 2024-09-17T07:29:36.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568347+0000", 2024-09-17T07:29:36.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3379999999999999e-06 2024-09-17T07:29:36.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568354+0000", 2024-09-17T07:29:36.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3610000000000003e-06 2024-09-17T07:29:36.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568359+0000", 2024-09-17T07:29:36.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8849999999999998e-06 2024-09-17T07:29:36.814 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.814 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.814 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.814 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69102 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.815 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555590+0000", 2024-09-17T07:29:36.815 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011931341999999999, 2024-09-17T07:29:36.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.015172329999999999, 2024-09-17T07:29:36.815 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.815 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.815 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.815 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.815 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69102 2024-09-17T07:29:36.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.815 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555590+0000", 2024-09-17T07:29:36.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555590+0000", 2024-09-17T07:29:36.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555591+0000", 2024-09-17T07:29:36.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.65e-07 2024-09-17T07:29:36.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555593+0000", 2024-09-17T07:29:36.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0059999999999999e-06 2024-09-17T07:29:36.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555599+0000", 2024-09-17T07:29:36.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:29:36.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555603+0000", 2024-09-17T07:29:36.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.724e-06 2024-09-17T07:29:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69358 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.568367+0000", 2024-09-17T07:29:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.999155, 2024-09-17T07:29:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024521479999999999, 2024-09-17T07:29:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.819 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69358 2024-09-17T07:29:36.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.819 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568367+0000", 2024-09-17T07:29:36.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568367+0000", 2024-09-17T07:29:36.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568367+0000", 2024-09-17T07:29:36.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:29:36.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568370+0000", 2024-09-17T07:29:36.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4729999999999999e-06 2024-09-17T07:29:36.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568376+0000", 2024-09-17T07:29:36.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3450000000000002e-06 2024-09-17T07:29:36.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568382+0000", 2024-09-17T07:29:36.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8830000000000003e-06 2024-09-17T07:29:36.821 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.822 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.822 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.822 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69103 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.822 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555611+0000", 2024-09-17T07:29:36.822 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011911059999999999, 2024-09-17T07:29:36.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.015272911, 2024-09-17T07:29:36.822 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.822 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.822 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.823 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.823 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69103 2024-09-17T07:29:36.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.823 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555611+0000", 2024-09-17T07:29:36.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555611+0000", 2024-09-17T07:29:36.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555611+0000", 2024-09-17T07:29:36.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5299999999999999e-07 2024-09-17T07:29:36.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555613+0000", 2024-09-17T07:29:36.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0449999999999999e-06 2024-09-17T07:29:36.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555620+0000", 2024-09-17T07:29:36.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4459999999999998e-06 2024-09-17T07:29:36.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555624+0000", 2024-09-17T07:29:36.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7149999999999997e-06 2024-09-17T07:29:36.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.825 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.825 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69359 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.825 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.568389+0000", 2024-09-17T07:29:36.826 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.9991326, 2024-09-17T07:29:36.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025580220000000001, 2024-09-17T07:29:36.826 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.826 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.826 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.826 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.826 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.826 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69359 2024-09-17T07:29:36.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.826 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568389+0000", 2024-09-17T07:29:36.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568389+0000", 2024-09-17T07:29:36.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568389+0000", 2024-09-17T07:29:36.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1300000000000001e-07 2024-09-17T07:29:36.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568391+0000", 2024-09-17T07:29:36.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8789999999999999e-06 2024-09-17T07:29:36.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568398+0000", 2024-09-17T07:29:36.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4500000000000001e-06 2024-09-17T07:29:36.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568403+0000", 2024-09-17T07:29:36.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4129999999999997e-06 2024-09-17T07:29:36.829 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.829 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.829 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.829 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69104 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.829 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.555631+0000", 2024-09-17T07:29:36.829 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011891158000000001, 2024-09-17T07:29:36.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.015367909000000001, 2024-09-17T07:29:36.829 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.830 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.830 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.830 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.830 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69104 2024-09-17T07:29:36.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.830 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555631+0000", 2024-09-17T07:29:36.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555631+0000", 2024-09-17T07:29:36.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555631+0000", 2024-09-17T07:29:36.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:29:36.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555633+0000", 2024-09-17T07:29:36.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.88e-06 2024-09-17T07:29:36.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555639+0000", 2024-09-17T07:29:36.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3149999999999997e-06 2024-09-17T07:29:36.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.555644+0000", 2024-09-17T07:29:36.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0710000000000002e-06 2024-09-17T07:29:36.832 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.832 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.832 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.833 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:69360 3.17 3.35fec0b7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e71)", 2024-09-17T07:29:36.833 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:36.568475+0000", 2024-09-17T07:29:36.833 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4294967295.9990468, 2024-09-17T07:29:36.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0026258829999999999, 2024-09-17T07:29:36.833 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:36.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:36.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:36.833 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:36.833 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:36.833 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:36.833 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 69360 2024-09-17T07:29:36.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.834 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:36.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:36.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568475+0000", 2024-09-17T07:29:36.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:36.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568475+0000", 2024-09-17T07:29:36.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:36.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:36.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568476+0000", 2024-09-17T07:29:36.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8599999999999998e-07 2024-09-17T07:29:36.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:36.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568479+0000", 2024-09-17T07:29:36.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0730000000000001e-06 2024-09-17T07:29:36.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:36.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568488+0000", 2024-09-17T07:29:36.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2159999999999995e-06 2024-09-17T07:29:36.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:36.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:36.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:36.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:36.568494+0000", 2024-09-17T07:29:36.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.066e-06 2024-09-17T07:29:36.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.836 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:36.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:36.836 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:29:36.836 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 64 2024-09-17T07:29:36.836 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:36.838 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:36.839 INFO:teuthology.orchestra.run.smithi045.stdout:3056 write 0x4200bc9 thru 0x420633c (0x5774 bytes) 2024-09-17T07:29:36.839 INFO:teuthology.orchestra.run.smithi045.stdout:3057 trunc from 0x85ba343 to 0xc880c0e 2024-09-17T07:29:36.839 INFO:teuthology.orchestra.run.smithi045.stdout:3058 write 0xaf0827e thru 0xaf15eff (0xdc82 bytes) 2024-09-17T07:29:36.839 INFO:teuthology.orchestra.run.smithi045.stdout:3059 writesame 0xd31ecda thru 0xd3232c5 data_size 0x32(0x45ec bytes) 2024-09-17T07:29:36.839 INFO:teuthology.orchestra.run.smithi045.stdout:3060 writesame 0x50c29b1 thru 0x50ce9ae data_size 0x19(0xbffe bytes) 2024-09-17T07:29:36.839 INFO:teuthology.orchestra.run.smithi045.stdout:3061 write 0x212cb95 thru 0x213a7fe (0xdc6a bytes) 2024-09-17T07:29:36.839 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:36.837+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:36.864 INFO:teuthology.orchestra.run.smithi045.stdout:3063 punch from 0x2873eed to 0x287b384, (0x7497 bytes) 2024-09-17T07:29:36.864 INFO:teuthology.orchestra.run.smithi045.stdout:3064 read 0x9a73b8d thru 0x9a7a66d (0x6ae1 bytes) 2024-09-17T07:29:36.864 INFO:teuthology.orchestra.run.smithi045.stdout:3066 writesame 0x8ed1d93 thru 0x8edd57a data_size 0x58(0xb7e8 bytes) 2024-09-17T07:29:36.910 INFO:teuthology.orchestra.run.smithi045.stdout:3067 punch from 0x19f19e5 to 0x19f814f, (0x676a bytes) 2024-09-17T07:29:36.910 INFO:teuthology.orchestra.run.smithi045.stdout:3068 read 0x8120e7d thru 0x812b7de (0xa962 bytes) 2024-09-17T07:29:36.910 INFO:teuthology.orchestra.run.smithi045.stdout:3069 write 0x6de27cf thru 0x6df148a (0xecbc bytes) 2024-09-17T07:29:36.939 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:36.938+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:37.013 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:37.014 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:37.014 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:37.014 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:37.014 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:37.024 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:37.026 INFO:teuthology.orchestra.run.smithi045.stdout:3073 trunc from 0xd3232c6 to 0x681afb7 2024-09-17T07:29:37.039 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:37.039+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:37.040 INFO:teuthology.orchestra.run.smithi045.stdout:3074 read 0x377d6 thru 0x44428 (0xcc53 bytes) 2024-09-17T07:29:37.043 INFO:teuthology.orchestra.run.smithi045.stdout:3075 writesame 0x3831dd9 thru 0x3840c98 data_size 0x20(0xeec0 bytes) 2024-09-17T07:29:37.141 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:37.140+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:37.141 INFO:teuthology.orchestra.run.smithi045.stdout:3078 write 0xe9010d1 thru 0xe909148 (0x8078 bytes) 2024-09-17T07:29:37.147 INFO:teuthology.orchestra.run.smithi045.stdout:3079 read 0x81a48ba thru 0x81a95fa (0x4d41 bytes) 2024-09-17T07:29:37.147 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.147 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.152 INFO:teuthology.orchestra.run.smithi045.stdout:3083 write 0xeb865b7 thru 0xeb9526e (0xecb8 bytes) 2024-09-17T07:29:37.157 INFO:teuthology.orchestra.run.smithi045.stdout:3085 write 0xbfcfb45 thru 0xbfd8ab3 (0x8f6f bytes) 2024-09-17T07:29:37.166 INFO:teuthology.orchestra.run.smithi045.stdout:3086 read 0xa72feb5 thru 0xa73292f (0x2a7b bytes) 2024-09-17T07:29:37.166 INFO:teuthology.orchestra.run.smithi045.stdout:3087 read 0xc9f87a7 thru 0xca01b28 (0x9382 bytes) 2024-09-17T07:29:37.166 INFO:teuthology.orchestra.run.smithi045.stdout:3088 read 0x207823 thru 0x216d8a (0xf568 bytes) 2024-09-17T07:29:37.168 INFO:teuthology.orchestra.run.smithi045.stdout:3091 punch from 0x22f66d9 to 0x22fae6e, (0x4795 bytes) 2024-09-17T07:29:37.168 INFO:teuthology.orchestra.run.smithi045.stdout:3094 writesame 0x5c4e7f0 thru 0x5c5bc2e data_size 0x1(0xd43f bytes) 2024-09-17T07:29:37.180 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:37.180 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:37.180 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:37.180 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:37.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.181 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:37.181 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:37.181 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 39.046657551000003, 2024-09-17T07:29:37.181 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:29:37.181 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.181 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.181 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:37.181 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.181 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.181 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.181 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:29:37.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.182 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.182 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.182 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:37.182 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.182 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.182 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:37.182 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.182 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.182 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:29:37.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:29:37.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.183 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:29:37.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:29:37.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.183 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:29:37.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:29:37.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.184 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:29:37.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:29:37.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.184 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.184 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:29:37.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:29:37.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.184 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:29:37.184 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:29:37.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:29:37.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.185 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.185 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:29:37.185 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:29:37.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.185 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:37.185 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:29:37.185 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:29:37.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.185 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.185 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:29:37.186 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:29:37.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.186 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:37.186 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:29:37.186 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:29:37.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.186 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.186 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:29:37.186 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:29:37.186 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.186 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.186 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.187 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.187 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:37.187 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.740690847, 2024-09-17T07:29:37.187 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:29:37.187 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.187 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.187 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:37.187 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.187 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.187 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:29:37.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.188 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:37.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:37.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:29:37.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:29:37.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.189 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.189 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:29:37.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:29:37.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.189 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.189 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:29:37.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:29:37.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.190 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.190 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:29:37.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:29:37.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.190 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.190 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:29:37.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:29:37.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.190 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:37.190 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:29:37.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:29:37.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.191 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.191 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:29:37.191 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:29:37.191 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.191 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.191 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.191 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.191 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:37.191 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.740668732, 2024-09-17T07:29:37.191 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:29:37.191 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.191 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.192 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.192 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.192 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.192 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.192 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:29:37.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.192 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.192 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.192 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:37.192 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.192 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.193 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:37.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.193 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.193 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:29:37.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:29:37.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.193 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.193 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:29:37.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:29:37.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.194 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.194 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:29:37.194 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:29:37.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.194 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.194 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:29:37.194 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:29:37.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.194 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.194 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:29:37.194 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:29:37.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.195 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.195 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:29:37.195 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:29:37.195 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.195 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.195 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.195 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.195 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:37.195 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.740644627999998, 2024-09-17T07:29:37.195 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:29:37.195 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.196 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.196 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.196 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.196 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.196 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.196 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:29:37.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.196 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.196 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:37.196 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.197 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.197 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:37.197 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.197 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.197 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:29:37.197 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:29:37.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.197 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.197 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:29:37.197 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:29:37.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.198 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.198 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:29:37.198 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:29:37.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.198 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.198 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:29:37.198 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:29:37.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.198 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.198 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:29:37.198 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:29:37.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.199 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.199 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:29:37.199 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:29:37.199 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.199 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.199 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.199 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.199 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:37.199 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.740620538999998, 2024-09-17T07:29:37.199 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:29:37.199 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.200 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.200 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.200 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.200 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.200 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.200 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:29:37.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.200 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.200 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:37.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:37.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:37.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:29:37.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:29:37.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:29:37.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.202 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.202 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:29:37.202 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:29:37.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.202 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.202 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:29:37.202 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:29:37.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.202 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.202 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:29:37.202 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:29:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:29:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:29:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.740596119999999, 2024-09-17T07:29:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:29:37.203 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:29:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:29:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:29:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:29:37.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:29:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:29:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:29:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:29:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:29:37.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:29:37.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.207 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.207 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:29:37.207 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:29:37.207 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.207 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.207 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.207 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.740574420000002, 2024-09-17T07:29:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:29:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:29:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.208 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:29:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.210 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.210 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:29:37.210 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:29:37.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.210 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.210 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:29:37.210 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:29:37.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.210 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.210 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:29:37.210 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:29:37.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.211 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.211 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:29:37.211 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:29:37.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.211 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.211 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:29:37.211 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:29:37.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.211 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.212 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.212 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:37.212 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.740529687999999, 2024-09-17T07:29:37.212 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:29:37.212 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.212 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.212 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.212 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.212 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.212 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:29:37.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.212 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.212 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.213 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.213 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:37.213 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.213 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.213 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:37.213 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.213 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.213 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:29:37.213 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:29:37.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.214 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.214 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:29:37.214 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:29:37.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.214 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.214 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:29:37.214 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:29:37.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.214 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:29:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:29:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:29:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:29:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:29:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:29:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.216 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.216 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.216 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:37.216 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.740501221999999, 2024-09-17T07:29:37.216 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:29:37.216 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.216 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.216 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.216 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.216 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.216 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.216 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:29:37.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.217 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.217 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.217 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:37.217 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.217 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.217 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:37.217 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.217 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.218 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:37.218 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:29:37.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.218 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.218 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:29:37.218 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:29:37.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.218 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.218 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:29:37.218 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:29:37.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.218 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.219 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:29:37.219 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:29:37.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.219 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.219 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:29:37.219 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:29:37.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.219 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.219 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:29:37.219 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:29:37.219 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.219 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.219 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.220 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.220 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:37.220 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.74038397, 2024-09-17T07:29:37.220 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:29:37.220 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.220 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.220 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.220 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.220 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.220 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.220 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:29:37.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.221 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.221 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.221 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:37.221 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.221 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.221 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:37.221 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.221 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.221 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:29:37.222 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:29:37.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.222 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.222 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:29:37.222 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:29:37.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.222 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.222 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:29:37.222 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:29:37.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.223 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.223 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:29:37.223 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:29:37.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.223 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.223 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:29:37.223 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:29:37.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.223 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.223 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:29:37.223 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:29:37.223 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.224 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.224 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.224 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.224 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:37.224 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.739957868000001, 2024-09-17T07:29:37.224 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:29:37.224 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.224 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.224 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.224 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.224 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.225 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.225 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:29:37.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.225 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.225 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.225 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:37.225 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.225 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.225 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:37.225 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.226 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.226 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:29:37.226 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:37.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.226 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.226 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:29:37.226 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:29:37.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.226 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.226 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:29:37.226 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:29:37.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.227 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.227 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:29:37.227 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:29:37.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.227 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.227 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:29:37.227 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:29:37.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.227 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.228 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:29:37.228 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:29:37.228 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.228 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.228 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.228 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.228 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:37.228 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.739938197000001, 2024-09-17T07:29:37.228 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:29:37.228 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.228 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.229 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.229 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.229 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.229 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.229 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:29:37.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.229 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.229 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.229 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:37.229 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.229 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.230 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:37.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.230 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.230 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:37.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:29:37.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.230 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.230 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:29:37.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:29:37.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:29:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:29:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:29:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:29:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:29:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:29:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.232 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.232 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:29:37.232 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:29:37.232 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.232 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.232 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.232 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.232 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:37.232 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.739915332999999, 2024-09-17T07:29:37.232 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:29:37.232 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.232 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.233 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.233 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.233 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.233 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.233 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:29:37.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.233 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.233 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.233 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:37.233 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.233 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:37.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:37.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:29:37.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:29:37.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:29:37.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:29:37.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:29:37.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.235 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:29:37.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:29:37.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.235 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:29:37.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:29:37.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.236 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:29:37.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:29:37.236 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.236 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.236 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.236 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.236 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:37.236 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.739894031999999, 2024-09-17T07:29:37.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:29:37.236 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.236 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.237 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.237 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.237 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.237 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.237 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:29:37.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.237 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.237 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:37.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:37.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:29:37.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:29:37.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:29:37.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:29:37.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:29:37.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:29:37.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:29:37.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:29:37.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:29:37.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:29:37.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.240 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.240 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:29:37.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:29:37.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.240 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.240 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.240 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:37.240 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.739846243999999, 2024-09-17T07:29:37.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:29:37.240 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.241 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.241 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.241 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.241 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.241 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.241 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:29:37.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.241 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:37.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:37.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:29:37.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:29:37.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:29:37.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:29:37.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:29:37.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:29:37.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:29:37.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:29:37.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:29:37.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:29:37.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.244 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:29:37.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:29:37.244 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.244 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.244 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.244 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.244 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:37.244 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.739819021999999, 2024-09-17T07:29:37.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:29:37.245 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.245 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.245 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.245 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.245 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.245 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.245 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:29:37.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.245 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.245 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.245 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:37.245 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:37.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:37.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:29:37.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:29:37.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:29:37.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:29:37.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:29:37.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:29:37.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:29:37.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:29:37.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:29:37.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.248 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.248 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:29:37.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:29:37.248 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.248 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.248 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.248 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.248 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:37.248 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.739798635, 2024-09-17T07:29:37.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:29:37.249 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.249 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.249 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.249 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.249 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.249 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:29:37.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.249 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.249 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.249 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:37.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:37.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:29:37.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:29:37.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:29:37.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:29:37.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:29:37.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:29:37.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:29:37.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:29:37.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:29:37.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:29:37.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:29:37.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:29:37.252 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.252 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.252 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.252 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.252 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:37.253 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.73977893, 2024-09-17T07:29:37.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:29:37.253 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.253 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.253 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.253 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.253 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.253 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.253 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:29:37.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.253 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:37.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:37.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:29:37.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:29:37.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:29:37.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:29:37.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:29:37.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:29:37.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:29:37.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:29:37.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:29:37.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:29:37.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:29:37.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:29:37.256 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.256 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.256 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.256 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.257 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:37.257 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.739755852000002, 2024-09-17T07:29:37.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:29:37.257 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.257 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.257 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.257 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.257 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.257 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.257 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:29:37.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.257 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.258 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:37.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.258 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.258 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:37.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.258 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.258 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:29:37.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:29:37.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:29:37.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:29:37.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:29:37.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:29:37.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:29:37.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:29:37.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:29:37.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:29:37.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:29:37.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:29:37.260 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.260 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.260 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.261 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:37.261 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:37.261 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 28.739735289999999, 2024-09-17T07:29:37.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:29:37.261 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:37.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:37.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:37.261 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:37.261 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:37.261 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:37.261 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:29:37.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.261 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:37.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:37.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:37.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:37.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:37.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:37.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:37.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:37.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:29:37.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:37.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:29:37.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:29:37.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:37.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:29:37.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:29:37.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:37.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:29:37.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:29:37.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:37.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:29:37.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:29:37.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:37.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:37.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:37.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:29:37.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:29:37.264 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.264 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.264 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.264 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:37.264 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:37.265 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:37.266 INFO:teuthology.orchestra.run.smithi045.stdout:3097 write 0xc8b08cf thru 0xc8b1fb8 (0x16ea bytes) 2024-09-17T07:29:37.266 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:37.265+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:37.267 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:3098 write 0xb7e9987 thru 0xb7f0ce9 (0x7363 bytes) 2024-09-17T07:29:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:3100 write 0x155a3aa thru 0x1568ed0 (0xeb27 bytes) 2024-09-17T07:29:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:3102 writesame 0x664089b thru 0x66453e9 data_size 0x1(0x4b4f bytes) 2024-09-17T07:29:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:3103 read 0xddf29e6 thru 0xddf552d (0x2b48 bytes) 2024-09-17T07:29:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:3104 punch from 0x85bc31d to 0x85c4053, (0x7d36 bytes) 2024-09-17T07:29:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:3106 trunc from 0xeb9526f to 0x871af05 2024-09-17T07:29:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:3107 trunc from 0x871af05 to 0xca8c904 2024-09-17T07:29:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.269 INFO:teuthology.orchestra.run.smithi045.stdout:3109 trunc from 0xca8c904 to 0xaf73072 2024-09-17T07:29:37.270 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.270 INFO:teuthology.orchestra.run.smithi045.stdout:3111 trunc from 0xaf73072 to 0xbcc90ee 2024-09-17T07:29:37.270 INFO:teuthology.orchestra.run.smithi045.stdout:3113 write 0x39dfaf7 thru 0x39e79b5 (0x7ebf bytes) 2024-09-17T07:29:37.270 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.270 INFO:teuthology.orchestra.run.smithi045.stdout:3115 punch from 0xcd7d81 to 0xce00c1, (0x8340 bytes) 2024-09-17T07:29:37.270 INFO:teuthology.orchestra.run.smithi045.stdout:3116 writesame 0xe057eda thru 0xe06650e data_size 0x1(0xe635 bytes) 2024-09-17T07:29:37.280 INFO:teuthology.orchestra.run.smithi045.stdout:3117 trunc from 0xe06650f to 0x25669b0 2024-09-17T07:29:37.309 INFO:teuthology.orchestra.run.smithi045.stdout:3118 punch from 0x126faf2 to 0x1274d9e, (0x52ac bytes) 2024-09-17T07:29:37.309 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-17T07:29:37.333 INFO:teuthology.orchestra.run.smithi045.stdout:3120 writesame 0xe657aa5 thru 0xe661bc9 data_size 0x1(0xa125 bytes) 2024-09-17T07:29:37.333 INFO:teuthology.orchestra.run.smithi045.stdout:3121 read 0x4789909 thru 0x479460c (0xad04 bytes) 2024-09-17T07:29:37.333 INFO:teuthology.orchestra.run.smithi045.stdout:3123 write 0x444fb64 thru 0x445ac9b (0xb138 bytes) 2024-09-17T07:29:37.343 INFO:teuthology.orchestra.run.smithi045.stdout:3124 writesame 0x1568b64 thru 0x156f64b data_size 0x2c(0x6ae8 bytes) 2024-09-17T07:29:37.345 INFO:teuthology.orchestra.run.smithi045.stdout:3126 write 0x7471e52 thru 0x7474da8 (0x2f57 bytes) 2024-09-17T07:29:37.350 INFO:teuthology.orchestra.run.smithi045.stdout:3128 read 0x13de706 thru 0x13df259 (0xb54 bytes) 2024-09-17T07:29:37.350 INFO:teuthology.orchestra.run.smithi045.stdout:3130 write 0x248db8 thru 0x251662 (0x88ab bytes) 2024-09-17T07:29:37.366 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:37.366+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:37.421 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:37.421 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:37.421 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:37.421 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:37.431 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:37.436 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.436 INFO:teuthology.orchestra.run.smithi045.stdout:3133 write 0x52192a9 thru 0x521cc9d (0x39f5 bytes) 2024-09-17T07:29:37.441 INFO:teuthology.orchestra.run.smithi045.stdout:3134 writesame 0x8f77bd2 thru 0x8f855a1 data_size 0xaa(0xd9d0 bytes) 2024-09-17T07:29:37.453 INFO:teuthology.orchestra.run.smithi045.stdout:3135 read 0xd108dae thru 0xd10fcc2 (0x6f15 bytes) 2024-09-17T07:29:37.453 INFO:teuthology.orchestra.run.smithi045.stdout:3139 write 0xc448027 thru 0xc44f655 (0x762f bytes) 2024-09-17T07:29:37.461 INFO:teuthology.orchestra.run.smithi045.stdout:3140 write 0x8da2d25 thru 0x8da532e (0x260a bytes) 2024-09-17T07:29:37.462 INFO:teuthology.orchestra.run.smithi045.stdout:3141 write 0x7508c67 thru 0x7511b5f (0x8ef9 bytes) 2024-09-17T07:29:37.467 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:37.467+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:37.469 INFO:teuthology.orchestra.run.smithi045.stdout:3142 flatten 2024-09-17T07:29:37.504 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:29:37.531 INFO:teuthology.orchestra.run.smithi045.stdout:3143 writesame 0x422c8f thru 0x430647 data_size 0x21(0xd9b9 bytes) 2024-09-17T07:29:37.534 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.534 INFO:teuthology.orchestra.run.smithi045.stdout:3145 write 0x46f5d2c thru 0x46fa4e4 (0x47b9 bytes) 2024-09-17T07:29:37.536 INFO:teuthology.orchestra.run.smithi045.stdout:3146 punch from 0x73e1358 to 0x73ef3c0, (0xe068 bytes) 2024-09-17T07:29:37.536 INFO:teuthology.orchestra.run.smithi045.stdout:3148 read 0xa57045c thru 0xa57befc (0xbaa1 bytes) 2024-09-17T07:29:37.536 INFO:teuthology.orchestra.run.smithi045.stdout:3149 punch from 0x23149a9 to 0x231fbe5, (0xb23c bytes) 2024-09-17T07:29:37.536 INFO:teuthology.orchestra.run.smithi045.stdout:3151 write 0x74942ad thru 0x74a344d (0xf1a1 bytes) 2024-09-17T07:29:37.540 INFO:teuthology.orchestra.run.smithi045.stdout:3152 read 0xbe3e931 thru 0xbe4d824 (0xeef4 bytes) 2024-09-17T07:29:37.540 INFO:teuthology.orchestra.run.smithi045.stdout:3153 read 0xd1eba0d thru 0xd1fa1aa (0xe79e bytes) 2024-09-17T07:29:37.540 INFO:teuthology.orchestra.run.smithi045.stdout:3154 writesame 0x5d6a9ea thru 0x5d6cd1d data_size 0x6(0x2334 bytes) 2024-09-17T07:29:37.542 INFO:teuthology.orchestra.run.smithi045.stdout:3155 punch from 0xd74011 to 0xd803a4, (0xc393 bytes) 2024-09-17T07:29:37.542 INFO:teuthology.orchestra.run.smithi045.stdout:3158 punch from 0x7ce0565 to 0x7ce5dfa, (0x5895 bytes) 2024-09-17T07:29:37.542 INFO:teuthology.orchestra.run.smithi045.stdout:3159 write 0xa683ca5 thru 0xa68a0fa (0x6456 bytes) 2024-09-17T07:29:37.549 INFO:teuthology.orchestra.run.smithi045.stdout:3161 read 0xce02b85 thru 0xce0e0e8 (0xb564 bytes) 2024-09-17T07:29:37.549 INFO:teuthology.orchestra.run.smithi045.stdout:3162 read 0xa7ec55e thru 0xa7fc104 (0xfba7 bytes) 2024-09-17T07:29:37.550 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.550 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.550 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.550 INFO:teuthology.orchestra.run.smithi045.stdout:3169 writesame 0xaf2b94e thru 0xaf32510 data_size 0x7(0x6bc3 bytes) 2024-09-17T07:29:37.556 INFO:teuthology.orchestra.run.smithi045.stdout:3171 write 0x811da9d thru 0x8123cd4 (0x6238 bytes) 2024-09-17T07:29:37.559 INFO:teuthology.orchestra.run.smithi045.stdout:3172 read 0x1b219fb thru 0x1b295ff (0x7c05 bytes) 2024-09-17T07:29:37.560 INFO:teuthology.orchestra.run.smithi045.stdout:3173 read 0x1862014 thru 0x186d59c (0xb589 bytes) 2024-09-17T07:29:37.561 INFO:teuthology.orchestra.run.smithi045.stdout:3175 punch from 0x4063d04 to 0x40693ad, (0x56a9 bytes) 2024-09-17T07:29:37.561 INFO:teuthology.orchestra.run.smithi045.stdout:3176 writesame 0x7a73ccd thru 0x7a7bca8 data_size 0x62(0x7fdc bytes) 2024-09-17T07:29:37.563 INFO:teuthology.orchestra.run.smithi045.stdout:3178 write 0x8d745e0 thru 0x8d82101 (0xdb22 bytes) 2024-09-17T07:29:37.568 INFO:teuthology.orchestra.run.smithi045.stdout:3179 write 0x4f6af64 thru 0x4f6f222 (0x42bf bytes) 2024-09-17T07:29:37.568 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:37.568+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:37.570 INFO:teuthology.orchestra.run.smithi045.stdout:3180 read 0xa0cbcb1 thru 0xa0cf57b (0x38cb bytes) 2024-09-17T07:29:37.571 INFO:teuthology.orchestra.run.smithi045.stdout:3181 punch from 0x419b357 to 0x41a0c8f, (0x5938 bytes) 2024-09-17T07:29:37.571 INFO:teuthology.orchestra.run.smithi045.stdout:3182 punch from 0x5f7e3a to 0x5fff8c, (0x8152 bytes) 2024-09-17T07:29:37.571 INFO:teuthology.orchestra.run.smithi045.stdout:3184 read 0x2999243 thru 0x299c533 (0x32f1 bytes) 2024-09-17T07:29:37.572 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.572 INFO:teuthology.orchestra.run.smithi045.stdout:3186 write 0xbfb3c3a thru 0xbfbcf46 (0x930d bytes) 2024-09-17T07:29:37.572 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:37.573 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:37.573 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:37.573 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:37.573 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:37.580 INFO:teuthology.orchestra.run.smithi045.stdout:3187 writesame 0x64e7f6b thru 0x64f6bfb data_size 0x9(0xec91 bytes) 2024-09-17T07:29:37.585 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:37.591 INFO:teuthology.orchestra.run.smithi045.stdout:3188 write 0x7d2e299 thru 0x7d31e7d (0x3be5 bytes) 2024-09-17T07:29:37.593 INFO:teuthology.orchestra.run.smithi045.stdout:3190 read 0x25f1003 thru 0x25fac91 (0x9c8f bytes) 2024-09-17T07:29:37.608 INFO:teuthology.orchestra.run.smithi045.stdout:3191 trunc from 0xe661bca to 0x34a0a74 2024-09-17T07:29:37.638 INFO:teuthology.orchestra.run.smithi045.stdout:3192 write 0x1135d66 thru 0x1139fc7 (0x4262 bytes) 2024-09-17T07:29:37.641 INFO:teuthology.orchestra.run.smithi045.stdout:3193 punch from 0x257d604 to 0x2583152, (0x5b4e bytes) 2024-09-17T07:29:37.659 INFO:teuthology.orchestra.run.smithi045.stdout:3194 write 0xdb7b813 thru 0xdb80f31 (0x571f bytes) 2024-09-17T07:29:37.665 INFO:teuthology.orchestra.run.smithi045.stdout:3195 read 0xcafeca thru 0xcbe1ff (0xe336 bytes) 2024-09-17T07:29:37.669 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:37.669+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:37.673 INFO:teuthology.orchestra.run.smithi045.stdout:3196 write 0xed86704 thru 0xed8bd86 (0x5683 bytes) 2024-09-17T07:29:37.677 INFO:teuthology.orchestra.run.smithi045.stdout:3200 write 0x9137357 thru 0x914154f (0xa1f9 bytes) 2024-09-17T07:29:37.684 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:37.684 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:37.685 INFO:teuthology.orchestra.run.smithi045.stdout:3201 read 0x2c270cc thru 0x2c2c7f9 (0x572e bytes) 2024-09-17T07:29:37.685 INFO:teuthology.orchestra.run.smithi045.stdout:3203 write 0x21aabf thru 0x228cf5 (0xe237 bytes) 2024-09-17T07:29:37.689 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.690 INFO:teuthology.orchestra.run.smithi045.stdout:3206 writesame 0xb3f4406 thru 0xb4013db data_size 0x2(0xcfd6 bytes) 2024-09-17T07:29:37.699 INFO:teuthology.orchestra.run.smithi045.stdout:3207 write 0x88dace5 thru 0x88e83b4 (0xd6d0 bytes) 2024-09-17T07:29:37.709 INFO:teuthology.orchestra.run.smithi045.stdout:3208 write 0xebb12b7 thru 0xebb971d (0x8467 bytes) 2024-09-17T07:29:37.714 INFO:teuthology.orchestra.run.smithi045.stdout:3209 read 0x2710dad thru 0x27157ca (0x4a1e bytes) 2024-09-17T07:29:37.715 INFO:teuthology.orchestra.run.smithi045.stdout:3210 writesame 0x74b91f0 thru 0x74c4f1b data_size 0x2(0xbd2c bytes) 2024-09-17T07:29:37.720 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:37.720 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:37.726 INFO:teuthology.orchestra.run.smithi045.stdout:3211 read 0x3ba924f thru 0x3bad69f (0x4451 bytes) 2024-09-17T07:29:37.730 INFO:teuthology.orchestra.run.smithi045.stdout:3212 trunc from 0xed8bd87 to 0xb627d6c 2024-09-17T07:29:37.748 INFO:teuthology.orchestra.run.smithi045.stdout:3213 trunc from 0xb627d6c to 0x6bfda9b 2024-09-17T07:29:37.755 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:37.755 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:37.755 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:37.758 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:37.758 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:37.763 INFO:teuthology.orchestra.run.smithi045.stdout:3214 write 0x38bdd3 thru 0x399af5 (0xdd23 bytes) 2024-09-17T07:29:37.766 INFO:teuthology.orchestra.run.smithi045.stdout:3217 punch from 0x55f8ae8 to 0x55fc1e1, (0x36f9 bytes) 2024-09-17T07:29:37.769 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:37.774 INFO:teuthology.orchestra.run.smithi045.stdout:3218 write 0x8b91200 thru 0x8b94411 (0x3212 bytes) 2024-09-17T07:29:37.774 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:37.773+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:37.776 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:37.776 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:37.789 INFO:teuthology.orchestra.run.smithi045.stdout:3220 write 0xd2b9dd4 thru 0xd2ba6d3 (0x900 bytes) 2024-09-17T07:29:37.794 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.794 INFO:teuthology.orchestra.run.smithi045.stdout:3222 write 0x553a7fd thru 0x5543193 (0x8997 bytes) 2024-09-17T07:29:37.795 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:37.795 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:37.801 INFO:teuthology.orchestra.run.smithi045.stdout:3223 punch from 0x73f7df6 to 0x73fe424, (0x662e bytes) 2024-09-17T07:29:37.801 INFO:teuthology.orchestra.run.smithi045.stdout:3224 punch from 0x3befe79 to 0x3bfe3b0, (0xe537 bytes) 2024-09-17T07:29:37.801 INFO:teuthology.orchestra.run.smithi045.stdout:3226 punch from 0xc22dbc8 to 0xc23b688, (0xdac0 bytes) 2024-09-17T07:29:37.801 INFO:teuthology.orchestra.run.smithi045.stdout:3227 write 0x35cbd93 thru 0x35d565c (0x98ca bytes) 2024-09-17T07:29:37.809 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.809 INFO:teuthology.orchestra.run.smithi045.stdout:3229 punch from 0xad69940 to 0xad73f50, (0xa610 bytes) 2024-09-17T07:29:37.809 INFO:teuthology.orchestra.run.smithi045.stdout:3230 writesame 0x4450865 thru 0x445896e data_size 0x53(0x810a bytes) 2024-09-17T07:29:37.816 INFO:teuthology.orchestra.run.smithi045.stdout:3231 punch from 0xb93a37a to 0xb93f06d, (0x4cf3 bytes) 2024-09-17T07:29:37.816 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.816 INFO:teuthology.orchestra.run.smithi045.stdout:3236 write 0x16b7872 thru 0x16c5daf (0xe53e bytes) 2024-09-17T07:29:37.819 INFO:teuthology.orchestra.run.smithi045.stdout:3238 writesame 0x5f782d1 thru 0x5f867f8 data_size 0x4(0xe528 bytes) 2024-09-17T07:29:37.820 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:37.820 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:37.827 INFO:teuthology.orchestra.run.smithi045.stdout:3239 read 0x444c967 thru 0x444f9ff (0x3099 bytes) 2024-09-17T07:29:37.828 INFO:teuthology.orchestra.run.smithi045.stdout:3240 write 0x2916e82 thru 0x291dd67 (0x6ee6 bytes) 2024-09-17T07:29:37.830 INFO:teuthology.orchestra.run.smithi045.stdout:3242 write 0x4a6293 thru 0x4b3737 (0xd4a5 bytes) 2024-09-17T07:29:37.833 INFO:teuthology.orchestra.run.smithi045.stdout:3243 read 0x8ef1aea thru 0x8ef6661 (0x4b78 bytes) 2024-09-17T07:29:37.841 INFO:teuthology.orchestra.run.smithi045.stdout:3244 write 0xeaf9984 thru 0xeafefa3 (0x5620 bytes) 2024-09-17T07:29:37.842 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:37.842 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:37.846 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:37.847 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:37.847 INFO:teuthology.orchestra.run.smithi045.stdout:3245 read 0x79c27d9 thru 0x79c7c3d (0x5465 bytes) 2024-09-17T07:29:37.847 INFO:teuthology.orchestra.run.smithi045.stdout:3246 writesame 0x15b4582 thru 0x15be6e9 data_size 0x14(0xa168 bytes) 2024-09-17T07:29:37.850 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.850 INFO:teuthology.orchestra.run.smithi045.stdout:3249 write 0x7ad38bd thru 0x7ada837 (0x6f7b bytes) 2024-09-17T07:29:37.856 INFO:teuthology.orchestra.run.smithi045.stdout:3251 read 0xcb174d6 thru 0xcb1759c (0xc7 bytes) 2024-09-17T07:29:37.861 INFO:teuthology.orchestra.run.smithi045.stdout:3252 trunc from 0xeafefa4 to 0x9ed2253 2024-09-17T07:29:37.870 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:37.870+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:37.872 INFO:teuthology.orchestra.run.smithi045.stdout:3253 write 0x37a63a8 thru 0x37aa1f9 (0x3e52 bytes) 2024-09-17T07:29:37.886 INFO:teuthology.orchestra.run.smithi045.stdout:3254 writesame 0xe133c4d thru 0xe13eaaf data_size 0x17(0xae63 bytes) 2024-09-17T07:29:37.886 INFO:teuthology.orchestra.run.smithi045.stdout:3255 write 0x745a71a thru 0x7469e49 (0xf730 bytes) 2024-09-17T07:29:37.891 INFO:teuthology.orchestra.run.smithi045.stdout:3256 write 0x6bc13cc thru 0x6bc7145 (0x5d7a bytes) 2024-09-17T07:29:37.901 INFO:teuthology.orchestra.run.smithi045.stdout:3258 trunc from 0xe13eab0 to 0xa6c0891 2024-09-17T07:29:37.906 INFO:teuthology.orchestra.run.smithi045.stdout:3259 read 0xd432c7 thru 0xd49932 (0x666c bytes) 2024-09-17T07:29:37.907 INFO:teuthology.orchestra.run.smithi045.stdout:3261 write 0x9cd407c thru 0x9cdf73e (0xb6c3 bytes) 2024-09-17T07:29:37.907 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:37.908 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:37.917 INFO:teuthology.orchestra.run.smithi045.stdout:3263 read 0x8f9264c thru 0x8f9d821 (0xb1d6 bytes) 2024-09-17T07:29:37.919 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:37.925 INFO:teuthology.orchestra.run.smithi045.stdout:3264 write 0xc627c4d thru 0xc62cdab (0x515f bytes) 2024-09-17T07:29:37.941 INFO:teuthology.orchestra.run.smithi045.stdout:3265 write 0xde6a83e thru 0xde739d4 (0x9197 bytes) 2024-09-17T07:29:37.948 INFO:teuthology.orchestra.run.smithi045.stdout:3271 write 0xa6720fa thru 0xa67d059 (0xaf60 bytes) 2024-09-17T07:29:37.958 INFO:teuthology.orchestra.run.smithi045.stdout:3273 read 0xaab8550 thru 0xaac02d9 (0x7d8a bytes) 2024-09-17T07:29:37.958 INFO:teuthology.orchestra.run.smithi045.stdout:3274 read 0x9c7d085 thru 0x9c80876 (0x37f2 bytes) 2024-09-17T07:29:37.959 INFO:teuthology.orchestra.run.smithi045.stdout:3276 write 0x5f276f3 thru 0x5f2bcae (0x45bc bytes) 2024-09-17T07:29:37.961 INFO:teuthology.orchestra.run.smithi045.stdout:3277 punch from 0xafccddb to 0xafdcad6, (0xfcfb bytes) 2024-09-17T07:29:37.961 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.961 INFO:teuthology.orchestra.run.smithi045.stdout:3279 write 0xd98ed38 thru 0xd996af4 (0x7dbd bytes) 2024-09-17T07:29:37.964 INFO:teuthology.orchestra.run.smithi045.stdout:3280 write 0x43cd3aa thru 0x43dbe02 (0xea59 bytes) 2024-09-17T07:29:37.971 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.971 INFO:teuthology.orchestra.run.smithi045.stdout:3284 read 0xce47e9c thru 0xce4af35 (0x309a bytes) 2024-09-17T07:29:37.972 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:37.971+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:37.972 INFO:teuthology.orchestra.run.smithi045.stdout:3285 writesame 0xd579d99 thru 0xd587064 data_size 0x12(0xd2cc bytes) 2024-09-17T07:29:37.975 INFO:teuthology.orchestra.run.smithi045.stdout:3286 writesame 0x4ebad97 thru 0x4ebd721 data_size 0x1(0x298b bytes) 2024-09-17T07:29:37.979 INFO:teuthology.orchestra.run.smithi045.stdout:3288 write 0xb53c1f1 thru 0xb5487bf (0xc5cf bytes) 2024-09-17T07:29:37.989 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.989 INFO:teuthology.orchestra.run.smithi045.stdout:3291 write 0xc337aea thru 0xc33f4fc (0x7a13 bytes) 2024-09-17T07:29:37.991 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.991 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:37.991 INFO:teuthology.orchestra.run.smithi045.stdout:3294 punch from 0x1cff1d3 to 0x1d0c3f6, (0xd223 bytes) 2024-09-17T07:29:37.992 INFO:teuthology.orchestra.run.smithi045.stdout:3296 write 0xdc0ea8f thru 0xdc12e96 (0x4408 bytes) 2024-09-17T07:29:37.993 INFO:teuthology.orchestra.run.smithi045.stdout:3297 writesame 0xc13ab32 thru 0xc13c2d4 data_size 0x1(0x17a3 bytes) 2024-09-17T07:29:37.995 INFO:teuthology.orchestra.run.smithi045.stdout:3298 write 0xa93f692 thru 0xa94dc9e (0xe60d bytes) 2024-09-17T07:29:38.000 INFO:teuthology.orchestra.run.smithi045.stdout:3299 write 0x1e339e9 thru 0x1e39f79 (0x6591 bytes) 2024-09-17T07:29:38.016 INFO:teuthology.orchestra.run.smithi045.stdout:3300 trunc from 0xde739d5 to 0x87f101 2024-09-17T07:29:38.052 INFO:teuthology.orchestra.run.smithi045.stdout:3301 read 0x6c9ce2 thru 0x6d351a (0x9839 bytes) 2024-09-17T07:29:38.052 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:38.052 INFO:teuthology.orchestra.run.smithi045.stdout:3304 read 0x4738d3 thru 0x47bb5c (0x828a bytes) 2024-09-17T07:29:38.056 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:38.056 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:38.061 INFO:teuthology.orchestra.run.smithi045.stdout:3307 write 0x26d0a3a thru 0x26d668d (0x5c54 bytes) 2024-09-17T07:29:38.066 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:38.070 INFO:teuthology.orchestra.run.smithi045.stdout:3310 trunc from 0x26d668e to 0x417df2d 2024-09-17T07:29:38.072 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:38.071+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:38.075 INFO:teuthology.orchestra.run.smithi045.stdout:3311 write 0x19868a8 thru 0x199391a (0xd073 bytes) 2024-09-17T07:29:38.094 INFO:teuthology.orchestra.run.smithi045.stdout:3312 write 0x4fc18a3 thru 0x4fce5fc (0xcd5a bytes) 2024-09-17T07:29:38.112 INFO:teuthology.orchestra.run.smithi045.stdout:3313 write 0x6e815fa thru 0x6e827b7 (0x11be bytes) 2024-09-17T07:29:38.116 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:38.116 INFO:teuthology.orchestra.run.smithi045.stdout:3317 writesame 0x31ec257 thru 0x31f1908 data_size 0x51(0x56b2 bytes) 2024-09-17T07:29:38.121 INFO:teuthology.orchestra.run.smithi045.stdout:3318 read 0x1ab68e2 thru 0x1abe35a (0x7a79 bytes) 2024-09-17T07:29:38.259 INFO:teuthology.orchestra.run.smithi045.stdout:3319 trunc from 0x6e827b8 to 0xa56ff57 2024-09-17T07:29:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:3321 write 0xdaed62 thru 0xdaf9b8 (0xc57 bytes) 2024-09-17T07:29:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:3322 read 0x5acd2d2 thru 0x5ad9fba (0xcce9 bytes) 2024-09-17T07:29:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:3325 write 0x160ee09 thru 0x1611642 (0x283a bytes) 2024-09-17T07:29:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:3326 read 0x37ed912 thru 0x37f0291 (0x2980 bytes) 2024-09-17T07:29:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:3327 writesame 0x15cb3e7 thru 0x15cd2f0 data_size 0x1d(0x1f0a bytes) 2024-09-17T07:29:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:3328 write 0xd36a287 thru 0xd376c78 (0xc9f2 bytes) 2024-09-17T07:29:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:38.260 INFO:teuthology.orchestra.run.smithi045.stdout:3331 trunc from 0xd376c79 to 0x22b0352 2024-09-17T07:29:38.261 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:38.261 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:38.261 INFO:teuthology.orchestra.run.smithi045.stdout:3333 write 0xd904a5b thru 0xd9073b6 (0x295c bytes) 2024-09-17T07:29:38.262 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:38.263 INFO:teuthology.orchestra.run.smithi045.stdout:3337 trunc from 0xd9073b7 to 0x4669af5 2024-09-17T07:29:38.263 INFO:teuthology.orchestra.run.smithi045.stdout:3338 read 0x26b6616 thru 0x26c5a11 (0xf3fc bytes) 2024-09-17T07:29:38.263 INFO:teuthology.orchestra.run.smithi045.stdout:3339 writesame 0xd54ab60 thru 0xd5578f7 data_size 0x18(0xcd98 bytes) 2024-09-17T07:29:38.263 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:38.263 INFO:teuthology.orchestra.run.smithi045.stdout:3341 punch from 0x9a41c66 to 0x9a47d31, (0x60cb bytes) 2024-09-17T07:29:38.263 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:38.263 INFO:teuthology.orchestra.run.smithi045.stdout:3345 writesame 0xa1daa4f thru 0xa1e700f data_size 0xf(0xc5c1 bytes) 2024-09-17T07:29:38.264 INFO:teuthology.orchestra.run.smithi045.stdout:3346 read 0x63c2285 thru 0x63ceda4 (0xcb20 bytes) 2024-09-17T07:29:38.264 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:38.259+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:38.264 INFO:teuthology.orchestra.run.smithi045.stdout:3347 trunc from 0xd5578f8 to 0x856c922 2024-09-17T07:29:38.278 INFO:teuthology.orchestra.run.smithi045.stdout:3348 trunc from 0x856c922 to 0xd5af5ec 2024-09-17T07:29:38.285 INFO:teuthology.orchestra.run.smithi045.stdout:3350 writesame 0x16a0169 thru 0x16ae8a4 data_size 0x6(0xe73c bytes) 2024-09-17T07:29:38.290 INFO:teuthology.orchestra.run.smithi045.stdout:3352 write 0x337c128 thru 0x3387dd3 (0xbcac bytes) 2024-09-17T07:29:38.298 INFO:teuthology.orchestra.run.smithi045.stdout:3353 read 0xa60ce9c thru 0xa61558a (0x86ef bytes) 2024-09-17T07:29:38.298 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:38.299 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:38.299 INFO:teuthology.orchestra.run.smithi045.stdout:3359 write 0x6dacd26 thru 0x6db718e (0xa469 bytes) 2024-09-17T07:29:38.306 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:38.306 INFO:teuthology.orchestra.run.smithi045.stdout:3361 write 0x36d07e5 thru 0x36dd9e2 (0xd1fe bytes) 2024-09-17T07:29:38.310 INFO:teuthology.orchestra.run.smithi045.stdout:3363 write 0xb1d92d3 thru 0xb1e1077 (0x7da5 bytes) 2024-09-17T07:29:38.317 INFO:teuthology.orchestra.run.smithi045.stdout:3365 read 0xb2c6d6c thru 0xb2c85bc (0x1851 bytes) 2024-09-17T07:29:38.318 INFO:teuthology.orchestra.run.smithi045.stdout:3366 read 0x4ba4d92 thru 0x4bb2154 (0xd3c3 bytes) 2024-09-17T07:29:38.318 INFO:teuthology.orchestra.run.smithi045.stdout:3368 punch from 0x9c23b4f to 0x9c2bd0b, (0x81bc bytes) 2024-09-17T07:29:38.318 INFO:teuthology.orchestra.run.smithi045.stdout:3369 read 0xc39ad69 thru 0xc39e7dd (0x3a75 bytes) 2024-09-17T07:29:38.318 INFO:teuthology.orchestra.run.smithi045.stdout:3371 read 0x445725b thru 0x4466a31 (0xf7d7 bytes) 2024-09-17T07:29:38.318 INFO:teuthology.orchestra.run.smithi045.stdout:3372 write 0x6389957 thru 0x6396303 (0xc9ad bytes) 2024-09-17T07:29:38.335 INFO:teuthology.orchestra.run.smithi045.stdout:3373 trunc from 0xd5af5ec to 0x1334214 2024-09-17T07:29:38.354 INFO:teuthology.orchestra.run.smithi045.stdout:3374 read 0xb3c993 thru 0xb4978e (0xcdfc bytes) 2024-09-17T07:29:38.354 INFO:teuthology.orchestra.run.smithi045.stdout:3375 read 0x19834e thru 0x19a7fd (0x24b0 bytes) 2024-09-17T07:29:38.355 INFO:teuthology.orchestra.run.smithi045.stdout:3376 write 0xa097c thru 0xa66ea (0x5d6f bytes) 2024-09-17T07:29:38.360 INFO:teuthology.orchestra.run.smithi045.stdout:3377 trunc from 0x1334214 to 0x3f0f4e9 2024-09-17T07:29:38.360 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:38.360+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:38.378 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:38.378 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:38.383 INFO:teuthology.orchestra.run.smithi045.stdout:3378 write 0xead9df2 thru 0xeae89b6 (0xebc5 bytes) 2024-09-17T07:29:38.387 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:38.393 INFO:teuthology.orchestra.run.smithi045.stdout:3380 read 0xcf12f99 thru 0xcf14828 (0x1890 bytes) 2024-09-17T07:29:38.393 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:38.406 INFO:teuthology.orchestra.run.smithi045.stdout:3383 trunc from 0xeae89b7 to 0x2f250fe 2024-09-17T07:29:38.417 INFO:teuthology.orchestra.run.smithi045.stdout:3384 punch from 0x26ff5c8 to 0x2705252, (0x5c8a bytes) 2024-09-17T07:29:38.417 INFO:teuthology.orchestra.run.smithi045.stdout:3385 read 0x13a6a34 thru 0x13ae697 (0x7c64 bytes) 2024-09-17T07:29:38.418 INFO:teuthology.orchestra.run.smithi045.stdout:3386 punch from 0x25a3729 to 0x25ab303, (0x7bda bytes) 2024-09-17T07:29:38.418 INFO:teuthology.orchestra.run.smithi045.stdout:3387 read 0x2f140de thru 0x2f218af (0xd7d2 bytes) 2024-09-17T07:29:38.418 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:38.418 INFO:teuthology.orchestra.run.smithi045.stdout:3389 read 0x2dd7d8 thru 0x2ec456 (0xec7f bytes) 2024-09-17T07:29:38.418 INFO:teuthology.orchestra.run.smithi045.stdout:3391 punch from 0x13161c4 to 0x1319c41, (0x3a7d bytes) 2024-09-17T07:29:38.419 INFO:teuthology.orchestra.run.smithi045.stdout:3393 read 0x1e179b4 thru 0x1e1eb6b (0x71b8 bytes) 2024-09-17T07:29:38.419 INFO:teuthology.orchestra.run.smithi045.stdout:3395 read 0x9562ff thru 0x95d672 (0x7374 bytes) 2024-09-17T07:29:38.429 INFO:teuthology.orchestra.run.smithi045.stdout:3397 writesame 0x4e852cd thru 0x4e8d72f data_size 0x21(0x8463 bytes) 2024-09-17T07:29:38.429 INFO:teuthology.orchestra.run.smithi045.stdout:3398 read 0x405ace3 thru 0x405cb7b (0x1e99 bytes) 2024-09-17T07:29:38.429 INFO:teuthology.orchestra.run.smithi045.stdout:3399 writesame 0xbbe94a thru 0xbc8ae5 data_size 0x2(0xa19c bytes) 2024-09-17T07:29:38.432 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:38.432 INFO:teuthology.orchestra.run.smithi045.stdout:3401 read 0x4bf0b65 thru 0x4bf0beb (0x87 bytes) 2024-09-17T07:29:38.433 INFO:teuthology.orchestra.run.smithi045.stdout:3403 read 0x2b30334 thru 0x2b335bf (0x328c bytes) 2024-09-17T07:29:38.439 INFO:teuthology.orchestra.run.smithi045.stdout:3404 trunc from 0x4e8d730 to 0xab6c6d5 2024-09-17T07:29:38.455 INFO:teuthology.orchestra.run.smithi045.stdout:3405 trunc from 0xab6c6d5 to 0x2072732 2024-09-17T07:29:38.460 INFO:teuthology.orchestra.run.smithi045.stdout:3406 read 0x1263a1d thru 0x126d348 (0x992c bytes) 2024-09-17T07:29:38.461 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:38.460+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:38.461 INFO:teuthology.orchestra.run.smithi045.stdout:3407 writesame 0x1a26387 thru 0x1a350bf data_size 0x1f(0xed39 bytes) 2024-09-17T07:29:38.475 INFO:teuthology.orchestra.run.smithi045.stdout:3409 writesame 0x47dda81 thru 0x47e4415 data_size 0x1(0x6995 bytes) 2024-09-17T07:29:38.475 INFO:teuthology.orchestra.run.smithi045.stdout:3410 write 0x858991 thru 0x863a77 (0xb0e7 bytes) 2024-09-17T07:29:38.477 INFO:teuthology.orchestra.run.smithi045.stdout:3411 read 0x1acf5ea thru 0x1add4de (0xdef5 bytes) 2024-09-17T07:29:38.477 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:38.480 INFO:teuthology.orchestra.run.smithi045.stdout:3413 trunc from 0x47e4416 to 0x1c9467f 2024-09-17T07:29:38.487 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-17T07:29:38.487 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:29:38.487 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T07:29:38.487 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-17T07:29:38.490 INFO:teuthology.orchestra.run.smithi045.stdout:3414 punch from 0x18efcea to 0x18f9a2a, (0x9d40 bytes) 2024-09-17T07:29:38.490 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:38.499 INFO:teuthology.orchestra.run.smithi045.stdout:3416 write 0x4e65940 thru 0x4e691a3 (0x3864 bytes) 2024-09-17T07:29:38.499 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:38.499 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:38.504 INFO:teuthology.orchestra.run.smithi045.stdout:3417 punch from 0x23a7ce9 to 0x23afaca, (0x7de1 bytes) 2024-09-17T07:29:38.504 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:38.504 INFO:teuthology.orchestra.run.smithi045.stdout:3419 read 0xef1be5 thru 0xefb8b0 (0x9ccc bytes) 2024-09-17T07:29:38.504 INFO:teuthology.orchestra.run.smithi045.stdout:3420 read 0x567df8 thru 0x573ab0 (0xbcb9 bytes) 2024-09-17T07:29:38.506 INFO:teuthology.orchestra.run.smithi045.stdout:3424 write 0x1f984bc thru 0x1f9fcd8 (0x781d bytes) 2024-09-17T07:29:38.508 INFO:teuthology.orchestra.run.smithi045.stdout:3425 read 0x767806 thru 0x76d8af (0x60aa bytes) 2024-09-17T07:29:38.508 INFO:teuthology.orchestra.run.smithi045.stdout:3426 read 0x311c389 thru 0x311f58b (0x3203 bytes) 2024-09-17T07:29:38.510 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:38.525 INFO:teuthology.orchestra.run.smithi045.stdout:3427 write 0xe37bb65 thru 0xe3839c0 (0x7e5c bytes) 2024-09-17T07:29:38.532 INFO:teuthology.orchestra.run.smithi045.stdout:3428 write 0xda5abca thru 0xda6a30e (0xf745 bytes) 2024-09-17T07:29:38.546 INFO:teuthology.orchestra.run.smithi045.stdout:3429 write 0xe5f7aca thru 0xe605519 (0xda50 bytes) 2024-09-17T07:29:38.552 INFO:teuthology.orchestra.run.smithi045.stdout:3430 punch from 0xd0b7935 to 0xd0bc19d, (0x4868 bytes) 2024-09-17T07:29:38.552 INFO:teuthology.orchestra.run.smithi045.stdout:3431 clone 32 order 18 su 8192 sc 5 2024-09-17T07:29:38.561 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:38.561+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:38.652 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:38.652 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:38.660 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:38.662 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:38.661+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:38.763 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:38.763+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:38.765 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:38.765 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:38.773 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:38.808 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-17T07:29:38.808+0000 7f6d53957640 -1 mon.a@0(leader).osd e72 definitely_dead 0 2024-09-17T07:29:38.864 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:38.864+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:38.917 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:38.917 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:38.925 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:38.965 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:38.965+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:39.067 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:39.066+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:39.068 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:39.068 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:39.078 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:39.167 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:39.167+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:39.180 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:39.180 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:39.189 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:39.268 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:39.267+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:39.293 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:39.293 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:39.302 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:39.319 INFO:teuthology.orchestra.run.smithi045.stderr:marked out osd.3. 2024-09-17T07:29:39.331 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T07:29:39.377 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:39.377+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:39.444 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:39.444 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:39.453 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:39.496 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:39.496+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:39.556 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:39.556 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:39.564 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:39.570 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:39.569+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:39.670 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:39.670+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:39.709 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:39.709 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:39.717 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:39.771 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:39.771+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:39.848 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:29:39.860 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:39.860 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:39.868 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:39.872 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:39.871+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:39.973 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:39.972+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:39.976 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:39.976 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:39.993 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:39.993 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:40.002 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:40.002 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:40.002 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:40.002 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:40.009 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:40.009 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:40.009 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:40.011 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:40.026 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:40.027 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:40.043 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:40.044 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:40.060 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:40.060 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:40.074 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:40.073+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:40.077 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:40.077 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:40.080 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:40.081 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:40.119 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:40.120 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:40.120 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:40.120 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:40.120 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:40.128 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:40.174 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:40.174+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:40.275 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:40.274+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:40.276 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:40.276 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:40.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:40.276 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:40.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.276 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.276 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:40.276 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.142540337, 2024-09-17T07:29:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:29:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:29:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:40.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:29:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:29:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:29:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:29:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:29:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:29:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:29:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:29:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:29:40.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:29:40.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:29:40.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:29:40.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:29:40.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:29:40.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:29:40.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:29:40.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:29:40.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:29:40.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:29:40.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:29:40.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:29:40.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:29:40.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:29:40.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.283 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.283 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.283 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.283 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:40.283 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.836573633, 2024-09-17T07:29:40.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:29:40.283 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.283 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.283 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:29:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:29:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:29:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:29:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:29:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:29:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:29:40.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:29:40.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:29:40.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:29:40.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:29:40.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:29:40.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:29:40.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:29:40.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:29:40.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.287 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.287 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.287 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.287 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:40.287 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.836551518, 2024-09-17T07:29:40.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:29:40.288 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.288 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.288 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.288 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.288 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.288 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.288 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:29:40.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.288 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:40.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:40.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:29:40.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:29:40.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:29:40.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:29:40.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:29:40.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:29:40.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:29:40.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:29:40.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:29:40.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:29:40.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:29:40.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:29:40.291 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.291 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.291 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.291 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.291 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:40.291 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.836527413999999, 2024-09-17T07:29:40.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:29:40.292 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.292 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.292 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.292 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.292 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.292 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:29:40.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.292 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:40.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:40.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:29:40.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:29:40.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:29:40.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:29:40.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:29:40.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:29:40.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:29:40.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:29:40.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:29:40.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:29:40.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:29:40.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:29:40.295 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.295 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.295 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.295 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.295 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:40.295 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.836503324999999, 2024-09-17T07:29:40.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:29:40.296 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.296 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.296 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.296 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.296 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.296 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:29:40.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.296 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:40.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:40.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:40.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:29:40.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:29:40.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:29:40.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:29:40.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:29:40.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:29:40.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:29:40.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:29:40.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:29:40.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:29:40.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:29:40.299 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.299 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.299 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.299 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.299 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:40.300 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.836478906, 2024-09-17T07:29:40.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:29:40.300 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.300 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.300 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.300 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.300 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.300 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.300 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:29:40.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.300 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:40.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:40.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:40.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:29:40.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:29:40.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:29:40.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:29:40.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:29:40.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:29:40.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:29:40.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:29:40.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:29:40.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:29:40.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:29:40.303 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.303 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.303 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.303 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.304 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:40.304 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.836457205999999, 2024-09-17T07:29:40.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:29:40.304 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.304 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.304 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.304 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.304 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:29:40.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.304 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:40.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:40.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:40.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:29:40.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:29:40.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:29:40.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:29:40.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:29:40.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:29:40.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:29:40.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:29:40.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:29:40.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:29:40.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:29:40.307 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.308 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.308 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.308 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.308 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:40.308 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.836412473999999, 2024-09-17T07:29:40.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:29:40.308 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.308 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.308 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.309 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.309 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.309 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:29:40.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.309 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:40.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:40.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:29:40.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:29:40.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:29:40.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:29:40.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:29:40.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:29:40.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:29:40.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:29:40.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:29:40.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:29:40.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:29:40.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:29:40.312 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.312 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.312 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.312 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.312 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:40.312 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.836384008, 2024-09-17T07:29:40.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:29:40.312 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.313 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.313 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.313 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.313 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.313 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:29:40.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.313 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:40.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:40.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:40.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:29:40.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:29:40.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:29:40.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:29:40.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:29:40.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:29:40.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:29:40.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:29:40.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:29:40.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:29:40.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:29:40.316 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.316 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.316 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.317 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.317 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:40.317 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.836266756000001, 2024-09-17T07:29:40.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:29:40.317 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.317 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.317 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.317 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.317 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.317 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:29:40.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.317 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:40.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:40.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:29:40.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:29:40.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:29:40.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:29:40.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:29:40.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:29:40.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:29:40.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:29:40.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:29:40.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:29:40.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:29:40.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:29:40.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.320 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.321 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.321 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:40.321 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.835840653999998, 2024-09-17T07:29:40.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:29:40.321 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.321 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.321 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.321 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.321 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.321 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:29:40.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.321 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:40.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:40.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:29:40.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:40.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:29:40.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:29:40.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:29:40.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:29:40.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:29:40.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:29:40.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:29:40.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:29:40.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:29:40.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:29:40.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.324 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.325 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.325 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:40.325 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.835820983000001, 2024-09-17T07:29:40.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:29:40.325 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.325 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.325 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.325 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.325 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.325 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.325 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:29:40.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.326 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:40.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:40.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:40.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:29:40.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:29:40.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:29:40.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:29:40.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:29:40.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:29:40.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:29:40.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:29:40.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:29:40.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:29:40.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:29:40.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.329 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.329 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.329 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.329 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:40.329 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.835798119, 2024-09-17T07:29:40.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:29:40.329 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.329 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.329 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.329 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.329 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.330 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:29:40.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.330 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:40.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:40.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:40.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:29:40.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:29:40.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:29:40.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:29:40.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:29:40.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:29:40.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:29:40.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:29:40.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:29:40.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:29:40.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:29:40.332 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.333 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.333 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.333 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.333 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:40.333 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.835776817999999, 2024-09-17T07:29:40.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:29:40.333 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.333 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.333 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.334 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.334 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:29:40.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.334 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:40.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:40.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:29:40.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:29:40.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:29:40.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:29:40.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:29:40.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:29:40.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:29:40.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:29:40.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:29:40.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:29:40.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:29:40.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:29:40.337 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.337 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.337 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.337 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.337 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:40.337 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.83572903, 2024-09-17T07:29:40.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:29:40.337 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.338 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.338 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.338 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:29:40.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.338 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:40.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:40.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:29:40.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:29:40.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:29:40.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:29:40.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:29:40.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:29:40.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:29:40.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:29:40.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:29:40.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:29:40.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:29:40.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:29:40.341 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.341 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.341 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.341 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.341 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:40.341 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.835701808, 2024-09-17T07:29:40.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:29:40.342 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.342 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.342 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.342 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.342 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.342 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.342 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:29:40.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.342 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:40.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:40.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:40.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:29:40.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:29:40.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:29:40.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:29:40.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:29:40.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:29:40.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:29:40.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.345 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:29:40.345 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:29:40.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.345 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.345 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:29:40.345 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:29:40.345 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.345 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.345 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.345 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.345 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:40.345 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.835681421, 2024-09-17T07:29:40.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:29:40.346 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.346 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.346 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.346 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.346 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.346 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:29:40.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.346 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:29:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:29:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.347 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:29:40.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:29:40.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:29:40.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:29:40.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:29:40.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:29:40.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:29:40.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:29:40.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:29:40.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:29:40.349 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.349 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.349 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.349 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.350 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:40.350 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.835661716000001, 2024-09-17T07:29:40.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:29:40.350 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.350 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.350 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.350 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.350 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.350 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:29:40.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.352 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:40.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:40.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.353 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.353 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:29:40.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:29:40.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.353 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.353 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:29:40.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:29:40.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.353 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.353 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:29:40.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:29:40.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:29:40.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:29:40.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:29:40.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:29:40.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:29:40.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:29:40.355 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.355 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.355 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.355 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.355 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:40.355 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.835638637999999, 2024-09-17T07:29:40.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:29:40.355 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.355 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.355 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.356 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.356 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.356 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.356 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:29:40.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.356 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.356 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.356 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:40.356 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.356 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.357 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:40.357 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.357 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.357 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:29:40.357 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:29:40.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.357 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.357 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:29:40.357 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:29:40.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.358 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.358 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:29:40.358 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:29:40.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.358 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.358 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:29:40.358 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:29:40.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.358 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.358 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:29:40.358 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:29:40.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.359 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.359 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:29:40.359 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:29:40.359 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.359 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.359 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.359 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:40.359 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:40.359 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.835618075999999, 2024-09-17T07:29:40.359 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:29:40.359 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.360 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.360 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:40.360 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.360 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.360 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:29:40.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.360 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.360 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.360 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:40.360 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.361 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.361 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:40.361 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.361 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.361 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:40.361 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:29:40.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.361 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.361 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:29:40.361 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:29:40.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.362 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.362 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:29:40.362 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:29:40.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.362 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.362 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:29:40.362 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:29:40.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.363 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.363 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:29:40.363 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:29:40.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.363 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.363 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.363 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:29:40.363 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:29:40.363 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.363 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.363 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.363 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.363 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.363 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:40.365 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-17T07:29:40.406 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:40.405+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:40.477 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:40.477+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:40.512 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:40.512 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:40.512 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:40.512 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:40.520 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:40.579 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:40.578+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:40.667 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:40.668 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:40.668 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:40.668 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:40.668 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:40.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:40.679 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:40.678+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:40.780 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:40.779+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:40.785 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:40.785 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:40.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:40.785 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:40.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.786 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.786 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:40.786 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650893904999997, 2024-09-17T07:29:40.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:29:40.786 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.786 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.786 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.786 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.786 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.786 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.786 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:29:40.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.787 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.787 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.787 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:40.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.787 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.787 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:40.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.787 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.787 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:29:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:29:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:29:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:29:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:29:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:29:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:29:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:29:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:29:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:29:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:29:40.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:29:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:29:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:29:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:29:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:29:40.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:29:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:29:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:29:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:29:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:29:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:29:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650857008000003, 2024-09-17T07:29:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:29:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.792 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:29:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:29:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:29:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:29:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:29:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:29:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:29:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:29:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:29:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:29:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:29:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:29:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:29:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:29:40.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:29:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:29:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:29:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:29:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:29:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:29:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:29:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:29:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:40.798 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650831041000004, 2024-09-17T07:29:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:29:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:29:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:29:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:29:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:29:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:29:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:29:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:29:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:29:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:29:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:29:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:29:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:29:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:29:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:29:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:29:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:29:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:29:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:29:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:29:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:29:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:29:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:29:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650806445999997, 2024-09-17T07:29:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:29:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:29:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:29:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:29:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:29:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:29:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:29:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:29:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:29:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:29:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:29:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:29:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:29:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:29:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:29:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:29:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:29:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:29:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:29:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:29:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:29:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:29:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:29:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650782958999997, 2024-09-17T07:29:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:29:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:29:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:29:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:29:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:29:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:29:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:29:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:29:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:29:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:29:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:29:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:29:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:29:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:29:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:29:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:29:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:29:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:29:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:29:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:29:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:29:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:29:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650769220999997, 2024-09-17T07:29:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:29:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:29:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:29:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:29:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:29:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:29:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:29:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:29:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:29:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:29:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:29:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:29:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:29:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:29:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:29:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:29:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:29:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:29:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:29:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:29:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:29:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650739031999997, 2024-09-17T07:29:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:29:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:29:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:29:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:29:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:29:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:29:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:29:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:29:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:29:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:29:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:29:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:29:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:29:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:29:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:29:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:29:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:29:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:29:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:29:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:29:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:29:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:29:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650702527, 2024-09-17T07:29:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:29:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:29:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:29:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:29:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:29:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:29:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:29:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:29:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:29:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:29:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:29:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:29:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:29:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:29:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:29:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:29:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:29:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:29:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:29:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:29:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:29:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:29:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650674768000002, 2024-09-17T07:29:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:29:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:29:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:29:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:29:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:29:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:29:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:29:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:29:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:29:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:29:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:29:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:29:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:29:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:29:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:29:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:29:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:29:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:29:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:29:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:29:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:29:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:29:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650652608000001, 2024-09-17T07:29:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:29:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:29:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:29:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:29:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:29:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:29:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:29:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:29:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:29:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:29:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:29:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:29:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:29:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:29:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:29:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:29:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:29:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:29:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:29:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:29:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:29:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:29:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650639447000003, 2024-09-17T07:29:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:29:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:29:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:29:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:29:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:29:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:29:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:29:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:29:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:29:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:29:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:29:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:29:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:29:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:29:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:29:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:29:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:29:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:29:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:29:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:29:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:29:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:29:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650620859, 2024-09-17T07:29:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:29:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:29:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:29:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:29:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:29:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:29:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:29:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:29:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:29:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:29:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:29:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:29:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:29:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:29:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:29:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:29:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:29:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:29:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:29:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:29:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650588048000003, 2024-09-17T07:29:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:29:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:29:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:29:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:29:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:29:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:29:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:29:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:29:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:29:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:29:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:29:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:29:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:29:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:29:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:29:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:29:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:29:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:29:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:29:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:29:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650568991, 2024-09-17T07:29:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:29:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:29:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:29:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:29:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:29:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:29:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:29:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:29:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:29:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:29:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:29:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:29:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:29:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:29:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:29:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:29:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:29:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:29:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:29:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:29:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:29:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:29:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:29:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:29:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650546882, 2024-09-17T07:29:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:29:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:29:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:29:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:29:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:29:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:29:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:29:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:29:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:29:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:29:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:29:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:29:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:29:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:29:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:29:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:29:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:29:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:29:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:29:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:29:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:29:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:29:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:29:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650520774999997, 2024-09-17T07:29:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:29:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:29:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:29:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:29:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:29:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:29:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:29:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:29:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:29:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:29:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:29:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:29:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:29:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:29:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:29:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:29:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:29:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:29:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:29:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:29:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:29:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:29:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:29:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650491629999998, 2024-09-17T07:29:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:29:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:29:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:29:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:29:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:29:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:29:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:29:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:29:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:29:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:29:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:29:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:29:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:29:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:29:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:29:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:29:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:29:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:29:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:29:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:29:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:29:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:29:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650459826999999, 2024-09-17T07:29:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:29:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:29:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:29:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:29:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:29:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:29:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:29:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:29:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:29:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:29:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:29:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:29:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:29:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:29:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:29:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:29:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:29:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:29:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:29:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:29:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:29:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:29:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:29:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650432879999997, 2024-09-17T07:29:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:29:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:29:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:29:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:29:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:29:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:29:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:29:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:29:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:29:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:29:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:29:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:29:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:29:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:29:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:29:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:29:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:29:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:29:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:29:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:29:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:29:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:29:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.650391931000001, 2024-09-17T07:29:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:29:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:29:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:29:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:29:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:29:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:29:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:29:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:29:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:29:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:29:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:29:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:29:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:29:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:29:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:29:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:29:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:29:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:29:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:29:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:29:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:29:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:29:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:29:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:29:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:40.907 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:40.909 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:40.909 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:40.908+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:41.010 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:41.009+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:41.017 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:41.017 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:41.017 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:41.017 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:41.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:41.110 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:41.110+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:41.131 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:41.131 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:41.132 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:41.132 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:41.132 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:41.140 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:41.212 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:41.211+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:41.249 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:41.249 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:41.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:4848 3.9 e33/23 3:97363921:::rbd_data.1183cff49e3a.0000000000000004:head v 33'102, mlcod=32'100)", 2024-09-17T07:29:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 47.007927522999999, 2024-09-17T07:29:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023715110000000002, 2024-09-17T07:29:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.250 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239496+0000", 2024-09-17T07:29:41.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.239497+0000", 2024-09-17T07:29:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.53e-06 2024-09-17T07:29:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240142+0000", 2024-09-17T07:29:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00064481300000000005 2024-09-17T07:29:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240160+0000", 2024-09-17T07:29:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8349999999999999e-05 2024-09-17T07:29:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240169+0000", 2024-09-17T07:29:41.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7690000000000002e-06 2024-09-17T07:29:41.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240246+0000", 2024-09-17T07:29:41.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6277000000000002e-05 2024-09-17T07:29:41.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.240270+0000", 2024-09-17T07:29:41.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4575000000000001e-05 2024-09-17T07:29:41.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.241813+0000", 2024-09-17T07:29:41.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015432860000000001 2024-09-17T07:29:41.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.257 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.241867+0000", 2024-09-17T07:29:41.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3910999999999999e-05 2024-09-17T07:29:41.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.258 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.258 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.258 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:5460 3.9 e33/23 3:90773955:::rbd_data.1183cff49e3a.000000000000000a:head v 33'106, mlcod=33'104)", 2024-09-17T07:29:41.259 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:41.259 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.720830937000002, 2024-09-17T07:29:41.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020825850000000001, 2024-09-17T07:29:41.259 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.260 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.260 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:41.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.261 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.261 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526592+0000", 2024-09-17T07:29:41.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.526593+0000", 2024-09-17T07:29:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.54e-07 2024-09-17T07:29:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527050+0000", 2024-09-17T07:29:41.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000456522 2024-09-17T07:29:41.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527061+0000", 2024-09-17T07:29:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1722000000000001e-05 2024-09-17T07:29:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527067+0000", 2024-09-17T07:29:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0599999999999998e-06 2024-09-17T07:29:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527112+0000", 2024-09-17T07:29:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999000000000001e-05 2024-09-17T07:29:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.527128+0000", 2024-09-17T07:29:41.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6920999999999999e-05 2024-09-17T07:29:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528636+0000", 2024-09-17T07:29:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001507836 2024-09-17T07:29:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.528675+0000", 2024-09-17T07:29:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8671000000000003e-05 2024-09-17T07:29:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.267 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:6005 3.0 e33/23 3:037cd38f:::rbd_data.1183cff49e3a.0000000000000012:head v 33'25, mlcod=32'23)", 2024-09-17T07:29:41.268 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:41.268 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.622797802999997, 2024-09-17T07:29:41.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019123899999999999, 2024-09-17T07:29:41.268 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.268 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.268 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624626+0000", 2024-09-17T07:29:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624627+0000", 2024-09-17T07:29:41.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5600000000000001e-06 2024-09-17T07:29:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624777+0000", 2024-09-17T07:29:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015030800000000001 2024-09-17T07:29:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624792+0000", 2024-09-17T07:29:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5089000000000001e-05 2024-09-17T07:29:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624799+0000", 2024-09-17T07:29:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7939999999999997e-06 2024-09-17T07:29:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624855+0000", 2024-09-17T07:29:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5782999999999998e-05 2024-09-17T07:29:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.624871+0000", 2024-09-17T07:29:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6192e-05 2024-09-17T07:29:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.626507+0000", 2024-09-17T07:29:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016354500000000001 2024-09-17T07:29:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:54.626538+0000", 2024-09-17T07:29:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1214000000000003e-05 2024-09-17T07:29:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.272 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:8595 3.4 e33/23 3:237b2a73:::rbd_data.1183cff49e3a.0000000000000049:head v 33'37, mlcod=33'35)", 2024-09-17T07:29:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.172780844000002, 2024-09-17T07:29:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020228939999999999, 2024-09-17T07:29:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.273 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074642+0000", 2024-09-17T07:29:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074643+0000", 2024-09-17T07:29:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3099999999999997e-07 2024-09-17T07:29:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074834+0000", 2024-09-17T07:29:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000191137 2024-09-17T07:29:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074850+0000", 2024-09-17T07:29:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6101999999999999e-05 2024-09-17T07:29:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074859+0000", 2024-09-17T07:29:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2169999999999994e-06 2024-09-17T07:29:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074919+0000", 2024-09-17T07:29:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.003e-05 2024-09-17T07:29:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.074944+0000", 2024-09-17T07:29:41.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5761e-05 2024-09-17T07:29:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076622+0000", 2024-09-17T07:29:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001677977 2024-09-17T07:29:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:55.076665+0000", 2024-09-17T07:29:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2938999999999997e-05 2024-09-17T07:29:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17069 3.0 e41/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 41'110, mlcod=40'108)", 2024-09-17T07:29:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 38.507671119999998, 2024-09-17T07:29:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011408908000000001, 2024-09-17T07:29:41.279 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.280 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.280 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:41.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:41.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739753+0000", 2024-09-17T07:29:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:29:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743532+0000", 2024-09-17T07:29:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037794510000000001 2024-09-17T07:29:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743557+0000", 2024-09-17T07:29:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5253e-05 2024-09-17T07:29:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743569+0000", 2024-09-17T07:29:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1161999999999999e-05 2024-09-17T07:29:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743650+0000", 2024-09-17T07:29:41.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1087999999999993e-05 2024-09-17T07:29:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743674+0000", 2024-09-17T07:29:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4562e-05 2024-09-17T07:29:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751095+0000", 2024-09-17T07:29:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074211140000000004 2024-09-17T07:29:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751161+0000", 2024-09-17T07:29:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5707e-05 2024-09-17T07:29:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17240 3.0 e42/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 42'113, mlcod=41'110)", 2024-09-17T07:29:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 38.022012635999999, 2024-09-17T07:29:41.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064392440000000002, 2024-09-17T07:29:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:29:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225418+0000", 2024-09-17T07:29:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8360000000000003e-06 2024-09-17T07:29:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225426+0000", 2024-09-17T07:29:41.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2670000000000006e-06 2024-09-17T07:29:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225434+0000", 2024-09-17T07:29:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0560000000000005e-06 2024-09-17T07:29:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225514+0000", 2024-09-17T07:29:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9979000000000001e-05 2024-09-17T07:29:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225535+0000", 2024-09-17T07:29:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0965e-05 2024-09-17T07:29:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231797+0000", 2024-09-17T07:29:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062621129999999997 2024-09-17T07:29:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231850+0000", 2024-09-17T07:29:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2611999999999997e-05 2024-09-17T07:29:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.289 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20054 3.9 e44/23 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:head v 44'131, mlcod=44'129)", 2024-09-17T07:29:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.97986152, 2024-09-17T07:29:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028510570000000002, 2024-09-17T07:29:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.290 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267563+0000", 2024-09-17T07:29:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.584e-06 2024-09-17T07:29:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268269+0000", 2024-09-17T07:29:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00070567399999999999 2024-09-17T07:29:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268287+0000", 2024-09-17T07:29:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.84e-05 2024-09-17T07:29:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268297+0000", 2024-09-17T07:29:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9059999999999997e-06 2024-09-17T07:29:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268368+0000", 2024-09-17T07:29:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0562999999999995e-05 2024-09-17T07:29:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268393+0000", 2024-09-17T07:29:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4958999999999999e-05 2024-09-17T07:29:41.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270330+0000", 2024-09-17T07:29:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001936644 2024-09-17T07:29:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270413+0000", 2024-09-17T07:29:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3326999999999997e-05 2024-09-17T07:29:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20055 3.9 e44/23 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:head v 44'133, mlcod=44'129)", 2024-09-17T07:29:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.268886+0000", 2024-09-17T07:29:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.978537449000001, 2024-09-17T07:29:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019232920000000001, 2024-09-17T07:29:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268886+0000", 2024-09-17T07:29:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268886+0000", 2024-09-17T07:29:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268887+0000", 2024-09-17T07:29:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.85e-07 2024-09-17T07:29:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268905+0000", 2024-09-17T07:29:41.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8263000000000001e-05 2024-09-17T07:29:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268916+0000", 2024-09-17T07:29:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0876000000000001e-05 2024-09-17T07:29:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268922+0000", 2024-09-17T07:29:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7579999999999998e-06 2024-09-17T07:29:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.269201+0000", 2024-09-17T07:29:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027979200000000001 2024-09-17T07:29:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.269224+0000", 2024-09-17T07:29:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2838999999999999e-05 2024-09-17T07:29:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270774+0000", 2024-09-17T07:29:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015495629999999999 2024-09-17T07:29:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270809+0000", 2024-09-17T07:29:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5416000000000001e-05 2024-09-17T07:29:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.1f 51/50 [PushOp(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head, version: 40'83, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head@40'83, size: 19583394, copy_subset: [(0, 19583394)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:29:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:13.339735+0000", 2024-09-17T07:29:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 27.907688750999998, 2024-09-17T07:29:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0018873760000000001, 2024-09-17T07:29:41.299 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.339735+0000", 2024-09-17T07:29:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.339735+0000", 2024-09-17T07:29:41.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.339736+0000", 2024-09-17T07:29:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.392e-06 2024-09-17T07:29:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340637+0000", 2024-09-17T07:29:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00090133599999999998 2024-09-17T07:29:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340668+0000", 2024-09-17T07:29:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0488e-05 2024-09-17T07:29:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340683+0000", 2024-09-17T07:29:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5315000000000002e-05 2024-09-17T07:29:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340777+0000", 2024-09-17T07:29:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3456999999999996e-05 2024-09-17T07:29:41.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.340786+0000", 2024-09-17T07:29:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2220000000000008e-06 2024-09-17T07:29:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.341622+0000", 2024-09-17T07:29:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083616599999999995 2024-09-17T07:29:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.303 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.1f 51/50 [PushOp(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head, version: 40'83, data_included: [3407872~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:fb5d8155:::rbd_data.118383c6fe43.0000000000000003:head@40'83, size: 19583394, copy_subset: [(0, 19583394)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4456448, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3407872, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:29:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:13.350296+0000", 2024-09-17T07:29:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 27.897127806, 2024-09-17T07:29:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00191949, 2024-09-17T07:29:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.350296+0000", 2024-09-17T07:29:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.350296+0000", 2024-09-17T07:29:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.350297+0000", 2024-09-17T07:29:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4440000000000001e-06 2024-09-17T07:29:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351304+0000", 2024-09-17T07:29:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001007319 2024-09-17T07:29:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351331+0000", 2024-09-17T07:29:41.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6701000000000001e-05 2024-09-17T07:29:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351346+0000", 2024-09-17T07:29:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4911e-05 2024-09-17T07:29:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351425+0000", 2024-09-17T07:29:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9328999999999999e-05 2024-09-17T07:29:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.351436+0000", 2024-09-17T07:29:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0467e-05 2024-09-17T07:29:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:13.352215+0000", 2024-09-17T07:29:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000779319 2024-09-17T07:29:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:32543 3.0 e55/23 3:03bfd97d:::rbd_data.11839a97b753.0000000000000007:head v 55'150, mlcod=54'148)", 2024-09-17T07:29:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 23.792786382999999, 2024-09-17T07:29:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031813480000000001, 2024-09-17T07:29:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5199999999999998e-07 2024-09-17T07:29:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455083+0000", 2024-09-17T07:29:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044629199999999999 2024-09-17T07:29:41.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455098+0000", 2024-09-17T07:29:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4183e-05 2024-09-17T07:29:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455106+0000", 2024-09-17T07:29:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:29:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455173+0000", 2024-09-17T07:29:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6346000000000005e-05 2024-09-17T07:29:41.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455200+0000", 2024-09-17T07:29:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7319000000000001e-05 2024-09-17T07:29:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457774+0000", 2024-09-17T07:29:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025742650000000001 2024-09-17T07:29:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457818+0000", 2024-09-17T07:29:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3891999999999999e-05 2024-09-17T07:29:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33735 3.1f e55/52 3:fea54097:::rbd_data.11839a97b753.0000000000000000:head v 55'260, mlcod=55'258)", 2024-09-17T07:29:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:29:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 23.422853754999998, 2024-09-17T07:29:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035215200000000002, 2024-09-17T07:29:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.313 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:29:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:29:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824571+0000", 2024-09-17T07:29:41.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.164e-06 2024-09-17T07:29:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825118+0000", 2024-09-17T07:29:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054696400000000004 2024-09-17T07:29:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825134+0000", 2024-09-17T07:29:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6342999999999999e-05 2024-09-17T07:29:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825142+0000", 2024-09-17T07:29:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5270000000000001e-06 2024-09-17T07:29:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825200+0000", 2024-09-17T07:29:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.893e-05 2024-09-17T07:29:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825218+0000", 2024-09-17T07:29:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7935999999999999e-05 2024-09-17T07:29:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828037+0000", 2024-09-17T07:29:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002818961 2024-09-17T07:29:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828091+0000", 2024-09-17T07:29:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3695e-05 2024-09-17T07:29:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33869 3.9 e55/23 3:907b81b5:::rbd_data.11839a97b753.0000000000000001:head v 55'171, mlcod=55'169)", 2024-09-17T07:29:41.317 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:29:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 23.405797931999999, 2024-09-17T07:29:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033990859999999999, 2024-09-17T07:29:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:29:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:29:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841626+0000", 2024-09-17T07:29:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.023e-06 2024-09-17T07:29:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842067+0000", 2024-09-17T07:29:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000440612 2024-09-17T07:29:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842080+0000", 2024-09-17T07:29:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.29e-05 2024-09-17T07:29:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842086+0000", 2024-09-17T07:29:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2090000000000002e-06 2024-09-17T07:29:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842144+0000", 2024-09-17T07:29:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7627999999999998e-05 2024-09-17T07:29:41.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842171+0000", 2024-09-17T07:29:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7214000000000001e-05 2024-09-17T07:29:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.844973+0000", 2024-09-17T07:29:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002802232 2024-09-17T07:29:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.845024+0000", 2024-09-17T07:29:41.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1267999999999998e-05 2024-09-17T07:29:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51432 3.0 e59/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 59'177, mlcod=58'174)", 2024-09-17T07:29:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:29:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.426200914999999, 2024-09-17T07:29:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010790086000000001, 2024-09-17T07:29:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:29:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:29:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821224+0000", 2024-09-17T07:29:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.511e-06 2024-09-17T07:29:41.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825174+0000", 2024-09-17T07:29:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0039497949999999999 2024-09-17T07:29:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825196+0000", 2024-09-17T07:29:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2643999999999999e-05 2024-09-17T07:29:41.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825206+0000", 2024-09-17T07:29:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4320000000000005e-06 2024-09-17T07:29:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825272+0000", 2024-09-17T07:29:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5829999999999998e-05 2024-09-17T07:29:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825299+0000", 2024-09-17T07:29:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7107e-05 2024-09-17T07:29:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.831939+0000", 2024-09-17T07:29:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0066398510000000004 2024-09-17T07:29:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.832012+0000", 2024-09-17T07:29:41.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3916000000000002e-05 2024-09-17T07:29:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51969 3.0 e59/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 59'179, mlcod=59'177)", 2024-09-17T07:29:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:29:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.317332536999999, 2024-09-17T07:29:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060297249999999997, 2024-09-17T07:29:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:29:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:29:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930093+0000", 2024-09-17T07:29:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.764e-06 2024-09-17T07:29:41.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931779+0000", 2024-09-17T07:29:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001686379 2024-09-17T07:29:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931802+0000", 2024-09-17T07:29:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2682000000000001e-05 2024-09-17T07:29:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931811+0000", 2024-09-17T07:29:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5640000000000002e-06 2024-09-17T07:29:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931877+0000", 2024-09-17T07:29:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6155999999999995e-05 2024-09-17T07:29:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931896+0000", 2024-09-17T07:29:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9111e-05 2024-09-17T07:29:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936063+0000", 2024-09-17T07:29:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004166711 2024-09-17T07:29:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936121+0000", 2024-09-17T07:29:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7358000000000002e-05 2024-09-17T07:29:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54417 3.0 e60/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 60'182, mlcod=59'179)", 2024-09-17T07:29:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:29:41.331 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.809815545999999, 2024-09-17T07:29:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0059885219999999996, 2024-09-17T07:29:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:29:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:29:41.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437609+0000", 2024-09-17T07:29:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5829999999999999e-06 2024-09-17T07:29:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437623+0000", 2024-09-17T07:29:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3348e-05 2024-09-17T07:29:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.333 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437643+0000", 2024-09-17T07:29:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0463999999999999e-05 2024-09-17T07:29:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437653+0000", 2024-09-17T07:29:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6290000000000005e-06 2024-09-17T07:29:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437719+0000", 2024-09-17T07:29:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6428999999999997e-05 2024-09-17T07:29:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437745+0000", 2024-09-17T07:29:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5690999999999998e-05 2024-09-17T07:29:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443549+0000", 2024-09-17T07:29:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0058038259999999998 2024-09-17T07:29:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443596+0000", 2024-09-17T07:29:41.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7552000000000001e-05 2024-09-17T07:29:41.336 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.336 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.336 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54422 3.1f e60/52 3:fef8599a:::rbd_data.1183a58293a3.0000000000000007:head v 60'280, mlcod=58'277)", 2024-09-17T07:29:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.441086+0000", 2024-09-17T07:29:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.806337170999999, 2024-09-17T07:29:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001916933, 2024-09-17T07:29:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441086+0000", 2024-09-17T07:29:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441086+0000", 2024-09-17T07:29:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441088+0000", 2024-09-17T07:29:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.517e-06 2024-09-17T07:29:41.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441100+0000", 2024-09-17T07:29:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2532e-05 2024-09-17T07:29:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441118+0000", 2024-09-17T07:29:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7798000000000001e-05 2024-09-17T07:29:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441127+0000", 2024-09-17T07:29:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4749999999999999e-06 2024-09-17T07:29:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441675+0000", 2024-09-17T07:29:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054747699999999999 2024-09-17T07:29:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.441693+0000", 2024-09-17T07:29:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7621e-05 2024-09-17T07:29:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.442956+0000", 2024-09-17T07:29:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012633900000000001 2024-09-17T07:29:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443003+0000", 2024-09-17T07:29:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7123000000000003e-05 2024-09-17T07:29:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.341 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54419 3.0 e60/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 60'184, mlcod=59'179)", 2024-09-17T07:29:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:29:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.803463188999999, 2024-09-17T07:29:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004706844, 2024-09-17T07:29:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:29:41.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:29:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443961+0000", 2024-09-17T07:29:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:29:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443967+0000", 2024-09-17T07:29:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.268e-06 2024-09-17T07:29:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443983+0000", 2024-09-17T07:29:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5248999999999999e-05 2024-09-17T07:29:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443990+0000", 2024-09-17T07:29:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5120000000000002e-06 2024-09-17T07:29:41.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444056+0000", 2024-09-17T07:29:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6197000000000003e-05 2024-09-17T07:29:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444081+0000", 2024-09-17T07:29:41.345 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4791e-05 2024-09-17T07:29:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448616+0000", 2024-09-17T07:29:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045346129999999998 2024-09-17T07:29:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448667+0000", 2024-09-17T07:29:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-17T07:29:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54895 3.1f e60/52 3:fa1c1177:::rbd_data.1183ad87379e.0000000000000001:head v 60'283, mlcod=60'281)", 2024-09-17T07:29:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:29:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.455982193000001, 2024-09-17T07:29:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0029495210000000001, 2024-09-17T07:29:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.347 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:29:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:29:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791443+0000", 2024-09-17T07:29:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.153e-06 2024-09-17T07:29:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792619+0000", 2024-09-17T07:29:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001176011 2024-09-17T07:29:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792637+0000", 2024-09-17T07:29:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-05 2024-09-17T07:29:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.349 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792649+0000", 2024-09-17T07:29:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1712000000000001e-05 2024-09-17T07:29:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792722+0000", 2024-09-17T07:29:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3545000000000001e-05 2024-09-17T07:29:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792738+0000", 2024-09-17T07:29:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5223e-05 2024-09-17T07:29:41.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.794330+0000", 2024-09-17T07:29:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015922740000000001 2024-09-17T07:29:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.794391+0000", 2024-09-17T07:29:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0803e-05 2024-09-17T07:29:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:80794 3.9 e72/23 3:90311a61:::rbd_data.1183cb746bb8.0000000000000007:head v 72'222, mlcod=71'220)", 2024-09-17T07:29:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:29:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.815474305, 2024-09-17T07:29:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004371733, 2024-09-17T07:29:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:29:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:29:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:29:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0800000000000001e-07 2024-09-17T07:29:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434108+0000", 2024-09-17T07:29:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021587569999999999 2024-09-17T07:29:41.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434120+0000", 2024-09-17T07:29:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1821000000000001e-05 2024-09-17T07:29:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434128+0000", 2024-09-17T07:29:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7579999999999992e-06 2024-09-17T07:29:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434191+0000", 2024-09-17T07:29:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3202999999999998e-05 2024-09-17T07:29:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434218+0000", 2024-09-17T07:29:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6771000000000002e-05 2024-09-17T07:29:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436289+0000", 2024-09-17T07:29:41.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020717700000000001 2024-09-17T07:29:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436321+0000", 2024-09-17T07:29:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1445000000000001e-05 2024-09-17T07:29:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:41.356 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:41.356 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:41.358 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:41.358 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:41.357+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:41.458 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:41.458+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:41.485 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:41.485 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:41.485 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:41.486 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:41.497 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:41.581 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:41.581+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:41.606 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-17T07:29:41.614 INFO:teuthology.orchestra.run.smithi045.stdout:3432 writesame 0x5f86de0 thru 0x5f93e95 data_size 0x4e(0xd0b6 bytes) 2024-09-17T07:29:41.636 INFO:teuthology.orchestra.run.smithi045.stdout:3433 trunc from 0xe60551a to 0x9bb7eae 2024-09-17T07:29:41.656 INFO:teuthology.orchestra.run.smithi045.stdout:3434 read 0x264f447 thru 0x264f9fb (0x5b5 bytes) 2024-09-17T07:29:41.660 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:41.660+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:41.668 INFO:teuthology.orchestra.run.smithi045.stdout:3435 writesame 0xdc0dd49 thru 0xdc1be04 data_size 0x26(0xe0bc bytes) 2024-09-17T07:29:41.668 INFO:teuthology.orchestra.run.smithi045.stdout:3437 read 0x397dda6 thru 0x398124e (0x34a9 bytes) 2024-09-17T07:29:41.668 INFO:teuthology.orchestra.run.smithi045.stdout:3438 write 0x12c78f2 thru 0x12c805b (0x76a bytes) 2024-09-17T07:29:41.681 INFO:teuthology.orchestra.run.smithi045.stdout:3439 writesame 0x26acb34 thru 0x26b5d41 data_size 0x5(0x920e bytes) 2024-09-17T07:29:41.688 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:41.688 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:41.688 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:41.689 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:41.689 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:41.700 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:41.706 INFO:teuthology.orchestra.run.smithi045.stdout:3440 trunc from 0xdc1be05 to 0x1c6163d 2024-09-17T07:29:41.751 INFO:teuthology.orchestra.run.smithi045.stdout:3441 punch from 0x3d5b5 to 0x49f12, (0xc95d bytes) 2024-09-17T07:29:41.767 INFO:teuthology.orchestra.run.smithi045.stdout:3442 writesame 0xa085ca2 thru 0xa089638 data_size 0x1(0x3997 bytes) 2024-09-17T07:29:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:3443 read 0x8632430 thru 0x8636355 (0x3f26 bytes) 2024-09-17T07:29:41.771 INFO:teuthology.orchestra.run.smithi045.stdout:3444 trunc from 0xa089639 to 0x6d05c45 2024-09-17T07:29:41.779 INFO:teuthology.orchestra.run.smithi045.stdout:3445 read 0x66ee9b9 thru 0x66f435f (0x59a7 bytes) 2024-09-17T07:29:41.780 INFO:teuthology.orchestra.run.smithi045.stdout:3446 write 0x516aa93 thru 0x51703c1 (0x592f bytes) 2024-09-17T07:29:41.784 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:41.784+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:41.787 INFO:teuthology.orchestra.run.smithi045.stdout:3448 write 0x6b09d9f thru 0x6b11ff9 (0x825b bytes) 2024-09-17T07:29:41.800 INFO:teuthology.orchestra.run.smithi045.stdout:3451 trunc from 0x6d05c45 to 0x85368df 2024-09-17T07:29:41.809 INFO:teuthology.orchestra.run.smithi045.stdout:3452 read 0x12b1b86 thru 0x12b2214 (0x68f bytes) 2024-09-17T07:29:41.810 INFO:teuthology.orchestra.run.smithi045.stdout:3453 read 0x5f8bf71 thru 0x5f8f7e6 (0x3876 bytes) 2024-09-17T07:29:41.810 INFO:teuthology.orchestra.run.smithi045.stdout:3455 read 0x3ca0f80 thru 0x3ca6101 (0x5182 bytes) 2024-09-17T07:29:41.810 INFO:teuthology.orchestra.run.smithi045.stdout:3456 trunc from 0x85368df to 0x911dd3f 2024-09-17T07:29:41.816 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:41.828 INFO:teuthology.orchestra.run.smithi045.stdout:3460 writesame 0xd93b8f8 thru 0xd93c053 data_size 0x6(0x75c bytes) 2024-09-17T07:29:41.828 INFO:teuthology.orchestra.run.smithi045.stdout:3461 read 0x9a1a239 thru 0x9a21172 (0x6f3a bytes) 2024-09-17T07:29:41.828 INFO:teuthology.orchestra.run.smithi045.stdout:3462 read 0x9fe35b7 thru 0x9fed8d5 (0xa31f bytes) 2024-09-17T07:29:41.828 INFO:teuthology.orchestra.run.smithi045.stdout:3465 write 0xecfe8b thru 0xedc990 (0xcb06 bytes) 2024-09-17T07:29:41.846 INFO:teuthology.orchestra.run.smithi045.stdout:3466 writesame 0x2a2cc1a thru 0x2a2e955 data_size 0x2(0x1d3c bytes) 2024-09-17T07:29:41.854 INFO:teuthology.orchestra.run.smithi045.stdout:3468 writesame 0x855dad3 thru 0x8568de2 data_size 0x14(0xb310 bytes) 2024-09-17T07:29:41.863 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:41.862+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:41.868 INFO:teuthology.orchestra.run.smithi045.stdout:3469 write 0xace95c9 thru 0xacf33f3 (0x9e2b bytes) 2024-09-17T07:29:41.876 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:41.876 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:41.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:41.876 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:41.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.876 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:30525 3.1b 3:def7c8f0:::rbd_data.118381ffbbab.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T07:29:41.877 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:41.877 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.352323348999999, 2024-09-17T07:29:41.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55515850300000003, 2024-09-17T07:29:41.877 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.877 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.877 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:41.877 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.877 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.877 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.877 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30525 2024-09-17T07:29:41.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.877 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:41.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:41.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522300+0000", 2024-09-17T07:29:41.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:29:41.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522305+0000", 2024-09-17T07:29:41.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-17T07:29:41.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522318+0000", 2024-09-17T07:29:41.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3030999999999999e-05 2024-09-17T07:29:41.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522326+0000", 2024-09-17T07:29:41.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4200000000000001e-06 2024-09-17T07:29:41.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522385+0000", 2024-09-17T07:29:41.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9005000000000002e-05 2024-09-17T07:29:41.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:29:41.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522393+0000", 2024-09-17T07:29:41.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-06 2024-09-17T07:29:41.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075347+0000", 2024-09-17T07:29:41.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55295417800000002 2024-09-17T07:29:41.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075502+0000", 2024-09-17T07:29:41.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154272 2024-09-17T07:29:41.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:41.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075630+0000", 2024-09-17T07:29:41.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012816300000000001 2024-09-17T07:29:41.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:41.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077221+0000", 2024-09-17T07:29:41.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015912929999999999 2024-09-17T07:29:41.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:41.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077353+0000", 2024-09-17T07:29:41.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013215699999999999 2024-09-17T07:29:41.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:41.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077413+0000", 2024-09-17T07:29:41.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0034999999999997e-05 2024-09-17T07:29:41.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077457+0000", 2024-09-17T07:29:41.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4017999999999998e-05 2024-09-17T07:29:41.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.884 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.884 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44181 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13922304~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.884 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:29:41.884 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.237144700999998, 2024-09-17T07:29:41.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046010502000000002, 2024-09-17T07:29:41.885 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.885 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.885 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.885 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44181 2024-09-17T07:29:41.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.885 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:29:41.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:29:41.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637478+0000", 2024-09-17T07:29:41.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:29:41.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637480+0000", 2024-09-17T07:29:41.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.843e-06 2024-09-17T07:29:41.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637487+0000", 2024-09-17T07:29:41.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6159999999999996e-06 2024-09-17T07:29:41.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637492+0000", 2024-09-17T07:29:41.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5360000000000003e-06 2024-09-17T07:29:41.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683351+0000", 2024-09-17T07:29:41.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045858758999999999 2024-09-17T07:29:41.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683422+0000", 2024-09-17T07:29:41.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1705999999999994e-05 2024-09-17T07:29:41.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683488+0000", 2024-09-17T07:29:41.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5612999999999997e-05 2024-09-17T07:29:41.889 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.889 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.889 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.889 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44182 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13938688~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.889 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:41.889 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.23712403, 2024-09-17T07:29:41.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046153118999999999, 2024-09-17T07:29:41.889 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.889 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.889 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.889 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.890 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44182 2024-09-17T07:29:41.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.890 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:41.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:41.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:41.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:29:41.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637506+0000", 2024-09-17T07:29:41.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1520000000000003e-06 2024-09-17T07:29:41.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637512+0000", 2024-09-17T07:29:41.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-09-17T07:29:41.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637517+0000", 2024-09-17T07:29:41.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-09-17T07:29:41.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683514+0000", 2024-09-17T07:29:41.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045997616999999998 2024-09-17T07:29:41.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683584+0000", 2024-09-17T07:29:41.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9942000000000002e-05 2024-09-17T07:29:41.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683651+0000", 2024-09-17T07:29:41.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6951000000000003e-05 2024-09-17T07:29:41.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.896 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.897 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44183 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13955072~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.897 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:29:41.897 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.237098798000002, 2024-09-17T07:29:41.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046339793999999997, 2024-09-17T07:29:41.897 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.897 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.897 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.897 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44183 2024-09-17T07:29:41.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.897 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:29:41.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:29:41.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639583+0000", 2024-09-17T07:29:41.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002059412 2024-09-17T07:29:41.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639597+0000", 2024-09-17T07:29:41.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3832000000000001e-05 2024-09-17T07:29:41.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639607+0000", 2024-09-17T07:29:41.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485e-05 2024-09-17T07:29:41.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639614+0000", 2024-09-17T07:29:41.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0849999999999999e-06 2024-09-17T07:29:41.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683705+0000", 2024-09-17T07:29:41.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044091261 2024-09-17T07:29:41.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683792+0000", 2024-09-17T07:29:41.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6197000000000002e-05 2024-09-17T07:29:41.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683863+0000", 2024-09-17T07:29:41.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1521999999999997e-05 2024-09-17T07:29:41.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.901 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.901 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44199 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14217216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.901 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:41.901 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.233581246, 2024-09-17T07:29:41.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045995623999999999, 2024-09-17T07:29:41.902 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.902 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.902 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.902 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44199 2024-09-17T07:29:41.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.902 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:41.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:41.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:41.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:29:41.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641043+0000", 2024-09-17T07:29:41.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.624e-06 2024-09-17T07:29:41.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641048+0000", 2024-09-17T07:29:41.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4029999999999998e-06 2024-09-17T07:29:41.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641053+0000", 2024-09-17T07:29:41.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6670000000000004e-06 2024-09-17T07:29:41.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686918+0000", 2024-09-17T07:29:41.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045865131000000003 2024-09-17T07:29:41.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686986+0000", 2024-09-17T07:29:41.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8299000000000005e-05 2024-09-17T07:29:41.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687036+0000", 2024-09-17T07:29:41.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0164999999999998e-05 2024-09-17T07:29:41.906 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.906 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.906 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.906 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44200 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14233600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.906 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:41.907 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.233563416999999, 2024-09-17T07:29:41.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046183984999999997, 2024-09-17T07:29:41.907 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.907 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.907 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.907 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.907 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.907 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.907 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44200 2024-09-17T07:29:41.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.907 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:41.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:41.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:41.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:41.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641061+0000", 2024-09-17T07:29:41.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.767e-06 2024-09-17T07:29:41.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641067+0000", 2024-09-17T07:29:41.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7379999999999999e-06 2024-09-17T07:29:41.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641071+0000", 2024-09-17T07:29:41.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-09-17T07:29:41.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687078+0000", 2024-09-17T07:29:41.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046007095999999997 2024-09-17T07:29:41.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687158+0000", 2024-09-17T07:29:41.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0374e-05 2024-09-17T07:29:41.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687243+0000", 2024-09-17T07:29:41.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4221999999999994e-05 2024-09-17T07:29:41.913 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.913 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.913 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.913 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44201 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14249984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.914 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:29:41.914 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.233545770999999, 2024-09-17T07:29:41.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046303182999999998, 2024-09-17T07:29:41.914 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.914 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.914 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.914 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.915 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.915 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44201 2024-09-17T07:29:41.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.915 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:29:41.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:29:41.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641120+0000", 2024-09-17T07:29:41.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3933000000000001e-05 2024-09-17T07:29:41.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641123+0000", 2024-09-17T07:29:41.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.191e-06 2024-09-17T07:29:41.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641133+0000", 2024-09-17T07:29:41.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4020000000000008e-06 2024-09-17T07:29:41.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641138+0000", 2024-09-17T07:29:41.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6559999999999997e-06 2024-09-17T07:29:41.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687279+0000", 2024-09-17T07:29:41.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046140278999999999 2024-09-17T07:29:41.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687338+0000", 2024-09-17T07:29:41.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9268999999999998e-05 2024-09-17T07:29:41.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687379+0000", 2024-09-17T07:29:41.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1452999999999998e-05 2024-09-17T07:29:41.920 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.920 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.920 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.921 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44202 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14266368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.921 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:29:41.921 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.233475004999999, 2024-09-17T07:29:41.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046484299, 2024-09-17T07:29:41.921 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.921 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.921 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.921 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.921 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.921 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.921 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44202 2024-09-17T07:29:41.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.922 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:29:41.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:29:41.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642810+0000", 2024-09-17T07:29:41.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016627980000000001 2024-09-17T07:29:41.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642815+0000", 2024-09-17T07:29:41.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.091e-06 2024-09-17T07:29:41.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642829+0000", 2024-09-17T07:29:41.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4047e-05 2024-09-17T07:29:41.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642844+0000", 2024-09-17T07:29:41.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4515000000000001e-05 2024-09-17T07:29:41.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687426+0000", 2024-09-17T07:29:41.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044582201000000002 2024-09-17T07:29:41.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687529+0000", 2024-09-17T07:29:41.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103078 2024-09-17T07:29:41.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687631+0000", 2024-09-17T07:29:41.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102569 2024-09-17T07:29:41.925 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.925 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.925 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.925 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44209 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14381056~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.925 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:29:41.925 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.231576853, 2024-09-17T07:29:41.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045944444000000001, 2024-09-17T07:29:41.926 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.926 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.926 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.926 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.926 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.926 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.926 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44209 2024-09-17T07:29:41.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.926 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:29:41.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:29:41.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643046+0000", 2024-09-17T07:29:41.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:29:41.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643048+0000", 2024-09-17T07:29:41.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:29:41.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643054+0000", 2024-09-17T07:29:41.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.032e-06 2024-09-17T07:29:41.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643065+0000", 2024-09-17T07:29:41.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1861999999999999e-05 2024-09-17T07:29:41.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688806+0000", 2024-09-17T07:29:41.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045740617999999997 2024-09-17T07:29:41.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688927+0000", 2024-09-17T07:29:41.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000121327 2024-09-17T07:29:41.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688990+0000", 2024-09-17T07:29:41.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2293999999999997e-05 2024-09-17T07:29:41.930 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.930 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.930 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.930 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44210 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14397440~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.930 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:29:41.930 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.231549927, 2024-09-17T07:29:41.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046104635999999997, 2024-09-17T07:29:41.931 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.931 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.931 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.931 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.931 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.931 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.931 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44210 2024-09-17T07:29:41.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.931 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:29:41.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:29:41.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643073+0000", 2024-09-17T07:29:41.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.77e-07 2024-09-17T07:29:41.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643075+0000", 2024-09-17T07:29:41.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-09-17T07:29:41.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643082+0000", 2024-09-17T07:29:41.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7789999999999996e-06 2024-09-17T07:29:41.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643087+0000", 2024-09-17T07:29:41.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.967e-06 2024-09-17T07:29:41.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689022+0000", 2024-09-17T07:29:41.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045934349999999999 2024-09-17T07:29:41.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689112+0000", 2024-09-17T07:29:41.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0039999999999999e-05 2024-09-17T07:29:41.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689177+0000", 2024-09-17T07:29:41.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5041000000000003e-05 2024-09-17T07:29:41.934 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.934 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.935 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44211 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14413824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.935 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:29:41.935 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.231528793999999, 2024-09-17T07:29:41.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046309003000000001, 2024-09-17T07:29:41.935 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.935 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.935 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.935 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.935 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.935 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.935 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44211 2024-09-17T07:29:41.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.936 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:29:41.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:29:41.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643094+0000", 2024-09-17T07:29:41.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:29:41.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643095+0000", 2024-09-17T07:29:41.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-17T07:29:41.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643101+0000", 2024-09-17T07:29:41.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8730000000000004e-06 2024-09-17T07:29:41.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643106+0000", 2024-09-17T07:29:41.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6859999999999997e-06 2024-09-17T07:29:41.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689220+0000", 2024-09-17T07:29:41.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046113945000000003 2024-09-17T07:29:41.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689309+0000", 2024-09-17T07:29:41.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9560000000000003e-05 2024-09-17T07:29:41.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689402+0000", 2024-09-17T07:29:41.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2786000000000002e-05 2024-09-17T07:29:41.940 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.940 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.940 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.940 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44212 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14430208~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.940 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:41.940 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.231484232, 2024-09-17T07:29:41.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046448912000000002, 2024-09-17T07:29:41.941 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.941 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.941 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.941 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44212 2024-09-17T07:29:41.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.941 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:41.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:41.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:41.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2200000000000004e-07 2024-09-17T07:29:41.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643141+0000", 2024-09-17T07:29:41.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5210000000000001e-06 2024-09-17T07:29:41.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643149+0000", 2024-09-17T07:29:41.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6490000000000008e-06 2024-09-17T07:29:41.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643161+0000", 2024-09-17T07:29:41.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2241e-05 2024-09-17T07:29:41.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689423+0000", 2024-09-17T07:29:41.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046262323000000001 2024-09-17T07:29:41.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689505+0000", 2024-09-17T07:29:41.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2103999999999999e-05 2024-09-17T07:29:41.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689587+0000", 2024-09-17T07:29:41.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1452000000000006e-05 2024-09-17T07:29:41.945 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.945 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.945 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.945 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44213 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14446592~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.945 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:29:41.946 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.231452733000001, 2024-09-17T07:29:41.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046601780000000002, 2024-09-17T07:29:41.946 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.946 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.946 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.946 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44213 2024-09-17T07:29:41.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.946 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:29:41.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:29:41.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643170+0000", 2024-09-17T07:29:41.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-17T07:29:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643172+0000", 2024-09-17T07:29:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1660000000000001e-06 2024-09-17T07:29:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643180+0000", 2024-09-17T07:29:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-09-17T07:29:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643185+0000", 2024-09-17T07:29:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-09-17T07:29:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689622+0000", 2024-09-17T07:29:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046436742000000003 2024-09-17T07:29:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689697+0000", 2024-09-17T07:29:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5179999999999995e-05 2024-09-17T07:29:41.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689771+0000", 2024-09-17T07:29:41.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3928000000000001e-05 2024-09-17T07:29:41.950 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.950 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.950 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.950 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44214 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14462976~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.950 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:29:41.950 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.231429673000001, 2024-09-17T07:29:41.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046736038000000001, 2024-09-17T07:29:41.950 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.951 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.951 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.951 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.951 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.951 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.951 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44214 2024-09-17T07:29:41.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.951 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:29:41.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:29:41.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643193+0000", 2024-09-17T07:29:41.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:29:41.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643195+0000", 2024-09-17T07:29:41.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0640000000000001e-06 2024-09-17T07:29:41.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643201+0000", 2024-09-17T07:29:41.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2859999999999996e-06 2024-09-17T07:29:41.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643206+0000", 2024-09-17T07:29:41.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-17T07:29:41.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689814+0000", 2024-09-17T07:29:41.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046607742000000001 2024-09-17T07:29:41.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689886+0000", 2024-09-17T07:29:41.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2020999999999994e-05 2024-09-17T07:29:41.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689928+0000", 2024-09-17T07:29:41.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2771000000000001e-05 2024-09-17T07:29:41.955 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.955 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.955 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.955 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44215 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14479360~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.955 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:29:41.955 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.231385776, 2024-09-17T07:29:41.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046868543999999998, 2024-09-17T07:29:41.956 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.956 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.956 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.956 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.956 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.956 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.956 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44215 2024-09-17T07:29:41.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.956 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:29:41.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:29:41.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643237+0000", 2024-09-17T07:29:41.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:29:41.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643245+0000", 2024-09-17T07:29:41.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3270000000000001e-06 2024-09-17T07:29:41.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643254+0000", 2024-09-17T07:29:41.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6049999999999998e-06 2024-09-17T07:29:41.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643260+0000", 2024-09-17T07:29:41.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7849999999999997e-06 2024-09-17T07:29:41.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689961+0000", 2024-09-17T07:29:41.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046701196 2024-09-17T07:29:41.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690036+0000", 2024-09-17T07:29:41.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5153000000000004e-05 2024-09-17T07:29:41.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690105+0000", 2024-09-17T07:29:41.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8874000000000005e-05 2024-09-17T07:29:41.960 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.960 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.960 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.960 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44216 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14495744~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.960 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:29:41.960 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.231354063000001, 2024-09-17T07:29:41.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047003047999999999, 2024-09-17T07:29:41.960 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.961 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.961 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.961 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44216 2024-09-17T07:29:41.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.961 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:29:41.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:29:41.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643269+0000", 2024-09-17T07:29:41.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:29:41.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643271+0000", 2024-09-17T07:29:41.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-09-17T07:29:41.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643278+0000", 2024-09-17T07:29:41.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8850000000000001e-06 2024-09-17T07:29:41.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643282+0000", 2024-09-17T07:29:41.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5979999999999999e-06 2024-09-17T07:29:41.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690133+0000", 2024-09-17T07:29:41.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850470999999998 2024-09-17T07:29:41.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690205+0000", 2024-09-17T07:29:41.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2442999999999997e-05 2024-09-17T07:29:41.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690271+0000", 2024-09-17T07:29:41.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5874e-05 2024-09-17T07:29:41.965 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.965 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.965 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.965 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44217 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14512128~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.965 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:41.965 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.231333051, 2024-09-17T07:29:41.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047166485000000001, 2024-09-17T07:29:41.966 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.966 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.966 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.966 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.966 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.966 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.966 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44217 2024-09-17T07:29:41.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.966 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:41.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:41.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:41.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-17T07:29:41.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643291+0000", 2024-09-17T07:29:41.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.962e-06 2024-09-17T07:29:41.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643297+0000", 2024-09-17T07:29:41.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0700000000000003e-06 2024-09-17T07:29:41.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643302+0000", 2024-09-17T07:29:41.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:29:41.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690308+0000", 2024-09-17T07:29:41.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047005354999999999 2024-09-17T07:29:41.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690385+0000", 2024-09-17T07:29:41.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7541999999999997e-05 2024-09-17T07:29:41.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690455+0000", 2024-09-17T07:29:41.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0328999999999997e-05 2024-09-17T07:29:41.970 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.970 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.970 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.970 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44218 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14528512~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.970 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:41.970 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.231313373999999, 2024-09-17T07:29:41.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047321015000000001, 2024-09-17T07:29:41.970 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.970 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.970 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.971 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.971 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.971 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.971 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44218 2024-09-17T07:29:41.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.971 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:41.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:41.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:41.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-17T07:29:41.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643311+0000", 2024-09-17T07:29:41.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.933e-06 2024-09-17T07:29:41.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643318+0000", 2024-09-17T07:29:41.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3980000000000004e-06 2024-09-17T07:29:41.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643322+0000", 2024-09-17T07:29:41.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7609999999999998e-06 2024-09-17T07:29:41.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690484+0000", 2024-09-17T07:29:41.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047161428999999998 2024-09-17T07:29:41.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690564+0000", 2024-09-17T07:29:41.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9845000000000005e-05 2024-09-17T07:29:41.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690630+0000", 2024-09-17T07:29:41.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6021999999999999e-05 2024-09-17T07:29:41.975 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.975 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.975 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44219 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14544896~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.231293007000001, 2024-09-17T07:29:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048128708999999999, 2024-09-17T07:29:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.976 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44219 2024-09-17T07:29:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:29:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643337+0000", 2024-09-17T07:29:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.109e-06 2024-09-17T07:29:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643350+0000", 2024-09-17T07:29:41.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3046e-05 2024-09-17T07:29:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643356+0000", 2024-09-17T07:29:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8930000000000002e-06 2024-09-17T07:29:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690660+0000", 2024-09-17T07:29:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047304061000000001 2024-09-17T07:29:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691372+0000", 2024-09-17T07:29:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00071281299999999997 2024-09-17T07:29:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691458+0000", 2024-09-17T07:29:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5352000000000005e-05 2024-09-17T07:29:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44220 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14561280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:41.980 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:29:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.231258959000002, 2024-09-17T07:29:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048309849000000002, 2024-09-17T07:29:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44220 2024-09-17T07:29:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:41.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:41.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:29:41.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:41.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:29:41.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:41.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:41.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646869+0000", 2024-09-17T07:29:41.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003506267 2024-09-17T07:29:41.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:41.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646874+0000", 2024-09-17T07:29:41.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7500000000000003e-06 2024-09-17T07:29:41.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:41.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646887+0000", 2024-09-17T07:29:41.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3115999999999999e-05 2024-09-17T07:29:41.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:41.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646894+0000", 2024-09-17T07:29:41.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5289999999999997e-06 2024-09-17T07:29:41.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:41.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691547+0000", 2024-09-17T07:29:41.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044652743000000002 2024-09-17T07:29:41.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:41.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691622+0000", 2024-09-17T07:29:41.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5037999999999995e-05 2024-09-17T07:29:41.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:41.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:41.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:41.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691673+0000", 2024-09-17T07:29:41.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-17T07:29:41.986 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.986 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.986 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.986 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:41.986 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:41.986 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:41.987 INFO:teuthology.orchestra.run.smithi045.stdout:3470 trunc from 0xd93c054 to 0x199ea45 2024-09-17T07:29:41.987 INFO:teuthology.orchestra.run.smithi045.stdout:3472 trunc from 0x199ea45 to 0xac9a16f 2024-09-17T07:29:41.988 INFO:teuthology.orchestra.run.smithi045.stdout:3473 trunc from 0xac9a16f to 0x1524a99 2024-09-17T07:29:41.988 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:41.988 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:41.988 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:41.989 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:41.987+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:41.994 INFO:teuthology.orchestra.run.smithi045.stdout:3476 trunc from 0x1524a99 to 0x966d2b9 2024-09-17T07:29:42.004 INFO:teuthology.orchestra.run.smithi045.stdout:3477 punch from 0x10b095d to 0x10b874a, (0x7ded bytes) 2024-09-17T07:29:42.004 INFO:teuthology.orchestra.run.smithi045.stdout:3479 read 0x4251a4b thru 0x425cd55 (0xb30b bytes) 2024-09-17T07:29:42.004 INFO:teuthology.orchestra.run.smithi045.stdout:3481 read 0x66596c thru 0x666412 (0xaa7 bytes) 2024-09-17T07:29:42.010 INFO:teuthology.orchestra.run.smithi045.stdout:3482 trunc from 0x966d2b9 to 0x2a7f60a 2024-09-17T07:29:42.031 INFO:teuthology.orchestra.run.smithi045.stdout:3484 trunc from 0x2a7f60a to 0xe735b40 2024-09-17T07:29:42.038 INFO:teuthology.orchestra.run.smithi045.stdout:3486 writesame 0x2f95cf0 thru 0x2fa17e1 data_size 0x1(0xbaf2 bytes) 2024-09-17T07:29:42.053 INFO:teuthology.orchestra.run.smithi045.stdout:3487 read 0xdab86cb thru 0xdabbd80 (0x36b6 bytes) 2024-09-17T07:29:42.053 INFO:teuthology.orchestra.run.smithi045.stdout:3489 punch from 0xa942e to 0xb0575, (0x7147 bytes) 2024-09-17T07:29:42.057 INFO:teuthology.orchestra.run.smithi045.stdout:3490 trunc from 0xe735b40 to 0xa6386ad 2024-09-17T07:29:42.063 INFO:teuthology.orchestra.run.smithi045.stdout:3491 write 0x477d7a8 thru 0x4788e9a (0xb6f3 bytes) 2024-09-17T07:29:42.071 INFO:teuthology.orchestra.run.smithi045.stdout:3492 punch from 0x38d4fd6 to 0x38e23d9, (0xd403 bytes) 2024-09-17T07:29:42.071 INFO:teuthology.orchestra.run.smithi045.stdout:3493 read 0x7199d7f thru 0x71a29e0 (0x8c62 bytes) 2024-09-17T07:29:42.071 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:42.071 INFO:teuthology.orchestra.run.smithi045.stdout:3495 write 0x684bb1e thru 0x6851e73 (0x6356 bytes) 2024-09-17T07:29:42.082 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:29:42.088 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:42.088+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:42.168 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:42.169 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:42.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:42.169 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:84287 3.5 3.aa0a2a25 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T07:29:42.169 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:42.077663+0000", 2024-09-17T07:29:42.169 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.090859278000000002, 2024-09-17T07:29:42.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.090875599000000001, 2024-09-17T07:29:42.169 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:42.169 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:42.169 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "waiting for peered", 2024-09-17T07:29:42.169 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:42.169 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:42.169 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:42.170 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 84287 2024-09-17T07:29:42.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:42.170 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:42.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:42.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:42.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:42.077663+0000", 2024-09-17T07:29:42.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:42.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:42.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:42.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:42.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:42.077663+0000", 2024-09-17T07:29:42.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:42.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:42.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:42.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:42.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:42.077665+0000", 2024-09-17T07:29:42.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5600000000000001e-06 2024-09-17T07:29:42.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:42.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:42.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:42.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:42.077679+0000", 2024-09-17T07:29:42.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4412e-05 2024-09-17T07:29:42.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:42.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:42.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:42.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:42.077691+0000", 2024-09-17T07:29:42.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.254e-05 2024-09-17T07:29:42.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:42.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:42.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:42.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:42.077700+0000", 2024-09-17T07:29:42.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7420000000000003e-06 2024-09-17T07:29:42.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:42.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:42.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:42.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:42.077760+0000", 2024-09-17T07:29:42.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9982999999999998e-05 2024-09-17T07:29:42.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:42.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:42.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:29:42.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:42.077769+0000", 2024-09-17T07:29:42.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9849999999999995e-06 2024-09-17T07:29:42.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:42.173 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:42.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:42.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:42.173 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-09-17T07:29:42.173 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 1 2024-09-17T07:29:42.173 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:42.179 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:42.190 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:42.189+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:42.205 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:42.223 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:42.223 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:42.239 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:42.239 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:42.239 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:42.256 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:42.256 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:42.273 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:42.273 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:42.290 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:42.290 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:42.290 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:42.290+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:42.308 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:42.308 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:42.311 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:42.312 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:42.325 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:42.325 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:42.334 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:42.367 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T07:29:42.413 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:42.413+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:42.437 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:42.437 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:42.445 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:42.493 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:42.492+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:42.549 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:42.549 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:42.558 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:42.593 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:42.593+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:42.694 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:42.694+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:42.700 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:42.700 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:42.708 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:42.796 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:42.795+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:42.811 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:42.811 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:42.821 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:42.896 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:42.896+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:42.965 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:42.965 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:42.974 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:43.017 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:43.017+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:43.077 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-17T07:29:43.080 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:43.080 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:43.087 INFO:teuthology.orchestra.run.smithi045.stdout:3496 punch from 0x2b65df5 to 0x2b74449, (0xe654 bytes) 2024-09-17T07:29:43.087 INFO:teuthology.orchestra.run.smithi045.stdout:3497 write 0x8b0c8be thru 0x8b1b6a4 (0xede7 bytes) 2024-09-17T07:29:43.090 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:43.132 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:43.131+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:43.198 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:43.198+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:43.236 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:43.236 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:43.241 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:43.299 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:43.299+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:43.363 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:43.363 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:43.373 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:43.400 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:43.400+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:43.481 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:43.481 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:43.489 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:43.501 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:43.500+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:43.596 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:43.596 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:43.602 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:43.601+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:43.605 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:43.702 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:43.702+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:43.712 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:43.712 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:43.721 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:43.803 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:43.803+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:43.828 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:43.828 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:43.837 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:43.904 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:43.903+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:43.945 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:43.946 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:43.953 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:44.004 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:44.004+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:44.089 INFO:teuthology.orchestra.run.smithi045.stdout:3498 read 0x99dc48c thru 0x99e750f (0xb084 bytes) 2024-09-17T07:29:44.089 INFO:teuthology.orchestra.run.smithi045.stdout:3499 write 0x4818a29 thru 0x4825939 (0xcf11 bytes) 2024-09-17T07:29:44.093 INFO:teuthology.orchestra.run.smithi045.stdout:3500 read 0x78c7a95 thru 0x78c7d7d (0x2e9 bytes) 2024-09-17T07:29:44.095 INFO:teuthology.orchestra.run.smithi045.stdout:3503 trunc from 0xa6386ad to 0xbe81573 2024-09-17T07:29:44.101 INFO:teuthology.orchestra.run.smithi045.stdout:3504 write 0x71b4f08 thru 0x71c1c68 (0xcd61 bytes) 2024-09-17T07:29:44.105 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:44.105+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:44.107 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:44.107 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:44.110 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:44.110 INFO:teuthology.orchestra.run.smithi045.stdout:3506 punch from 0x7eaef09 to 0x7ebd727, (0xe81e bytes) 2024-09-17T07:29:44.110 INFO:teuthology.orchestra.run.smithi045.stdout:3507 write 0x5185338 thru 0x518ecef (0x99b8 bytes) 2024-09-17T07:29:44.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:44.120 INFO:teuthology.orchestra.run.smithi045.stdout:3508 writesame 0x8acae70 thru 0x8ad5bee data_size 0x9(0xad7f bytes) 2024-09-17T07:29:44.123 INFO:teuthology.orchestra.run.smithi045.stdout:3509 write 0xa7e78eb thru 0xa7f11e5 (0x98fb bytes) 2024-09-17T07:29:44.134 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:44.134 INFO:teuthology.orchestra.run.smithi045.stdout:3512 write 0x7d39eab thru 0x7d426e1 (0x8837 bytes) 2024-09-17T07:29:44.147 INFO:teuthology.orchestra.run.smithi045.stdout:3513 write 0x857c95e thru 0x8587b44 (0xb1e7 bytes) 2024-09-17T07:29:44.157 INFO:teuthology.orchestra.run.smithi045.stdout:3514 write 0x154aba thru 0x164236 (0xf77d bytes) 2024-09-17T07:29:44.182 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:44.182 INFO:teuthology.orchestra.run.smithi045.stdout:3516 read 0x23c3f14 thru 0x23ca8d8 (0x69c5 bytes) 2024-09-17T07:29:44.182 INFO:teuthology.orchestra.run.smithi045.stdout:3517 write 0xa4829ea thru 0xa484c9d (0x22b4 bytes) 2024-09-17T07:29:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:3518 read 0x6f635f4 thru 0x6f6426d (0xc7a bytes) 2024-09-17T07:29:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:3520 read 0x46b3554 thru 0x46c32d5 (0xfd82 bytes) 2024-09-17T07:29:44.187 INFO:teuthology.orchestra.run.smithi045.stdout:3523 write 0x55a59ae thru 0x55b0f6d (0xb5c0 bytes) 2024-09-17T07:29:44.206 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:44.206+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:44.207 INFO:teuthology.orchestra.run.smithi045.stdout:3524 trunc from 0xbe81573 to 0x521bfa6 2024-09-17T07:29:44.242 INFO:teuthology.orchestra.run.smithi045.stdout:3525 writesame 0xd491f11 thru 0xd4a050c data_size 0x52(0xe5fc bytes) 2024-09-17T07:29:44.243 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:44.243 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:44.243 INFO:teuthology.orchestra.run.smithi045.stdout:3526 punch from 0x61a8501 to 0x61afee6, (0x79e5 bytes) 2024-09-17T07:29:44.244 INFO:teuthology.orchestra.run.smithi045.stdout:3528 punch from 0x9d9a415 to 0x9d9ac90, (0x87b bytes) 2024-09-17T07:29:44.244 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:44.244 INFO:teuthology.orchestra.run.smithi045.stdout:3531 write 0xac7d003 thru 0xac7ef02 (0x1f00 bytes) 2024-09-17T07:29:44.256 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:44.257 INFO:teuthology.orchestra.run.smithi045.stdout:3533 trunc from 0xd4a050d to 0x43e2536 2024-09-17T07:29:44.267 INFO:teuthology.orchestra.run.smithi045.stdout:3534 punch from 0x24e8f32 to 0x24f61d6, (0xd2a4 bytes) 2024-09-17T07:29:44.278 INFO:teuthology.orchestra.run.smithi045.stdout:3535 writesame 0x8ae69c0 thru 0x8aeff4e data_size 0x1(0x958f bytes) 2024-09-17T07:29:44.278 INFO:teuthology.orchestra.run.smithi045.stdout:3538 write 0x8a713da thru 0x8a72bef (0x1816 bytes) 2024-09-17T07:29:44.282 INFO:teuthology.orchestra.run.smithi045.stdout:3539 clone 33 order 20 su 8192 sc 8 2024-09-17T07:29:44.307 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:44.307+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:44.313 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:29:44.337 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-17T07:29:44.337 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:29:44.337 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-17T07:29:44.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-17T07:29:44.368 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:44.368 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:44.378 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:44.419 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:44.419+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:44.440 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:44.441 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:44.460 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:44.461 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:44.477 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:44.477 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:44.477 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:44.483 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:44.483 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:44.491 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:44.495 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:44.495 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:bench [] [] [] [] 2024-09-17T07:29:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs debug_inject_read_zeros 2024-09-17T07:29:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs files list 2024-09-17T07:29:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs stats 2024-09-17T07:29:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:bluestore allocator dump block 2024-09-17T07:29:44.498 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: invalid command 2024-09-17T07:29:44.500 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:44.501 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T07:29:44.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-17T07:29:44.512 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:44.512 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:44.528 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:44.529 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:44.535 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:44.535+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:44.547 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:44.547 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:44.550 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:44.551 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:44.595 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:44.595 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:44.595 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:44.595 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:44.596 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:44.603 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:44.610 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:44.609+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:44.647 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:44.647 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:44.647 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:44.647 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:44.647 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:bench [] [] [] [] 2024-09-17T07:29:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs debug_inject_read_zeros 2024-09-17T07:29:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs files list 2024-09-17T07:29:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs stats 2024-09-17T07:29:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:bluestore allocator dump block 2024-09-17T07:29:44.648 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: invalid command 2024-09-17T07:29:44.650 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:44.650 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T07:29:44.651 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-17T07:29:44.710 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:44.710+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:44.748 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:44.748 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:44.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:44.748 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:44.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.749 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:30525 3.1b 3:def7c8f0:::rbd_data.118381ffbbab.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T07:29:44.749 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:44.749 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 31.224780509999999, 2024-09-17T07:29:44.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55515850300000003, 2024-09-17T07:29:44.749 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.749 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.749 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:44.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.749 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30525 2024-09-17T07:29:44.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.749 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:44.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:44.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522300+0000", 2024-09-17T07:29:44.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:29:44.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522305+0000", 2024-09-17T07:29:44.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-17T07:29:44.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522318+0000", 2024-09-17T07:29:44.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3030999999999999e-05 2024-09-17T07:29:44.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522326+0000", 2024-09-17T07:29:44.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4200000000000001e-06 2024-09-17T07:29:44.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522385+0000", 2024-09-17T07:29:44.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9005000000000002e-05 2024-09-17T07:29:44.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:29:44.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522393+0000", 2024-09-17T07:29:44.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-06 2024-09-17T07:29:44.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075347+0000", 2024-09-17T07:29:44.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55295417800000002 2024-09-17T07:29:44.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075502+0000", 2024-09-17T07:29:44.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154272 2024-09-17T07:29:44.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:44.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075630+0000", 2024-09-17T07:29:44.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012816300000000001 2024-09-17T07:29:44.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:44.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077221+0000", 2024-09-17T07:29:44.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015912929999999999 2024-09-17T07:29:44.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:44.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077353+0000", 2024-09-17T07:29:44.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013215699999999999 2024-09-17T07:29:44.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:44.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077413+0000", 2024-09-17T07:29:44.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0034999999999997e-05 2024-09-17T07:29:44.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077457+0000", 2024-09-17T07:29:44.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4017999999999998e-05 2024-09-17T07:29:44.755 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.755 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.755 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.756 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44181 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13922304~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.756 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:29:44.756 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.109601862000002, 2024-09-17T07:29:44.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046010502000000002, 2024-09-17T07:29:44.756 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.756 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.756 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.756 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.756 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.756 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.756 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44181 2024-09-17T07:29:44.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.757 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:29:44.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:29:44.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637478+0000", 2024-09-17T07:29:44.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:29:44.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637480+0000", 2024-09-17T07:29:44.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.843e-06 2024-09-17T07:29:44.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637487+0000", 2024-09-17T07:29:44.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6159999999999996e-06 2024-09-17T07:29:44.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637492+0000", 2024-09-17T07:29:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5360000000000003e-06 2024-09-17T07:29:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683351+0000", 2024-09-17T07:29:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045858758999999999 2024-09-17T07:29:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683422+0000", 2024-09-17T07:29:44.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1705999999999994e-05 2024-09-17T07:29:44.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683488+0000", 2024-09-17T07:29:44.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5612999999999997e-05 2024-09-17T07:29:44.760 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.760 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.760 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.760 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44182 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13938688~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.760 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:44.761 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.109581191, 2024-09-17T07:29:44.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046153118999999999, 2024-09-17T07:29:44.761 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.761 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.761 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.761 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.761 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.761 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.761 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44182 2024-09-17T07:29:44.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.761 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:44.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:44.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:44.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:29:44.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637506+0000", 2024-09-17T07:29:44.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1520000000000003e-06 2024-09-17T07:29:44.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637512+0000", 2024-09-17T07:29:44.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-09-17T07:29:44.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637517+0000", 2024-09-17T07:29:44.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-09-17T07:29:44.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683514+0000", 2024-09-17T07:29:44.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045997616999999998 2024-09-17T07:29:44.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683584+0000", 2024-09-17T07:29:44.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9942000000000002e-05 2024-09-17T07:29:44.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683651+0000", 2024-09-17T07:29:44.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6951000000000003e-05 2024-09-17T07:29:44.765 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.765 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.765 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.765 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44183 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13955072~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.765 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:29:44.765 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.109555959000001, 2024-09-17T07:29:44.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046339793999999997, 2024-09-17T07:29:44.765 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.765 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.766 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.766 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.766 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.766 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.766 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44183 2024-09-17T07:29:44.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.766 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:29:44.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:29:44.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639583+0000", 2024-09-17T07:29:44.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002059412 2024-09-17T07:29:44.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639597+0000", 2024-09-17T07:29:44.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3832000000000001e-05 2024-09-17T07:29:44.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639607+0000", 2024-09-17T07:29:44.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485e-05 2024-09-17T07:29:44.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639614+0000", 2024-09-17T07:29:44.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0849999999999999e-06 2024-09-17T07:29:44.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683705+0000", 2024-09-17T07:29:44.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044091261 2024-09-17T07:29:44.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683792+0000", 2024-09-17T07:29:44.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6197000000000002e-05 2024-09-17T07:29:44.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683863+0000", 2024-09-17T07:29:44.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1521999999999997e-05 2024-09-17T07:29:44.770 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.770 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.770 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.770 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44199 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14217216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.770 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:44.770 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.106038407, 2024-09-17T07:29:44.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045995623999999999, 2024-09-17T07:29:44.770 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.770 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.771 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.771 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.771 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.771 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.771 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44199 2024-09-17T07:29:44.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.771 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:44.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:44.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:44.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:29:44.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641043+0000", 2024-09-17T07:29:44.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.624e-06 2024-09-17T07:29:44.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641048+0000", 2024-09-17T07:29:44.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4029999999999998e-06 2024-09-17T07:29:44.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641053+0000", 2024-09-17T07:29:44.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6670000000000004e-06 2024-09-17T07:29:44.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686918+0000", 2024-09-17T07:29:44.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045865131000000003 2024-09-17T07:29:44.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686986+0000", 2024-09-17T07:29:44.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8299000000000005e-05 2024-09-17T07:29:44.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687036+0000", 2024-09-17T07:29:44.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0164999999999998e-05 2024-09-17T07:29:44.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.774 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.775 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44200 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14233600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.775 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:44.775 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.106020577999999, 2024-09-17T07:29:44.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046183984999999997, 2024-09-17T07:29:44.775 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.775 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.775 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.775 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44200 2024-09-17T07:29:44.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.776 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:44.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:44.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:44.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:44.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641061+0000", 2024-09-17T07:29:44.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.767e-06 2024-09-17T07:29:44.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641067+0000", 2024-09-17T07:29:44.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7379999999999999e-06 2024-09-17T07:29:44.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641071+0000", 2024-09-17T07:29:44.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-09-17T07:29:44.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687078+0000", 2024-09-17T07:29:44.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046007095999999997 2024-09-17T07:29:44.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687158+0000", 2024-09-17T07:29:44.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0374e-05 2024-09-17T07:29:44.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687243+0000", 2024-09-17T07:29:44.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4221999999999994e-05 2024-09-17T07:29:44.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.779 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.779 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44201 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14249984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.779 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:29:44.779 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.106002931999999, 2024-09-17T07:29:44.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046303182999999998, 2024-09-17T07:29:44.779 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.780 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.780 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.780 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44201 2024-09-17T07:29:44.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.780 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:29:44.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:29:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641120+0000", 2024-09-17T07:29:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3933000000000001e-05 2024-09-17T07:29:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641123+0000", 2024-09-17T07:29:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.191e-06 2024-09-17T07:29:44.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641133+0000", 2024-09-17T07:29:44.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4020000000000008e-06 2024-09-17T07:29:44.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641138+0000", 2024-09-17T07:29:44.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6559999999999997e-06 2024-09-17T07:29:44.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687279+0000", 2024-09-17T07:29:44.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046140278999999999 2024-09-17T07:29:44.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687338+0000", 2024-09-17T07:29:44.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9268999999999998e-05 2024-09-17T07:29:44.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687379+0000", 2024-09-17T07:29:44.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1452999999999998e-05 2024-09-17T07:29:44.784 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.784 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.784 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.784 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44202 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14266368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.784 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:29:44.784 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.105932165999999, 2024-09-17T07:29:44.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046484299, 2024-09-17T07:29:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44202 2024-09-17T07:29:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:29:44.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:29:44.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642810+0000", 2024-09-17T07:29:44.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016627980000000001 2024-09-17T07:29:44.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642815+0000", 2024-09-17T07:29:44.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.091e-06 2024-09-17T07:29:44.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642829+0000", 2024-09-17T07:29:44.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4047e-05 2024-09-17T07:29:44.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642844+0000", 2024-09-17T07:29:44.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4515000000000001e-05 2024-09-17T07:29:44.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687426+0000", 2024-09-17T07:29:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044582201000000002 2024-09-17T07:29:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687529+0000", 2024-09-17T07:29:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103078 2024-09-17T07:29:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687631+0000", 2024-09-17T07:29:44.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102569 2024-09-17T07:29:44.789 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.789 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.789 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.789 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44209 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14381056~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.789 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:29:44.789 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.104034014, 2024-09-17T07:29:44.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045944444000000001, 2024-09-17T07:29:44.789 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.789 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.789 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.790 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.790 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.790 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44209 2024-09-17T07:29:44.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.790 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:29:44.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:29:44.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643046+0000", 2024-09-17T07:29:44.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:29:44.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643048+0000", 2024-09-17T07:29:44.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:29:44.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643054+0000", 2024-09-17T07:29:44.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.032e-06 2024-09-17T07:29:44.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643065+0000", 2024-09-17T07:29:44.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1861999999999999e-05 2024-09-17T07:29:44.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688806+0000", 2024-09-17T07:29:44.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045740617999999997 2024-09-17T07:29:44.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688927+0000", 2024-09-17T07:29:44.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000121327 2024-09-17T07:29:44.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688990+0000", 2024-09-17T07:29:44.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2293999999999997e-05 2024-09-17T07:29:44.793 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.793 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.793 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.794 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44210 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14397440~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.794 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:29:44.794 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.104007087999999, 2024-09-17T07:29:44.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046104635999999997, 2024-09-17T07:29:44.794 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.794 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.794 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.794 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.795 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.795 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.795 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44210 2024-09-17T07:29:44.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.795 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:29:44.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:29:44.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643073+0000", 2024-09-17T07:29:44.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.77e-07 2024-09-17T07:29:44.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643075+0000", 2024-09-17T07:29:44.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-09-17T07:29:44.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643082+0000", 2024-09-17T07:29:44.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7789999999999996e-06 2024-09-17T07:29:44.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643087+0000", 2024-09-17T07:29:44.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.967e-06 2024-09-17T07:29:44.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689022+0000", 2024-09-17T07:29:44.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045934349999999999 2024-09-17T07:29:44.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689112+0000", 2024-09-17T07:29:44.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0039999999999999e-05 2024-09-17T07:29:44.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689177+0000", 2024-09-17T07:29:44.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5041000000000003e-05 2024-09-17T07:29:44.798 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.798 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.799 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.799 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44211 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14413824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.799 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:29:44.799 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.103985954999999, 2024-09-17T07:29:44.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046309003000000001, 2024-09-17T07:29:44.799 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.799 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.799 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.799 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.799 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.799 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.800 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44211 2024-09-17T07:29:44.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.800 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:29:44.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:29:44.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643094+0000", 2024-09-17T07:29:44.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:29:44.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643095+0000", 2024-09-17T07:29:44.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-17T07:29:44.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643101+0000", 2024-09-17T07:29:44.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8730000000000004e-06 2024-09-17T07:29:44.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643106+0000", 2024-09-17T07:29:44.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6859999999999997e-06 2024-09-17T07:29:44.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689220+0000", 2024-09-17T07:29:44.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046113945000000003 2024-09-17T07:29:44.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689309+0000", 2024-09-17T07:29:44.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9560000000000003e-05 2024-09-17T07:29:44.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689402+0000", 2024-09-17T07:29:44.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2786000000000002e-05 2024-09-17T07:29:44.803 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.803 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.803 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.804 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44212 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14430208~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.804 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:44.804 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.103941392999999, 2024-09-17T07:29:44.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046448912000000002, 2024-09-17T07:29:44.804 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.804 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.804 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.804 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44212 2024-09-17T07:29:44.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.805 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:44.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:44.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:44.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2200000000000004e-07 2024-09-17T07:29:44.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643141+0000", 2024-09-17T07:29:44.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5210000000000001e-06 2024-09-17T07:29:44.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643149+0000", 2024-09-17T07:29:44.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6490000000000008e-06 2024-09-17T07:29:44.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643161+0000", 2024-09-17T07:29:44.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2241e-05 2024-09-17T07:29:44.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689423+0000", 2024-09-17T07:29:44.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046262323000000001 2024-09-17T07:29:44.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689505+0000", 2024-09-17T07:29:44.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2103999999999999e-05 2024-09-17T07:29:44.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689587+0000", 2024-09-17T07:29:44.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1452000000000006e-05 2024-09-17T07:29:44.808 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.808 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.809 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.809 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44213 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14446592~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.809 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:29:44.809 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.103909894000001, 2024-09-17T07:29:44.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046601780000000002, 2024-09-17T07:29:44.809 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.809 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.809 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.809 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.809 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.809 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.810 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44213 2024-09-17T07:29:44.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.810 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:29:44.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:29:44.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643170+0000", 2024-09-17T07:29:44.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-17T07:29:44.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643172+0000", 2024-09-17T07:29:44.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1660000000000001e-06 2024-09-17T07:29:44.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643180+0000", 2024-09-17T07:29:44.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-09-17T07:29:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643185+0000", 2024-09-17T07:29:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-09-17T07:29:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689622+0000", 2024-09-17T07:29:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046436742000000003 2024-09-17T07:29:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689697+0000", 2024-09-17T07:29:44.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5179999999999995e-05 2024-09-17T07:29:44.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689771+0000", 2024-09-17T07:29:44.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3928000000000001e-05 2024-09-17T07:29:44.813 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.813 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.813 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.813 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44214 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14462976~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.813 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:29:44.814 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.103886834000001, 2024-09-17T07:29:44.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046736038000000001, 2024-09-17T07:29:44.814 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.814 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.814 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.814 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.814 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.814 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.814 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44214 2024-09-17T07:29:44.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.814 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:29:44.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:29:44.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643193+0000", 2024-09-17T07:29:44.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:29:44.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643195+0000", 2024-09-17T07:29:44.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0640000000000001e-06 2024-09-17T07:29:44.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643201+0000", 2024-09-17T07:29:44.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2859999999999996e-06 2024-09-17T07:29:44.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643206+0000", 2024-09-17T07:29:44.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-17T07:29:44.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689814+0000", 2024-09-17T07:29:44.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046607742000000001 2024-09-17T07:29:44.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689886+0000", 2024-09-17T07:29:44.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2020999999999994e-05 2024-09-17T07:29:44.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689928+0000", 2024-09-17T07:29:44.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2771000000000001e-05 2024-09-17T07:29:44.818 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.818 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.818 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.818 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44215 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14479360~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.818 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:29:44.818 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.103842937, 2024-09-17T07:29:44.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046868543999999998, 2024-09-17T07:29:44.818 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.818 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.819 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.819 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.819 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.819 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.819 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44215 2024-09-17T07:29:44.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.819 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:29:44.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:29:44.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643237+0000", 2024-09-17T07:29:44.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:29:44.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643245+0000", 2024-09-17T07:29:44.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3270000000000001e-06 2024-09-17T07:29:44.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643254+0000", 2024-09-17T07:29:44.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6049999999999998e-06 2024-09-17T07:29:44.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643260+0000", 2024-09-17T07:29:44.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7849999999999997e-06 2024-09-17T07:29:44.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689961+0000", 2024-09-17T07:29:44.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046701196 2024-09-17T07:29:44.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690036+0000", 2024-09-17T07:29:44.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5153000000000004e-05 2024-09-17T07:29:44.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690105+0000", 2024-09-17T07:29:44.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8874000000000005e-05 2024-09-17T07:29:44.822 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.823 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.823 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.823 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44216 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14495744~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.823 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:29:44.823 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.103811224000001, 2024-09-17T07:29:44.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047003047999999999, 2024-09-17T07:29:44.823 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.823 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.823 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.824 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44216 2024-09-17T07:29:44.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.824 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:29:44.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:29:44.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643269+0000", 2024-09-17T07:29:44.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:29:44.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643271+0000", 2024-09-17T07:29:44.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-09-17T07:29:44.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643278+0000", 2024-09-17T07:29:44.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8850000000000001e-06 2024-09-17T07:29:44.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643282+0000", 2024-09-17T07:29:44.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5979999999999999e-06 2024-09-17T07:29:44.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690133+0000", 2024-09-17T07:29:44.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850470999999998 2024-09-17T07:29:44.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690205+0000", 2024-09-17T07:29:44.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2442999999999997e-05 2024-09-17T07:29:44.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690271+0000", 2024-09-17T07:29:44.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5874e-05 2024-09-17T07:29:44.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.827 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.827 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44217 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14512128~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.827 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:44.828 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.103790212, 2024-09-17T07:29:44.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047166485000000001, 2024-09-17T07:29:44.828 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.828 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.828 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.828 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44217 2024-09-17T07:29:44.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.828 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:44.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:44.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:44.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-17T07:29:44.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643291+0000", 2024-09-17T07:29:44.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.962e-06 2024-09-17T07:29:44.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643297+0000", 2024-09-17T07:29:44.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0700000000000003e-06 2024-09-17T07:29:44.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643302+0000", 2024-09-17T07:29:44.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:29:44.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690308+0000", 2024-09-17T07:29:44.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047005354999999999 2024-09-17T07:29:44.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690385+0000", 2024-09-17T07:29:44.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7541999999999997e-05 2024-09-17T07:29:44.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690455+0000", 2024-09-17T07:29:44.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0328999999999997e-05 2024-09-17T07:29:44.832 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.832 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.832 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.832 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44218 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14528512~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.832 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:44.832 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.103770534999999, 2024-09-17T07:29:44.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047321015000000001, 2024-09-17T07:29:44.832 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.832 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.832 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.833 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.833 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.833 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.833 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44218 2024-09-17T07:29:44.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.833 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:44.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:44.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:44.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-17T07:29:44.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643311+0000", 2024-09-17T07:29:44.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.933e-06 2024-09-17T07:29:44.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643318+0000", 2024-09-17T07:29:44.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3980000000000004e-06 2024-09-17T07:29:44.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643322+0000", 2024-09-17T07:29:44.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7609999999999998e-06 2024-09-17T07:29:44.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690484+0000", 2024-09-17T07:29:44.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047161428999999998 2024-09-17T07:29:44.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690564+0000", 2024-09-17T07:29:44.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9845000000000005e-05 2024-09-17T07:29:44.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690630+0000", 2024-09-17T07:29:44.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6021999999999999e-05 2024-09-17T07:29:44.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.836 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.837 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.837 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44219 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14544896~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.837 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:44.837 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.103750168000001, 2024-09-17T07:29:44.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048128708999999999, 2024-09-17T07:29:44.837 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.837 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.837 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.838 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44219 2024-09-17T07:29:44.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.838 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:44.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:44.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:44.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:29:44.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643337+0000", 2024-09-17T07:29:44.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.109e-06 2024-09-17T07:29:44.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643350+0000", 2024-09-17T07:29:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3046e-05 2024-09-17T07:29:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643356+0000", 2024-09-17T07:29:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8930000000000002e-06 2024-09-17T07:29:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690660+0000", 2024-09-17T07:29:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047304061000000001 2024-09-17T07:29:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691372+0000", 2024-09-17T07:29:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00071281299999999997 2024-09-17T07:29:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691458+0000", 2024-09-17T07:29:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5352000000000005e-05 2024-09-17T07:29:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.841 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44220 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14561280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:44.842 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:29:44.842 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.103716120000001, 2024-09-17T07:29:44.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048309849000000002, 2024-09-17T07:29:44.842 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:44.842 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:44.842 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:44.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:44.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:44.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:44.842 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44220 2024-09-17T07:29:44.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.842 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:44.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:44.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:29:44.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:44.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:29:44.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:44.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:44.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646869+0000", 2024-09-17T07:29:44.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003506267 2024-09-17T07:29:44.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:44.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646874+0000", 2024-09-17T07:29:44.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7500000000000003e-06 2024-09-17T07:29:44.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:44.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646887+0000", 2024-09-17T07:29:44.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3115999999999999e-05 2024-09-17T07:29:44.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:44.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646894+0000", 2024-09-17T07:29:44.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5289999999999997e-06 2024-09-17T07:29:44.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:44.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691547+0000", 2024-09-17T07:29:44.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044652743000000002 2024-09-17T07:29:44.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:44.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691622+0000", 2024-09-17T07:29:44.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5037999999999995e-05 2024-09-17T07:29:44.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:44.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:44.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:44.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691673+0000", 2024-09-17T07:29:44.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-17T07:29:44.846 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.846 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.846 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.846 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:44.846 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:44.846 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:44.847 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:44.848 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:44.849 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:44.847+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:44.849 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:44.849 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:44.849 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:44.849 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:44.849 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:44.849 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:44.849 INFO:teuthology.orchestra.run.smithi045.stderr:bench [] [] [] [] 2024-09-17T07:29:44.850 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs debug_inject_read_zeros 2024-09-17T07:29:44.850 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs files list 2024-09-17T07:29:44.850 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs stats 2024-09-17T07:29:44.850 INFO:teuthology.orchestra.run.smithi045.stderr:bluestore allocator dump block 2024-09-17T07:29:44.850 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: invalid command 2024-09-17T07:29:44.850 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T07:29:44.850 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-17T07:29:44.948 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:44.948+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:44.985 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:bench [] [] [] [] 2024-09-17T07:29:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs debug_inject_read_zeros 2024-09-17T07:29:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs files list 2024-09-17T07:29:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs stats 2024-09-17T07:29:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:bluestore allocator dump block 2024-09-17T07:29:44.986 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: invalid command 2024-09-17T07:29:44.988 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:44.988 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T07:29:44.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-17T07:29:44.993 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:44.993 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:44.993 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:44.993 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:45.001 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:45.049 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:45.049+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:45.098 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T07:29:45.108 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:45.108 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:45.108 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:45.108 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:45.108 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:45.118 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:45.135 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone32 from 0x8aeff4f (overlap 0x1524a99) to 0xa9254c 2024-09-17T07:29:45.160 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:45.159+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:45.216 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:bench [] [] [] [] 2024-09-17T07:29:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs debug_inject_read_zeros 2024-09-17T07:29:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs files list 2024-09-17T07:29:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs stats 2024-09-17T07:29:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:bluestore allocator dump block 2024-09-17T07:29:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: invalid command 2024-09-17T07:29:45.220 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:45.220 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T07:29:45.221 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-17T07:29:45.251 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:45.251+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:45.273 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-17T07:29:45.286 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.286 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19985 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5275648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.032573382999999, 2024-09-17T07:29:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071819998999999995, 2024-09-17T07:29:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19985 2024-09-17T07:29:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:45.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252493+0000", 2024-09-17T07:29:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6300000000000004e-07 2024-09-17T07:29:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252496+0000", 2024-09-17T07:29:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-17T07:29:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252499+0000", 2024-09-17T07:29:45.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:29:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324036+0000", 2024-09-17T07:29:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071536587999999998 2024-09-17T07:29:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324118+0000", 2024-09-17T07:29:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2336999999999995e-05 2024-09-17T07:29:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324312+0000", 2024-09-17T07:29:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000193709 2024-09-17T07:29:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19986 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5505024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.032562231, 2024-09-17T07:29:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072083761999999996, 2024-09-17T07:29:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.291 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19986 2024-09-17T07:29:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:29:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252508+0000", 2024-09-17T07:29:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:29:45.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252511+0000", 2024-09-17T07:29:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.145e-06 2024-09-17T07:29:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252514+0000", 2024-09-17T07:29:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5569999999999998e-06 2024-09-17T07:29:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324326+0000", 2024-09-17T07:29:45.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071812681000000003 2024-09-17T07:29:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324372+0000", 2024-09-17T07:29:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5893000000000003e-05 2024-09-17T07:29:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324587+0000", 2024-09-17T07:29:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214608 2024-09-17T07:29:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19987 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5734400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.032548613000003, 2024-09-17T07:29:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072276676999999998, 2024-09-17T07:29:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19987 2024-09-17T07:29:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.296 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252518+0000", 2024-09-17T07:29:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-17T07:29:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252522+0000", 2024-09-17T07:29:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2440000000000004e-06 2024-09-17T07:29:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252525+0000", 2024-09-17T07:29:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5919999999999999e-06 2024-09-17T07:29:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324621+0000", 2024-09-17T07:29:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072096442999999996 2024-09-17T07:29:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324699+0000", 2024-09-17T07:29:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8061999999999999e-05 2024-09-17T07:29:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324793+0000", 2024-09-17T07:29:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4207000000000001e-05 2024-09-17T07:29:45.302 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19988 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.032537642000001, 2024-09-17T07:29:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072585798000000007, 2024-09-17T07:29:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19988 2024-09-17T07:29:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252529+0000", 2024-09-17T07:29:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:29:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252532+0000", 2024-09-17T07:29:45.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9579999999999999e-06 2024-09-17T07:29:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252534+0000", 2024-09-17T07:29:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5730000000000002e-06 2024-09-17T07:29:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324825+0000", 2024-09-17T07:29:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072290757999999997 2024-09-17T07:29:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324874+0000", 2024-09-17T07:29:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9360000000000002e-05 2024-09-17T07:29:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325113+0000", 2024-09-17T07:29:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000239025 2024-09-17T07:29:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19989 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.032528020999997, 2024-09-17T07:29:45.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072963494000000004, 2024-09-17T07:29:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19989 2024-09-17T07:29:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:45.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:29:45.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:29:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:29:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2900000000000002e-07 2024-09-17T07:29:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252542+0000", 2024-09-17T07:29:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8600000000000003e-06 2024-09-17T07:29:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252545+0000", 2024-09-17T07:29:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.605e-06 2024-09-17T07:29:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325144+0000", 2024-09-17T07:29:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072599460000000005 2024-09-17T07:29:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325223+0000", 2024-09-17T07:29:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8473999999999994e-05 2024-09-17T07:29:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325501+0000", 2024-09-17T07:29:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000278063 2024-09-17T07:29:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19990 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6422528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.032517550000001, 2024-09-17T07:29:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073241127000000003, 2024-09-17T07:29:45.312 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19990 2024-09-17T07:29:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252552+0000", 2024-09-17T07:29:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4719999999999997e-06 2024-09-17T07:29:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252557+0000", 2024-09-17T07:29:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3039999999999998e-06 2024-09-17T07:29:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252560+0000", 2024-09-17T07:29:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8830000000000002e-06 2024-09-17T07:29:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325529+0000", 2024-09-17T07:29:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072968880999999999 2024-09-17T07:29:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325574+0000", 2024-09-17T07:29:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466000000000001e-05 2024-09-17T07:29:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325789+0000", 2024-09-17T07:29:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021490999999999999 2024-09-17T07:29:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19991 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6651904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.032498248000003, 2024-09-17T07:29:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073720552999999994, 2024-09-17T07:29:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.317 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19991 2024-09-17T07:29:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4699999999999998e-07 2024-09-17T07:29:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252568+0000", 2024-09-17T07:29:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5300000000000002e-07 2024-09-17T07:29:45.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252572+0000", 2024-09-17T07:29:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9160000000000003e-06 2024-09-17T07:29:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252575+0000", 2024-09-17T07:29:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:29:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325820+0000", 2024-09-17T07:29:45.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073245597999999995 2024-09-17T07:29:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325901+0000", 2024-09-17T07:29:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0913000000000003e-05 2024-09-17T07:29:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326288+0000", 2024-09-17T07:29:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038646900000000002 2024-09-17T07:29:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19992 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6881280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.032487895000003, 2024-09-17T07:29:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074054683999999996, 2024-09-17T07:29:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19992 2024-09-17T07:29:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.322 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252578+0000", 2024-09-17T07:29:45.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:29:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252579+0000", 2024-09-17T07:29:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2500000000000004e-07 2024-09-17T07:29:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252582+0000", 2024-09-17T07:29:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-17T07:29:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252585+0000", 2024-09-17T07:29:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.542e-06 2024-09-17T07:29:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326315+0000", 2024-09-17T07:29:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073729921000000004 2024-09-17T07:29:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326359+0000", 2024-09-17T07:29:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4095999999999999e-05 2024-09-17T07:29:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326632+0000", 2024-09-17T07:29:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027314499999999997 2024-09-17T07:29:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19993 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7110656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.032477591000003, 2024-09-17T07:29:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074425142999999999, 2024-09-17T07:29:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19993 2024-09-17T07:29:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:45.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:29:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252589+0000", 2024-09-17T07:29:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.33e-07 2024-09-17T07:29:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252593+0000", 2024-09-17T07:29:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:29:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252595+0000", 2024-09-17T07:29:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.621e-06 2024-09-17T07:29:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326666+0000", 2024-09-17T07:29:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074070787999999999 2024-09-17T07:29:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326752+0000", 2024-09-17T07:29:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5748000000000006e-05 2024-09-17T07:29:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327013+0000", 2024-09-17T07:29:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026098400000000002 2024-09-17T07:29:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19994 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7340032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.330 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.032467132000001, 2024-09-17T07:29:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074765920999999999, 2024-09-17T07:29:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19994 2024-09-17T07:29:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:29:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252599+0000", 2024-09-17T07:29:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2099999999999995e-07 2024-09-17T07:29:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252603+0000", 2024-09-17T07:29:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:29:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252606+0000", 2024-09-17T07:29:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-09-17T07:29:45.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327047+0000", 2024-09-17T07:29:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074441050999999994 2024-09-17T07:29:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327121+0000", 2024-09-17T07:29:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4283999999999996e-05 2024-09-17T07:29:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327364+0000", 2024-09-17T07:29:45.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000243155 2024-09-17T07:29:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19995 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7569408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.032456435, 2024-09-17T07:29:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075172694999999998, 2024-09-17T07:29:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19995 2024-09-17T07:29:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252610+0000", 2024-09-17T07:29:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6799999999999999e-07 2024-09-17T07:29:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252614+0000", 2024-09-17T07:29:45.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000002e-06 2024-09-17T07:29:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252616+0000", 2024-09-17T07:29:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.419e-06 2024-09-17T07:29:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327407+0000", 2024-09-17T07:29:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074790617000000004 2024-09-17T07:29:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327498+0000", 2024-09-17T07:29:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0758000000000001e-05 2024-09-17T07:29:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327782+0000", 2024-09-17T07:29:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028395599999999998 2024-09-17T07:29:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7798784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:45.339 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.032446118999999, 2024-09-17T07:29:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075605832999999997, 2024-09-17T07:29:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:29:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:45.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:29:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:29:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:29:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:29:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:29:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:29:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327844+0000", 2024-09-17T07:29:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075217709999999993 2024-09-17T07:29:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327927+0000", 2024-09-17T07:29:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2628999999999998e-05 2024-09-17T07:29:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328225+0000", 2024-09-17T07:29:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029800600000000002 2024-09-17T07:29:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.343 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8028160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.032435765999999, 2024-09-17T07:29:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076061237000000004, 2024-09-17T07:29:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:29:45.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:45.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:29:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:29:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:29:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:29:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:29:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:29:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328268+0000", 2024-09-17T07:29:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075626323999999995 2024-09-17T07:29:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328352+0000", 2024-09-17T07:29:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3578999999999994e-05 2024-09-17T07:29:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328691+0000", 2024-09-17T07:29:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033915099999999999 2024-09-17T07:29:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8257536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.032420315000003, 2024-09-17T07:29:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076441903000000005, 2024-09-17T07:29:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.348 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:29:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:29:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:29:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:29:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:29:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:29:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:29:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:29:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328747+0000", 2024-09-17T07:29:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076094192000000005 2024-09-17T07:29:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328824+0000", 2024-09-17T07:29:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7096999999999997e-05 2024-09-17T07:29:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.329087+0000", 2024-09-17T07:29:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026249999999999998 2024-09-17T07:29:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.353 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20029 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.022335327, 2024-09-17T07:29:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071783696999999994, 2024-09-17T07:29:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20029 2024-09-17T07:29:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262735+0000", 2024-09-17T07:29:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.955e-06 2024-09-17T07:29:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262737+0000", 2024-09-17T07:29:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0090000000000002e-06 2024-09-17T07:29:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262743+0000", 2024-09-17T07:29:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2890000000000003e-06 2024-09-17T07:29:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262748+0000", 2024-09-17T07:29:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334163+0000", 2024-09-17T07:29:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071415336999999995 2024-09-17T07:29:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334243+0000", 2024-09-17T07:29:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9548000000000005e-05 2024-09-17T07:29:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334514+0000", 2024-09-17T07:29:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027086800000000001 2024-09-17T07:29:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7356416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.022304155999997, 2024-09-17T07:29:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072172616999999994, 2024-09-17T07:29:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:29:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:29:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:29:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:29:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:29:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:29:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:29:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:29:45.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:29:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334558+0000", 2024-09-17T07:29:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071780790999999997 2024-09-17T07:29:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334647+0000", 2024-09-17T07:29:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9448000000000001e-05 2024-09-17T07:29:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334934+0000", 2024-09-17T07:29:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028654599999999999 2024-09-17T07:29:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7585792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:45.363 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.022282443999998, 2024-09-17T07:29:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072606135000000002, 2024-09-17T07:29:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:29:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:29:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:29:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:29:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:29:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:29:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:29:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:29:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335116+0000", 2024-09-17T07:29:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072323897999999998 2024-09-17T07:29:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335166+0000", 2024-09-17T07:29:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9882000000000001e-05 2024-09-17T07:29:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335389+0000", 2024-09-17T07:29:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022334300000000001 2024-09-17T07:29:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.367 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7815168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.022270245000001, 2024-09-17T07:29:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073051230999999994, 2024-09-17T07:29:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:29:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:29:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:29:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:29:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:29:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:29:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:29:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335431+0000", 2024-09-17T07:29:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072627852000000007 2024-09-17T07:29:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335525+0000", 2024-09-17T07:29:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3708000000000005e-05 2024-09-17T07:29:45.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335846+0000", 2024-09-17T07:29:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032182000000000001 2024-09-17T07:29:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.022259386999998, 2024-09-17T07:29:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073284199999999994, 2024-09-17T07:29:45.372 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:29:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:29:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:29:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:29:45.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:29:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:29:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:29:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:29:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335863+0000", 2024-09-17T07:29:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073048921000000003 2024-09-17T07:29:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335904+0000", 2024-09-17T07:29:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1545000000000003e-05 2024-09-17T07:29:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336090+0000", 2024-09-17T07:29:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185787 2024-09-17T07:29:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8273920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.022245126999998, 2024-09-17T07:29:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073670478999999997, 2024-09-17T07:29:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:29:45.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:29:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:29:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:29:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:29:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:29:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:29:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:29:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336126+0000", 2024-09-17T07:29:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073297254000000006 2024-09-17T07:29:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336211+0000", 2024-09-17T07:29:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4728000000000006e-05 2024-09-17T07:29:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336491+0000", 2024-09-17T07:29:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027979399999999998 2024-09-17T07:29:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.381 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.381 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:45.383 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:45.383 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:45.382+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:bench [] [] [] [] 2024-09-17T07:29:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs debug_inject_read_zeros 2024-09-17T07:29:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs files list 2024-09-17T07:29:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs stats 2024-09-17T07:29:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:bluestore allocator dump block 2024-09-17T07:29:45.430 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: invalid command 2024-09-17T07:29:45.433 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:45.433 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T07:29:45.434 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-17T07:29:45.483 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:45.483+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:45.527 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:45.527 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:45.535 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:45.541 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:45.541 INFO:teuthology.orchestra.run.smithi045.stdout:3543 punch from 0x5f34a99 to 0x5f3c778, (0x7cdf bytes) 2024-09-17T07:29:45.545 INFO:teuthology.orchestra.run.smithi045.stdout:3544 write 0x313785c thru 0x3146e41 (0xf5e6 bytes) 2024-09-17T07:29:45.563 INFO:teuthology.orchestra.run.smithi045.stdout:3545 write 0x8f1b962 thru 0x8f1d86a (0x1f09 bytes) 2024-09-17T07:29:45.569 INFO:teuthology.orchestra.run.smithi045.stdout:3546 read 0x23fe16a thru 0x240b7e1 (0xd678 bytes) 2024-09-17T07:29:45.569 INFO:teuthology.orchestra.run.smithi045.stdout:3547 write 0x5dc66a0 thru 0x5dc6f3a (0x89b bytes) 2024-09-17T07:29:45.572 INFO:teuthology.orchestra.run.smithi045.stdout:3548 writesame 0x55a3b11 thru 0x55b285a data_size 0x7(0xed4a bytes) 2024-09-17T07:29:45.584 INFO:teuthology.orchestra.run.smithi045.stdout:3549 read 0x407fe37 thru 0x4085d2c (0x5ef6 bytes) 2024-09-17T07:29:45.584 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:45.596 INFO:teuthology.orchestra.run.smithi045.stdout:3552 write 0xecc5f92 thru 0xecce2eb (0x835a bytes) 2024-09-17T07:29:45.605 INFO:teuthology.orchestra.run.smithi045.stdout:3553 write 0x7cbff7f thru 0x7cc0d2c (0xdae bytes) 2024-09-17T07:29:45.609 INFO:teuthology.orchestra.run.smithi045.stdout:3554 writesame 0xc438114 thru 0xc440456 data_size 0x17(0x8343 bytes) 2024-09-17T07:29:45.616 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:45.616 INFO:teuthology.orchestra.run.smithi045.stdout:3556 write 0x8b672fe thru 0x8b6a6cc (0x33cf bytes) 2024-09-17T07:29:45.621 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:45.621+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:45.622 INFO:teuthology.orchestra.run.smithi045.stdout:3557 punch from 0x8d7c940 to 0x8d82e47, (0x6507 bytes) 2024-09-17T07:29:45.622 INFO:teuthology.orchestra.run.smithi045.stdout:3559 write 0x42f0bf6 thru 0x42fc5a1 (0xb9ac bytes) 2024-09-17T07:29:45.632 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:45.632 INFO:teuthology.orchestra.run.smithi045.stdout:3562 read 0x3486c81 thru 0x348995e (0x2cde bytes) 2024-09-17T07:29:45.632 INFO:teuthology.orchestra.run.smithi045.stdout:3563 punch from 0x7f3bd4e to 0x7f4b0a0, (0xf352 bytes) 2024-09-17T07:29:45.632 INFO:teuthology.orchestra.run.smithi045.stdout:3565 write 0x8f51a8f thru 0x8f5c5a2 (0xab14 bytes) 2024-09-17T07:29:45.640 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:45.641 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:45.641 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:45.641 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:45.641 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:45.641 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:45.641 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:45.641 INFO:teuthology.orchestra.run.smithi045.stderr:bench [] [] [] [] 2024-09-17T07:29:45.642 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs debug_inject_read_zeros 2024-09-17T07:29:45.642 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs files list 2024-09-17T07:29:45.642 INFO:teuthology.orchestra.run.smithi045.stderr:bluefs stats 2024-09-17T07:29:45.642 INFO:teuthology.orchestra.run.smithi045.stderr:bluestore allocator dump block 2024-09-17T07:29:45.642 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINVAL: invalid command 2024-09-17T07:29:45.642 INFO:teuthology.orchestra.run.smithi045.stdout:3568 trunc from 0xecce2ec to 0xd480519 2024-09-17T07:29:45.644 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:45.644 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T07:29:45.655 INFO:teuthology.orchestra.run.smithi045.stdout:3569 trunc from 0xd480519 to 0xae285da 2024-09-17T07:29:45.665 INFO:teuthology.orchestra.run.smithi045.stdout:3571 punch from 0x370b15f to 0x3714a56, (0x98f7 bytes) 2024-09-17T07:29:45.665 INFO:teuthology.orchestra.run.smithi045.stdout:3572 punch from 0x6e691f3 to 0x6e703dd, (0x71ea bytes) 2024-09-17T07:29:45.665 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:45.665 INFO:teuthology.orchestra.run.smithi045.stdout:3575 punch from 0x75c0c7f to 0x75c297b, (0x1cfc bytes) 2024-09-17T07:29:45.665 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:45.665 INFO:teuthology.orchestra.run.smithi045.stdout:3577 read 0xcc4cd6 thru 0xcd2c6e (0xdf99 bytes) 2024-09-17T07:29:45.665 INFO:teuthology.orchestra.run.smithi045.stdout:3578 read 0x30e854 thru 0x318402 (0x9baf bytes) 2024-09-17T07:29:45.673 INFO:teuthology.orchestra.run.smithi045.stdout:3579 write 0xc8c96d6 thru 0xc8d60ed (0xca18 bytes) 2024-09-17T07:29:45.682 INFO:teuthology.orchestra.run.smithi045.stdout:3580 write 0x83c79b4 thru 0x83d61f3 (0xe840 bytes) 2024-09-17T07:29:45.685 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:45.684+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:45.690 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:45.690 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:45.690 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:45.690 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:45.691 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:45.691 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:45.698 INFO:teuthology.orchestra.run.smithi045.stdout:3583 write 0xdd3ef95 thru 0xdd46e55 (0x7ec1 bytes) 2024-09-17T07:29:45.699 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:45.705 INFO:teuthology.orchestra.run.smithi045.stdout:3584 read 0x9d6353e thru 0x9d639f7 (0x4ba bytes) 2024-09-17T07:29:45.705 INFO:teuthology.orchestra.run.smithi045.stdout:3585 write 0xccd855b thru 0xccdc5d0 (0x4076 bytes) 2024-09-17T07:29:45.713 INFO:teuthology.orchestra.run.smithi045.stdout:3587 write 0xe87c07c thru 0xe888afc (0xca81 bytes) 2024-09-17T07:29:45.723 INFO:teuthology.orchestra.run.smithi045.stdout:3588 write 0x71ad492 thru 0x71b9049 (0xbbb8 bytes) 2024-09-17T07:29:45.733 INFO:teuthology.orchestra.run.smithi045.stdout:3589 read 0x9c864d5 thru 0x9c921b3 (0xbcdf bytes) 2024-09-17T07:29:45.733 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:45.733 INFO:teuthology.orchestra.run.smithi045.stdout:3592 write 0x7b1e958 thru 0x7b24fde (0x6687 bytes) 2024-09-17T07:29:45.736 INFO:teuthology.orchestra.run.smithi045.stdout:3594 writesame 0x2d36e67 thru 0x2d39aec data_size 0x2(0x2c86 bytes) 2024-09-17T07:29:45.742 INFO:teuthology.orchestra.run.smithi045.stdout:3595 write 0x9d2d73b thru 0x9d2e5ae (0xe74 bytes) 2024-09-17T07:29:45.746 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:45.746 INFO:teuthology.orchestra.run.smithi045.stdout:3597 read 0xc8a1b62 thru 0xc8a9984 (0x7e23 bytes) 2024-09-17T07:29:45.746 INFO:teuthology.orchestra.run.smithi045.stdout:3599 read 0x4dc519e thru 0x4dcae42 (0x5ca5 bytes) 2024-09-17T07:29:45.746 INFO:teuthology.orchestra.run.smithi045.stdout:3600 write 0x3811d6a thru 0x38204fc (0xe793 bytes) 2024-09-17T07:29:45.770 INFO:teuthology.orchestra.run.smithi045.stdout:3601 trunc from 0xe888afd to 0x545037f 2024-09-17T07:29:45.794 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:45.785+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:45.799 INFO:teuthology.orchestra.run.smithi045.stdout:3602 writesame 0x2fdcec7 thru 0x2fdf2c1 data_size 0x1(0x23fb bytes) 2024-09-17T07:29:45.817 INFO:teuthology.orchestra.run.smithi045.stdout:3603 writesame 0xa07ee3e thru 0xa080500 data_size 0x1(0x16c3 bytes) 2024-09-17T07:29:45.818 INFO:teuthology.orchestra.run.smithi045.stdout:3604 punch from 0x8a163e8 to 0x8a1c7a2, (0x63ba bytes) 2024-09-17T07:29:45.828 INFO:teuthology.orchestra.run.smithi045.stdout:3607 writesame 0xddb1ed8 thru 0xddb5fd9 data_size 0x35(0x4102 bytes) 2024-09-17T07:29:45.828 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:45.842 INFO:teuthology.orchestra.run.smithi045.stdout:3610 trunc from 0xddb5fda to 0x11b2586 2024-09-17T07:29:45.860 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:45.860 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.952970182999998, 2024-09-17T07:29:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:29:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:29:45.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:29:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:29:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:29:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:29:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:29:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:29:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:29:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:29:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:29:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:29:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:29:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:29:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:29:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.952951505999998, 2024-09-17T07:29:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:29:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:29:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:29:45.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:29:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:29:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:29:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:29:45.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:29:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:29:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:29:45.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:29:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:29:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:29:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:29:45.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:29:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.952933153000004, 2024-09-17T07:29:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:29:45.874 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:29:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:45.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:29:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:29:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:29:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:29:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:29:45.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:29:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:29:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:29:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:29:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:29:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:29:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:29:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:29:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.879 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.952913555000002, 2024-09-17T07:29:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:29:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.880 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:29:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:29:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:29:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:29:45.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:29:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:29:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:29:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:29:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:29:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:29:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:29:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:29:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:29:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:29:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:29:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.952893600000003, 2024-09-17T07:29:45.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:29:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:29:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:29:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:29:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:29:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:29:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:29:45.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:29:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:29:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:29:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:29:45.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:29:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:29:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:29:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:29:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.890 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.952874944999998, 2024-09-17T07:29:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:29:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:29:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:45.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:29:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:29:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:29:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:29:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:29:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:29:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:29:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:29:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:29:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:29:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:29:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:29:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:29:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.895 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.952856468999997, 2024-09-17T07:29:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:29:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.896 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:29:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:29:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:29:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:29:45.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:29:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:29:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:29:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:29:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:29:45.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:29:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:29:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:29:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:29:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:29:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:29:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:45.901 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.952836869999999, 2024-09-17T07:29:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:29:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:29:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:29:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:29:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:29:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:29:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:29:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:29:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:29:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:29:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:29:45.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:29:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:29:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:29:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:29:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.952818295, 2024-09-17T07:29:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:29:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.907 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:29:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:29:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:29:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:29:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:29:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:29:45.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:29:45.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:29:45.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:29:45.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:29:45.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:29:45.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:29:45.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:29:45.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:29:45.912 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.912 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.912 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.913 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:45.913 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:45.913 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.952795868000003, 2024-09-17T07:29:45.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:29:45.913 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.913 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.913 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.913 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:29:45.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.914 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:45.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:45.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:45.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:45.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:29:45.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:29:45.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:29:45.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:29:45.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:29:45.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:29:45.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:29:45.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:29:45.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:29:45.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:29:45.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:29:45.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:29:45.918 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.918 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.918 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.918 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:45.918 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:45.918 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.952768051, 2024-09-17T07:29:45.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:29:45.918 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.919 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.919 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.919 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:29:45.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.919 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:45.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:45.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:45.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:29:45.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:29:45.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:29:45.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:29:45.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:29:45.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:29:45.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:29:45.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:29:45.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:29:45.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:29:45.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:29:45.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:29:45.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:29:45.923 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.923 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.923 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.924 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:45.924 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:45.924 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.952746107000003, 2024-09-17T07:29:45.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:29:45.924 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.924 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.924 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.924 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.924 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.924 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.925 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:29:45.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.925 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:45.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:45.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:45.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:29:45.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:29:45.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:29:45.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:29:45.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:29:45.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:29:45.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:29:45.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:29:45.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:29:45.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:29:45.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:29:45.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:29:45.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:29:45.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.929 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.930 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:45.930 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:45.930 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.952725002999998, 2024-09-17T07:29:45.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:29:45.930 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.930 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.930 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.930 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.930 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.930 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.931 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:29:45.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.931 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:45.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:45.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:45.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:29:45.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:29:45.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:29:45.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:29:45.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:45.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:29:45.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:29:45.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:29:45.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:29:45.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:29:45.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:29:45.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:29:45.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:29:45.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.935 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.935 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:45.935 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:45.936 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.952705399999999, 2024-09-17T07:29:45.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:29:45.936 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.936 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.936 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.936 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:29:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:29:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:29:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:29:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:29:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:29:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:29:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:29:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:29:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:29:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:29:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:29:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:29:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:29:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:29:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.999764287000001, 2024-09-17T07:29:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:29:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:29:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:29:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:29:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:29:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:29:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:29:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:29:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:29:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:29:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:29:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:29:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:29:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:29:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:29:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:29:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.999750627000001, 2024-09-17T07:29:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:29:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:29:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:29:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:29:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:29:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:29:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:29:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:29:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:29:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:29:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:29:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:29:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:29:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:29:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:29:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.999735845, 2024-09-17T07:29:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:29:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:29:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:29:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:29:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:29:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:29:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:29:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:29:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:29:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:29:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:29:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:29:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:29:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:29:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:29:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.999724954000001, 2024-09-17T07:29:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:29:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:29:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:29:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:29:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:29:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:29:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:29:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:29:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:29:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:29:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:29:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:29:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:29:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:29:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.999715393999999, 2024-09-17T07:29:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:29:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:29:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:29:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:29:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:29:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:29:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:29:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:29:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:29:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:29:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:29:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:29:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:29:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:29:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:29:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.999704833999999, 2024-09-17T07:29:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:29:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:29:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:29:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:29:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:29:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:29:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:29:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:29:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:29:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:29:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:29:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:29:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:29:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:29:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:29:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:29:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:45.974 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:45.977 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:45.978 INFO:teuthology.orchestra.run.smithi045.stdout:3611 read 0x9ed53e thru 0x9fa935 (0xd3f8 bytes) 2024-09-17T07:29:45.978 INFO:teuthology.orchestra.run.smithi045.stdout:3612 punch from 0x108cc07 to 0x109899d, (0xbd96 bytes) 2024-09-17T07:29:45.978 INFO:teuthology.orchestra.run.smithi045.stdout:3613 trunc from 0x11b2586 to 0x1f2db84 2024-09-17T07:29:45.978 INFO:teuthology.orchestra.run.smithi045.stdout:3619 punch from 0x4dac80 to 0x4e74c2, (0xc842 bytes) 2024-09-17T07:29:45.978 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:45.978 INFO:teuthology.orchestra.run.smithi045.stdout:3621 punch from 0x19905a0 to 0x1998c7b, (0x86db bytes) 2024-09-17T07:29:45.978 INFO:teuthology.orchestra.run.smithi045.stdout:3623 write 0x2828a1c thru 0x282ab44 (0x2129 bytes) 2024-09-17T07:29:45.978 INFO:teuthology.orchestra.run.smithi045.stdout:3624 trunc from 0x282ab45 to 0x351a32e 2024-09-17T07:29:45.978 INFO:teuthology.orchestra.run.smithi045.stdout:3625 read 0x461638 thru 0x46314a (0x1b13 bytes) 2024-09-17T07:29:45.979 INFO:teuthology.orchestra.run.smithi045.stdout:3626 write 0xc9e60d4 thru 0xc9f3d71 (0xdc9e bytes) 2024-09-17T07:29:45.979 INFO:teuthology.orchestra.run.smithi045.stdout:3629 punch from 0xbd4c60b to 0xbd587c5, (0xc1ba bytes) 2024-09-17T07:29:45.979 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:45.979 INFO:teuthology.orchestra.run.smithi045.stdout:3631 trunc from 0xc9f3d72 to 0x858f9a0 2024-09-17T07:29:45.979 INFO:teuthology.orchestra.run.smithi045.stdout:3632 writesame 0x879db52 thru 0x87acdc3 data_size 0x1(0xf272 bytes) 2024-09-17T07:29:45.979 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:45.975+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:45.979 INFO:teuthology.orchestra.run.smithi045.stdout:3634 write 0xe1ca832 thru 0xe1cdb92 (0x3361 bytes) 2024-09-17T07:29:45.980 INFO:teuthology.orchestra.run.smithi045.stdout:3636 writesame 0xc52ea91 thru 0xc53b0c6 data_size 0x9(0xc636 bytes) 2024-09-17T07:29:45.989 INFO:teuthology.orchestra.run.smithi045.stdout:3637 writesame 0x2913e54 thru 0x291b023 data_size 0x18(0x71d0 bytes) 2024-09-17T07:29:45.994 INFO:teuthology.orchestra.run.smithi045.stdout:3639 read 0x3ae487e thru 0x3aeed95 (0xa518 bytes) 2024-09-17T07:29:45.994 INFO:teuthology.orchestra.run.smithi045.stdout:3642 read 0xb76e0c2 thru 0xb76ed47 (0xc86 bytes) 2024-09-17T07:29:45.994 INFO:teuthology.orchestra.run.smithi045.stdout:3643 read 0x15dccdb thru 0x15eab88 (0xdeae bytes) 2024-09-17T07:29:45.995 INFO:teuthology.orchestra.run.smithi045.stdout:3644 write 0x2b6a6d5 thru 0x2b75466 (0xad92 bytes) 2024-09-17T07:29:46.000 INFO:teuthology.orchestra.run.smithi045.stdout:3645 write 0xd371570 thru 0xd37f565 (0xdff6 bytes) 2024-09-17T07:29:46.019 INFO:teuthology.orchestra.run.smithi045.stdout:3646 trunc from 0xe1cdb93 to 0x5075240 2024-09-17T07:29:46.040 INFO:teuthology.orchestra.run.smithi045.stdout:3647 write 0x7b57753 thru 0x7b64394 (0xcc42 bytes) 2024-09-17T07:29:46.049 INFO:teuthology.orchestra.run.smithi045.stdout:3649 write 0x7042c4a thru 0x704a237 (0x75ee bytes) 2024-09-17T07:29:46.055 INFO:teuthology.orchestra.run.smithi045.stdout:3650 read 0x302bcbf thru 0x3038a55 (0xcd97 bytes) 2024-09-17T07:29:46.063 INFO:teuthology.orchestra.run.smithi045.stdout:3651 write 0x9d244f2 thru 0x9d27ef5 (0x3a04 bytes) 2024-09-17T07:29:46.066 INFO:teuthology.orchestra.run.smithi045.stdout:3652 punch from 0x8d86201 to 0x8d92072, (0xbe71 bytes) 2024-09-17T07:29:46.066 INFO:teuthology.orchestra.run.smithi045.stdout:3654 write 0x457e1bf thru 0x458784b (0x968d bytes) 2024-09-17T07:29:46.072 INFO:teuthology.orchestra.run.smithi045.stdout:3655 read 0x70b1fb7 thru 0x70b32a7 (0x12f1 bytes) 2024-09-17T07:29:46.075 INFO:teuthology.orchestra.run.smithi045.stdout:3656 trunc from 0x9d27ef6 to 0xc0b2f6f 2024-09-17T07:29:46.076 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:46.075+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:46.080 INFO:teuthology.orchestra.run.smithi045.stdout:3657 read 0x2e9e144 thru 0x2eae01e (0xfedb bytes) 2024-09-17T07:29:46.081 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:46.081 INFO:teuthology.orchestra.run.smithi045.stdout:3661 write 0x2ece4b7 thru 0x2ed8299 (0x9de3 bytes) 2024-09-17T07:29:46.082 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-17T07:29:46.085 INFO:teuthology.orchestra.run.smithi045.stdout:3663 write 0x44084fd thru 0x440b5a3 (0x30a7 bytes) 2024-09-17T07:29:46.096 INFO:teuthology.orchestra.run.smithi045.stdout:3664 trunc from 0xc0b2f6f to 0x769ebdd 2024-09-17T07:29:46.096 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:46.097 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:46.097 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:46.097 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:46.107 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:46.114 INFO:teuthology.orchestra.run.smithi045.stdout:3666 read 0x1ddb17a thru 0x1de9b3a (0xe9c1 bytes) 2024-09-17T07:29:46.114 INFO:teuthology.orchestra.run.smithi045.stdout:3667 write 0x4a9ecee thru 0x4aa02b4 (0x15c7 bytes) 2024-09-17T07:29:46.117 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:46.117 INFO:teuthology.orchestra.run.smithi045.stdout:3672 punch from 0x2570f83 to 0x2573187, (0x2204 bytes) 2024-09-17T07:29:46.117 INFO:teuthology.orchestra.run.smithi045.stdout:3675 write 0x5ed1e36 thru 0x5ed88aa (0x6a75 bytes) 2024-09-17T07:29:46.134 INFO:teuthology.orchestra.run.smithi045.stdout:3677 writesame 0xa286729 thru 0xa28d894 data_size 0x22(0x716c bytes) 2024-09-17T07:29:46.145 INFO:teuthology.orchestra.run.smithi045.stdout:3678 writesame 0xe61a12d thru 0xe6233f3 data_size 0x4b(0x92c7 bytes) 2024-09-17T07:29:46.145 INFO:teuthology.orchestra.run.smithi045.stdout:3679 writesame 0xdfec441 thru 0xdff1e4e data_size 0x1(0x5a0e bytes) 2024-09-17T07:29:46.153 INFO:teuthology.orchestra.run.smithi045.stdout:3680 read 0x582909f thru 0x58315c2 (0x8524 bytes) 2024-09-17T07:29:46.165 INFO:teuthology.orchestra.run.smithi045.stdout:3683 trunc from 0xe6233f4 to 0x47106ed 2024-09-17T07:29:46.177 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:46.177+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:46.183 INFO:teuthology.orchestra.run.smithi045.stdout:3685 punch from 0xc46e37 to 0xc4a8db, (0x3aa4 bytes) 2024-09-17T07:29:46.197 INFO:teuthology.orchestra.run.smithi045.stdout:3687 writesame 0xb63b2b2 thru 0xb63e85a data_size 0x13(0x35a9 bytes) 2024-09-17T07:29:46.197 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:46.197 INFO:teuthology.orchestra.run.smithi045.stdout:3690 writesame 0x7016f49 thru 0x70234ac data_size 0x6(0xc564 bytes) 2024-09-17T07:29:46.214 INFO:teuthology.orchestra.run.smithi045.stdout:3691 trunc from 0xb63e85b to 0x3e5d14b 2024-09-17T07:29:46.217 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:46.217 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:46.218 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:46.218 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:46.218 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:46.227 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:46.231 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:46.231 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:46.237 INFO:teuthology.orchestra.run.smithi045.stdout:3697 write 0x59a56c1 thru 0x59a93c0 (0x3d00 bytes) 2024-09-17T07:29:46.242 INFO:teuthology.orchestra.run.smithi045.stdout:3698 punch from 0x4782aa2 to 0x478776e, (0x4ccc bytes) 2024-09-17T07:29:46.250 INFO:teuthology.orchestra.run.smithi045.stdout:3699 write 0x73db585 thru 0x73eb54f (0xffcb bytes) 2024-09-17T07:29:46.261 INFO:teuthology.orchestra.run.smithi045.stdout:3700 punch from 0x2b822bc to 0x2b85584, (0x32c8 bytes) 2024-09-17T07:29:46.273 INFO:teuthology.orchestra.run.smithi045.stdout:3701 writesame 0xd067a2d thru 0xd0731e7 data_size 0x5(0xb7bb bytes) 2024-09-17T07:29:46.273 INFO:teuthology.orchestra.run.smithi045.stdout:3702 write 0x920c681 thru 0x9215564 (0x8ee4 bytes) 2024-09-17T07:29:46.278 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:46.278+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:46.280 INFO:teuthology.orchestra.run.smithi045.stdout:3704 trunc from 0xd0731e8 to 0xdbf6bad 2024-09-17T07:29:46.285 INFO:teuthology.orchestra.run.smithi045.stdout:3705 flatten 2024-09-17T07:29:46.380 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:29:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:46.380 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 80.043260927000006, 2024-09-17T07:29:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:29:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:29:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:29:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:29:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:29:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:29:46.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:29:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:29:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:29:46.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:29:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:29:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:29:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:29:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:29:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:29:46.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:29:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:29:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:29:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:29:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:29:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:29:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:29:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:29:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:29:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:29:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:29:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:29:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:29:46.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:29:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:29:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:29:46.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:29:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7378 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7176192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.454218892999997, 2024-09-17T07:29:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073003980999999996, 2024-09-17T07:29:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.388 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7378 2024-09-17T07:29:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:46.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1799999999999999e-07 2024-09-17T07:29:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923713+0000", 2024-09-17T07:29:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.057e-06 2024-09-17T07:29:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923716+0000", 2024-09-17T07:29:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0599999999999999e-06 2024-09-17T07:29:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923719+0000", 2024-09-17T07:29:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8219999999999998e-06 2024-09-17T07:29:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996401+0000", 2024-09-17T07:29:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072681467999999999 2024-09-17T07:29:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996452+0000", 2024-09-17T07:29:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.134e-05 2024-09-17T07:29:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996716+0000", 2024-09-17T07:29:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000264016 2024-09-17T07:29:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:46.393 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.454208776999998, 2024-09-17T07:29:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:29:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:29:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:29:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:29:46.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:29:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:29:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:29:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:29:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:29:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:29:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:29:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:29:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:29:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:29:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:29:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:29:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7398 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6389760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.448940243000003, 2024-09-17T07:29:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072891580999999997, 2024-09-17T07:29:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7398 2024-09-17T07:29:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928995+0000", 2024-09-17T07:29:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0759999999999996e-06 2024-09-17T07:29:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928999+0000", 2024-09-17T07:29:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1609999999999996e-06 2024-09-17T07:29:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929002+0000", 2024-09-17T07:29:46.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8449999999999999e-06 2024-09-17T07:29:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001598+0000", 2024-09-17T07:29:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072596318000000007 2024-09-17T07:29:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001663+0000", 2024-09-17T07:29:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4995999999999999e-05 2024-09-17T07:29:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001882+0000", 2024-09-17T07:29:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021897299999999999 2024-09-17T07:29:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.448925660999997, 2024-09-17T07:29:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:29:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.407 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:29:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:29:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:29:46.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:29:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:29:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:29:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:29:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:29:46.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:29:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:29:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:29:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:29:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:29:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:29:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.448894680999999, 2024-09-17T07:29:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:29:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.412 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:29:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:29:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:29:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:29:46.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:29:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:29:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:29:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:29:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:29:46.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:29:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:29:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:29:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:29:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:29:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:29:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.448864632000003, 2024-09-17T07:29:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:29:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.417 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:29:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:29:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:29:46.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:29:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:29:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:29:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:29:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:29:46.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:29:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:29:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:29:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:29:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:29:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:29:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:29:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.423 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.448836225000001, 2024-09-17T07:29:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:29:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:29:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:46.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:29:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:29:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:29:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:29:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:29:46.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:29:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:29:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:29:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:29:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:29:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:29:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:29:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.448816403999999, 2024-09-17T07:29:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:29:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.428 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:29:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:46.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:29:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:29:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:29:46.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:29:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:29:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:29:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:29:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:29:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:29:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:29:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:29:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:29:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:29:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:29:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.432 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.448792152000003, 2024-09-17T07:29:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:29:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.433 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:29:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.434 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:29:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:29:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:29:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:29:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:29:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:29:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:29:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:29:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:29:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:29:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:29:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:29:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:29:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:29:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7418 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7700480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.443386128999997, 2024-09-17T07:29:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072752965000000003, 2024-09-17T07:29:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.439 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7418 2024-09-17T07:29:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3599999999999999e-07 2024-09-17T07:29:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934547+0000", 2024-09-17T07:29:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3350000000000001e-06 2024-09-17T07:29:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934554+0000", 2024-09-17T07:29:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9859999999999997e-06 2024-09-17T07:29:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934557+0000", 2024-09-17T07:29:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.818e-06 2024-09-17T07:29:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007104+0000", 2024-09-17T07:29:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072546779000000006 2024-09-17T07:29:46.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007147+0000", 2024-09-17T07:29:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2904000000000001e-05 2024-09-17T07:29:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007298+0000", 2024-09-17T07:29:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015080700000000001 2024-09-17T07:29:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.443 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.443369564000001, 2024-09-17T07:29:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:29:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:29:46.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.445 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:46.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:46.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:29:46.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:29:46.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:29:46.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:29:46.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:29:46.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:29:46.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:29:46.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:29:46.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:29:46.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:29:46.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:29:46.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:29:46.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:29:46.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:29:46.448 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.448 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.448 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.449 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.449 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:46.449 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.443322688999999, 2024-09-17T07:29:46.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:29:46.449 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.449 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.449 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.449 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.449 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.449 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.450 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:29:46.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.450 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:46.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:46.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:46.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:46.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:29:46.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:29:46.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:29:46.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:29:46.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:29:46.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:29:46.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:29:46.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:29:46.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:29:46.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:29:46.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:29:46.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:29:46.453 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.453 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.453 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.454 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.454 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:46.454 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.443307984999997, 2024-09-17T07:29:46.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:29:46.455 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.455 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.455 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.455 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.455 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.455 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.455 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:29:46.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.455 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:46.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:46.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:46.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:46.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:29:46.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:29:46.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:29:46.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:29:46.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:29:46.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:29:46.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:29:46.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:29:46.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:29:46.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:29:46.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:29:46.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:29:46.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.460 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.460 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.460 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:46.460 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.443284890000001, 2024-09-17T07:29:46.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:29:46.460 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.460 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.460 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.460 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.461 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:29:46.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.461 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:46.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:46.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:46.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:29:46.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:29:46.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:29:46.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:29:46.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:29:46.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:29:46.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:29:46.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:29:46.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:29:46.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:29:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:29:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:29:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:29:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.443253515000002, 2024-09-17T07:29:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:29:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:29:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:29:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:29:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:29:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:29:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:29:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:29:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:29:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:29:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:29:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:29:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:29:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:29:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:29:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:29:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.443238254999997, 2024-09-17T07:29:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:29:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:29:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:29:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:29:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:29:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:29:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:29:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:29:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:29:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:29:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:29:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:29:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:29:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:29:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:29:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.443218184999999, 2024-09-17T07:29:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:29:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:29:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:29:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:29:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:29:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:29:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:29:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:29:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:29:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:29:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:29:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:29:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:29:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:29:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:29:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.443206244999999, 2024-09-17T07:29:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:29:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:29:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:29:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:29:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:29:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:29:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:29:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:29:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:29:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:29:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:29:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:29:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:29:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:29:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.443183087000001, 2024-09-17T07:29:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:29:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:29:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:29:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:29:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:29:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:29:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:29:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:29:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:29:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:29:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:29:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:29:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:29:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:29:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:29:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:29:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.488 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:46.488 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:46.488 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:46.488 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:46.379+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:46.490 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:46.492 INFO:teuthology.orchestra.run.smithi045.stdout:3706 read 0x2221317 thru 0x222d0eb (0xbdd5 bytes) 2024-09-17T07:29:46.493 INFO:teuthology.orchestra.run.smithi045.stdout:3707 read 0x64d64a7 thru 0x64d8866 (0x23c0 bytes) 2024-09-17T07:29:46.493 INFO:teuthology.orchestra.run.smithi045.stdout:3708 trunc from 0xdbf6bad to 0xc03686c 2024-09-17T07:29:46.493 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:46.493 INFO:teuthology.orchestra.run.smithi045.stdout:3710 read 0xad6c8a4 thru 0xad7acb7 (0xe414 bytes) 2024-09-17T07:29:46.493 INFO:teuthology.orchestra.run.smithi045.stdout:3711 read 0x326395a thru 0x32733ac (0xfa53 bytes) 2024-09-17T07:29:46.493 INFO:teuthology.orchestra.run.smithi045.stdout:3712 punch from 0x7cc1d0 to 0x7dbcc5, (0xfaf5 bytes) 2024-09-17T07:29:46.493 INFO:teuthology.orchestra.run.smithi045.stdout:3714 write 0x8211fd0 thru 0x821eb3d (0xcb6e bytes) 2024-09-17T07:29:46.493 INFO:teuthology.orchestra.run.smithi045.stdout:3715 read 0x9bbf16a thru 0x9bc6d0e (0x7ba5 bytes) 2024-09-17T07:29:46.493 INFO:teuthology.orchestra.run.smithi045.stdout:3716 write 0x4dadadf thru 0x4db1002 (0x3524 bytes) 2024-09-17T07:29:46.493 INFO:teuthology.orchestra.run.smithi045.stdout:3717 read 0x3c43c3d thru 0x3c4a611 (0x69d5 bytes) 2024-09-17T07:29:46.493 INFO:teuthology.orchestra.run.smithi045.stdout:3719 writesame 0x46f7af1 thru 0x470238d data_size 0x59(0xa89d bytes) 2024-09-17T07:29:46.493 INFO:teuthology.orchestra.run.smithi045.stdout:3720 write 0xbea1bc7 thru 0xbeb0a9e (0xeed8 bytes) 2024-09-17T07:29:46.494 INFO:teuthology.orchestra.run.smithi045.stdout:3721 read 0x7f7e50c thru 0x7f8a5b7 (0xc0ac bytes) 2024-09-17T07:29:46.494 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:46.494 INFO:teuthology.orchestra.run.smithi045.stdout:3724 writesame 0xe56eddf thru 0xe56f06b data_size 0x1(0x28d bytes) 2024-09-17T07:29:46.494 INFO:teuthology.orchestra.run.smithi045.stdout:3726 writesame 0x570f110 thru 0x57137a6 data_size 0x1(0x4697 bytes) 2024-09-17T07:29:46.494 INFO:teuthology.orchestra.run.smithi045.stdout:3727 read 0x563e463 thru 0x564ab90 (0xc72e bytes) 2024-09-17T07:29:46.494 INFO:teuthology.orchestra.run.smithi045.stdout:3728 punch from 0xd3c6b4d to 0xd3c7ed0, (0x1383 bytes) 2024-09-17T07:29:46.494 INFO:teuthology.orchestra.run.smithi045.stdout:3729 read 0x5155bca thru 0x515f8ac (0x9ce3 bytes) 2024-09-17T07:29:46.494 INFO:teuthology.orchestra.run.smithi045.stdout:3732 punch from 0x7b727d6 to 0x7b81dc3, (0xf5ed bytes) 2024-09-17T07:29:46.494 INFO:teuthology.orchestra.run.smithi045.stdout:3733 write 0xcb7f83a thru 0xcb8cc32 (0xd3f9 bytes) 2024-09-17T07:29:46.494 INFO:teuthology.orchestra.run.smithi045.stdout:3734 punch from 0xaf812df to 0xaf8a9ca, (0x96eb bytes) 2024-09-17T07:29:46.494 INFO:teuthology.orchestra.run.smithi045.stdout:3735 trunc from 0xe56f06c to 0x2f83725 2024-09-17T07:29:46.495 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:46.488+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:46.512 INFO:teuthology.orchestra.run.smithi045.stdout:3736 read 0x5f72ef thru 0x5fc424 (0x5136 bytes) 2024-09-17T07:29:46.519 INFO:teuthology.orchestra.run.smithi045.stdout:3737 write 0x3655c85 thru 0x365b55f (0x58db bytes) 2024-09-17T07:29:46.529 INFO:teuthology.orchestra.run.smithi045.stdout:3739 write 0x1c49320 thru 0x1c58bc4 (0xf8a5 bytes) 2024-09-17T07:29:46.552 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:29:46.558 INFO:teuthology.orchestra.run.smithi045.stdout:3741 write 0xeaa610b thru 0xeaafd3d (0x9c33 bytes) 2024-09-17T07:29:46.567 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:46.567 INFO:teuthology.orchestra.run.smithi045.stdout:3743 read 0x52421ba thru 0x52490c0 (0x6f07 bytes) 2024-09-17T07:29:46.567 INFO:teuthology.orchestra.run.smithi045.stdout:3744 write 0xe6ec626 thru 0xe6f5bdb (0x95b6 bytes) 2024-09-17T07:29:46.575 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:46.575 INFO:teuthology.orchestra.run.smithi045.stdout:3748 write 0x44be727 thru 0x44c799a (0x9274 bytes) 2024-09-17T07:29:46.583 INFO:teuthology.orchestra.run.smithi045.stdout:3750 writesame 0x4921485 thru 0x49264f4 data_size 0x4(0x5070 bytes) 2024-09-17T07:29:46.590 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:46.589+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:46.591 INFO:teuthology.orchestra.run.smithi045.stdout:3752 writesame 0x2c070fe thru 0x2c14380 data_size 0x1(0xd283 bytes) 2024-09-17T07:29:46.599 INFO:teuthology.orchestra.run.smithi045.stdout:3753 writesame 0x542dd3a thru 0x5436b18 data_size 0x1(0x8ddf bytes) 2024-09-17T07:29:46.613 INFO:teuthology.orchestra.run.smithi045.stdout:3754 read 0x6e0a242 thru 0x6e19576 (0xf335 bytes) 2024-09-17T07:29:46.613 INFO:teuthology.orchestra.run.smithi045.stdout:3755 read 0xcebe9ad thru 0xceca929 (0xbf7d bytes) 2024-09-17T07:29:46.613 INFO:teuthology.orchestra.run.smithi045.stdout:3756 write 0x6dfbd68 thru 0x6e09cdd (0xdf76 bytes) 2024-09-17T07:29:46.626 INFO:teuthology.orchestra.run.smithi045.stdout:3757 write 0xe972e1e thru 0xe979bdf (0x6dc2 bytes) 2024-09-17T07:29:46.630 INFO:teuthology.orchestra.run.smithi045.stdout:3758 read 0xdbdb661 thru 0xdbdfb63 (0x4503 bytes) 2024-09-17T07:29:46.630 INFO:teuthology.orchestra.run.smithi045.stdout:3759 write 0x28e47e4 thru 0x28f279e (0xdfbb bytes) 2024-09-17T07:29:46.633 INFO:teuthology.orchestra.run.smithi045.stdout:3760 read 0x30073ec thru 0x3016a4c (0xf661 bytes) 2024-09-17T07:29:46.633 INFO:teuthology.orchestra.run.smithi045.stdout:3761 read 0x9b1f59 thru 0x9b7282 (0x532a bytes) 2024-09-17T07:29:46.637 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:46.638 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:46.638 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:46.638 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:46.644 INFO:teuthology.orchestra.run.smithi045.stdout:3762 trunc from 0xeaafd3e to 0x5e7c718 2024-09-17T07:29:46.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:46.665 INFO:teuthology.orchestra.run.smithi045.stdout:3764 write 0x76da64a thru 0x76deca1 (0x4658 bytes) 2024-09-17T07:29:46.681 INFO:teuthology.orchestra.run.smithi045.stdout:3765 writesame 0xbe6226d thru 0xbe6d54c data_size 0x24(0xb2e0 bytes) 2024-09-17T07:29:46.681 INFO:teuthology.orchestra.run.smithi045.stdout:3768 writesame 0x89d17c2 thru 0x89d6f4b data_size 0x5a(0x578a bytes) 2024-09-17T07:29:46.687 INFO:teuthology.orchestra.run.smithi045.stdout:3769 punch from 0x87a9521 to 0x87b3921, (0xa400 bytes) 2024-09-17T07:29:46.687 INFO:teuthology.orchestra.run.smithi045.stdout:3770 read 0x14465ea thru 0x144f64d (0x9064 bytes) 2024-09-17T07:29:46.688 INFO:teuthology.orchestra.run.smithi045.stdout:3771 writesame 0xb2eace8 thru 0xb2f748a data_size 0x1(0xc7a3 bytes) 2024-09-17T07:29:46.691 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:46.690+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:46.703 INFO:teuthology.orchestra.run.smithi045.stdout:3772 read 0x8af2ae5 thru 0x8af83f1 (0x590d bytes) 2024-09-17T07:29:46.703 INFO:teuthology.orchestra.run.smithi045.stdout:3774 read 0x6a7915b thru 0x6a85ae2 (0xc988 bytes) 2024-09-17T07:29:46.703 INFO:teuthology.orchestra.run.smithi045.stdout:3777 write 0x44750e3 thru 0x44810a5 (0xbfc3 bytes) 2024-09-17T07:29:46.720 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:46.720 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:46.723 INFO:teuthology.orchestra.run.smithi045.stdout:3778 trunc from 0xbe6d54d to 0x1143909 2024-09-17T07:29:46.745 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:46.745 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:46.754 INFO:teuthology.orchestra.run.smithi045.stdout:3779 write 0x1b0014a thru 0x1b01722 (0x15d9 bytes) 2024-09-17T07:29:46.758 INFO:teuthology.orchestra.run.smithi045.stdout:3780 punch from 0xb2a653 to 0xb32054, (0x7a01 bytes) 2024-09-17T07:29:46.758 INFO:teuthology.orchestra.run.smithi045.stdout:3781 trunc from 0x1b01723 to 0x1896a07 2024-09-17T07:29:46.763 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:46.764 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:46.764 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:46.767 INFO:teuthology.orchestra.run.smithi045.stdout:3782 read 0x713bdb thru 0x71d468 (0x988e bytes) 2024-09-17T07:29:46.768 INFO:teuthology.orchestra.run.smithi045.stdout:3783 read 0xb70318 thru 0xb7a1b3 (0x9e9c bytes) 2024-09-17T07:29:46.780 INFO:teuthology.orchestra.run.smithi045.stdout:3784 writesame 0x6a76c33 thru 0x6a7d725 data_size 0x13(0x6af3 bytes) 2024-09-17T07:29:46.783 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:46.783 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:46.788 INFO:teuthology.orchestra.run.smithi045.stdout:3785 trunc from 0x6a7d726 to 0xdc3c0c7 2024-09-17T07:29:46.792 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:46.791+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:46.793 INFO:teuthology.orchestra.run.smithi045.stdout:3787 write 0xaf16381 thru 0xaf1771a (0x139a bytes) 2024-09-17T07:29:46.796 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:46.796 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:46.797 INFO:teuthology.orchestra.run.smithi045.stdout:3789 read 0x4833707 thru 0x483abe4 (0x74de bytes) 2024-09-17T07:29:46.797 INFO:teuthology.orchestra.run.smithi045.stdout:3792 write 0xcbeb3fd thru 0xcbf668a (0xb28e bytes) 2024-09-17T07:29:46.803 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:46.803 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:46.806 INFO:teuthology.orchestra.run.smithi045.stdout:3793 read 0x6a7afca thru 0x6a7c545 (0x157c bytes) 2024-09-17T07:29:46.806 INFO:teuthology.orchestra.run.smithi045.stdout:3794 writesame 0xf909 thru 0x1d2b7 data_size 0x1(0xd9af bytes) 2024-09-17T07:29:46.808 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:46.815 INFO:teuthology.orchestra.run.smithi045.stdout:3795 punch from 0xaaadc4 to 0xaacd91, (0x1fcd bytes) 2024-09-17T07:29:46.816 INFO:teuthology.orchestra.run.smithi045.stdout:3796 punch from 0x33309a3 to 0x333853f, (0x7b9c bytes) 2024-09-17T07:29:46.818 INFO:teuthology.orchestra.run.smithi045.stdout:3797 trunc from 0xdc3c0c7 to 0xc20bc8a 2024-09-17T07:29:46.824 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:46.827 INFO:teuthology.orchestra.run.smithi045.stdout:3798 punch from 0x9a2587f to 0x9a333f3, (0xdb74 bytes) 2024-09-17T07:29:46.828 INFO:teuthology.orchestra.run.smithi045.stdout:3800 punch from 0x9775610 to 0x9779e82, (0x4872 bytes) 2024-09-17T07:29:46.838 INFO:teuthology.orchestra.run.smithi045.stdout:3801 trunc from 0xc20bc8a to 0x2afaf85 2024-09-17T07:29:46.842 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:46.842 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:46.846 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:46.847 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:46.849 INFO:teuthology.orchestra.run.smithi045.stdout:3802 write 0x16f7e46 thru 0x16fae55 (0x3010 bytes) 2024-09-17T07:29:46.857 INFO:teuthology.orchestra.run.smithi045.stdout:3803 write 0x3326498 thru 0x3335c71 (0xf7da bytes) 2024-09-17T07:29:46.884 INFO:teuthology.orchestra.run.smithi045.stdout:3804 write 0xd839676 thru 0xd843717 (0xa0a2 bytes) 2024-09-17T07:29:46.892 INFO:teuthology.orchestra.run.smithi045.stdout:3805 write 0xf0129f thru 0xf10375 (0xf0d7 bytes) 2024-09-17T07:29:46.893 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:46.892+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:46.897 INFO:teuthology.orchestra.run.smithi045.stdout:3807 write 0x7169c85 thru 0x717317e (0x94fa bytes) 2024-09-17T07:29:46.905 INFO:teuthology.orchestra.run.smithi045.stdout:3808 writesame 0xb516c63 thru 0xb516fef data_size 0x3(0x38d bytes) 2024-09-17T07:29:46.908 INFO:teuthology.orchestra.run.smithi045.stdout:3809 write 0x5e59a99 thru 0x5e5a246 (0x7ae bytes) 2024-09-17T07:29:46.913 INFO:teuthology.orchestra.run.smithi045.stdout:3810 punch from 0x1494296 to 0x1496f36, (0x2ca0 bytes) 2024-09-17T07:29:46.913 INFO:teuthology.orchestra.run.smithi045.stdout:3813 write 0xac78d88 thru 0xac7bd2c (0x2fa5 bytes) 2024-09-17T07:29:46.916 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:46.916 INFO:teuthology.orchestra.run.smithi045.stdout:3817 writesame 0x4f831fc thru 0x4f91a88 data_size 0x1(0xe88d bytes) 2024-09-17T07:29:46.933 INFO:teuthology.orchestra.run.smithi045.stdout:3819 read 0x8afb5ff thru 0x8b0b313 (0xfd15 bytes) 2024-09-17T07:29:46.933 INFO:teuthology.orchestra.run.smithi045.stdout:3821 write 0x45cdf92 thru 0x45d77be (0x982d bytes) 2024-09-17T07:29:46.941 INFO:teuthology.orchestra.run.smithi045.stdout:3822 punch from 0x6baee77 to 0x6bb2b02, (0x3c8b bytes) 2024-09-17T07:29:46.941 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:46.941 INFO:teuthology.orchestra.run.smithi045.stdout:3824 write 0x5dcf6f0 thru 0x5dd2d99 (0x36aa bytes) 2024-09-17T07:29:46.945 INFO:teuthology.orchestra.run.smithi045.stdout:3825 write 0x73ebaef thru 0x73f535c (0x986e bytes) 2024-09-17T07:29:46.949 INFO:teuthology.orchestra.run.smithi045.stdout:3826 punch from 0x92cb175 to 0x92cf972, (0x47fd bytes) 2024-09-17T07:29:46.949 INFO:teuthology.orchestra.run.smithi045.stdout:3827 punch from 0x8d48a7b to 0x8d57fcd, (0xf552 bytes) 2024-09-17T07:29:46.949 INFO:teuthology.orchestra.run.smithi045.stdout:3828 writesame 0xd48a4d thru 0xd576a4 data_size 0xc(0xec58 bytes) 2024-09-17T07:29:46.951 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:46.951 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:46.952 INFO:teuthology.orchestra.run.smithi045.stdout:3829 read 0x996cf53 thru 0x996f379 (0x2427 bytes) 2024-09-17T07:29:46.953 INFO:teuthology.orchestra.run.smithi045.stdout:3831 read 0x4ddcbb0 thru 0x4ddda3f (0xe90 bytes) 2024-09-17T07:29:46.964 INFO:teuthology.orchestra.run.smithi045.stdout:3832 trunc from 0xd843718 to 0x3e7c73c 2024-09-17T07:29:46.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:46.998 INFO:teuthology.orchestra.run.smithi045.stdout:3833 write 0xc30a11b thru 0xc30f2ae (0x5194 bytes) 2024-09-17T07:29:46.998 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:46.997+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:47.002 INFO:teuthology.orchestra.run.smithi045.stdout:3835 clone 34 order 23 su 262144 sc 7 2024-09-17T07:29:47.094 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:47.094+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:47.108 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:47.108 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:47.118 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:47.202 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:47.202+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:47.272 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-17T07:29:47.274 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:47.274 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:47.288 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:47.297 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:47.296+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:47.338 INFO:teuthology.orchestra.run.smithi045.stdout:3837 read 0x2336c97 thru 0x233f450 (0x87ba bytes) 2024-09-17T07:29:47.339 INFO:teuthology.orchestra.run.smithi045.stdout:3838 write 0x499916f thru 0x499bb5f (0x29f1 bytes) 2024-09-17T07:29:47.347 INFO:teuthology.orchestra.run.smithi045.stdout:3839 write 0xacb507d thru 0xacb83e2 (0x3366 bytes) 2024-09-17T07:29:47.350 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.351 INFO:teuthology.orchestra.run.smithi045.stdout:3841 read 0xabf0ea6 thru 0xabf5765 (0x48c0 bytes) 2024-09-17T07:29:47.351 INFO:teuthology.orchestra.run.smithi045.stdout:3842 write 0x2cfc19e thru 0x2d0a10f (0xdf72 bytes) 2024-09-17T07:29:47.380 INFO:teuthology.orchestra.run.smithi045.stdout:3843 punch from 0x36ba76f to 0x36c5590, (0xae21 bytes) 2024-09-17T07:29:47.380 INFO:teuthology.orchestra.run.smithi045.stdout:3844 writesame 0x447727e thru 0x4482001 data_size 0xa(0xad84 bytes) 2024-09-17T07:29:47.387 INFO:teuthology.orchestra.run.smithi045.stdout:3845 read 0xac76cfe thru 0xac84e26 (0xe129 bytes) 2024-09-17T07:29:47.387 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.387 INFO:teuthology.orchestra.run.smithi045.stdout:3847 write 0x54ba8d3 thru 0x54c8c7c (0xe3aa bytes) 2024-09-17T07:29:47.394 INFO:teuthology.orchestra.run.smithi045.stdout:3848 read 0x6fec49b thru 0x6ffb77a (0xf2e0 bytes) 2024-09-17T07:29:47.394 INFO:teuthology.orchestra.run.smithi045.stdout:3849 punch from 0xbd4eb4f to 0xbd5c0fd, (0xd5ae bytes) 2024-09-17T07:29:47.397 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:47.397+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:47.401 INFO:teuthology.orchestra.run.smithi045.stdout:3853 writesame 0xcad6607 thru 0xcad90d6 data_size 0xa(0x2ad0 bytes) 2024-09-17T07:29:47.404 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.405 INFO:teuthology.orchestra.run.smithi045.stdout:3855 write 0x345ee9c thru 0x346a7b7 (0xb91c bytes) 2024-09-17T07:29:47.416 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.416 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.416 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.416 INFO:teuthology.orchestra.run.smithi045.stdout:3860 punch from 0x68648b4 to 0x686f6d8, (0xae24 bytes) 2024-09-17T07:29:47.417 INFO:teuthology.orchestra.run.smithi045.stdout:3861 trunc from 0xcad90d7 to 0xc1eefe6 2024-09-17T07:29:47.434 INFO:teuthology.orchestra.run.smithi045.stdout:3862 trunc from 0xc1eefe6 to 0xbffe190 2024-09-17T07:29:47.451 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:47.451 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:47.455 INFO:teuthology.orchestra.run.smithi045.stdout:3863 trunc from 0xbffe190 to 0x1878f51 2024-09-17T07:29:47.463 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:47.505 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:47.505+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:47.514 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.523 INFO:teuthology.orchestra.run.smithi045.stdout:3865 writesame 0x4c33d6b thru 0x4c43a9a data_size 0x8(0xfd30 bytes) 2024-09-17T07:29:47.523 INFO:teuthology.orchestra.run.smithi045.stdout:3866 read 0x67faaa thru 0x686918 (0x6e6f bytes) 2024-09-17T07:29:47.524 INFO:teuthology.orchestra.run.smithi045.stdout:3867 writesame 0x1edc2a4 thru 0x1edc387 data_size 0x4(0xe4 bytes) 2024-09-17T07:29:47.525 INFO:teuthology.orchestra.run.smithi045.stdout:3868 write 0x4be191 thru 0x4ccadc (0xe94c bytes) 2024-09-17T07:29:47.527 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.535 INFO:teuthology.orchestra.run.smithi045.stdout:3871 write 0x75b3d00 thru 0x75be58d (0xa88e bytes) 2024-09-17T07:29:47.551 INFO:teuthology.orchestra.run.smithi045.stdout:3872 write 0xe29b877 thru 0xe2a287d (0x7007 bytes) 2024-09-17T07:29:47.554 INFO:teuthology.orchestra.run.smithi045.stdout:3873 read 0x30e97a2 thru 0x30f6384 (0xcbe3 bytes) 2024-09-17T07:29:47.555 INFO:teuthology.orchestra.run.smithi045.stdout:3874 read 0x39855c3 thru 0x398f2eb (0x9d29 bytes) 2024-09-17T07:29:47.555 INFO:teuthology.orchestra.run.smithi045.stdout:3875 read 0x29e8088 thru 0x29ea628 (0x25a1 bytes) 2024-09-17T07:29:47.570 INFO:teuthology.orchestra.run.smithi045.stdout:3876 trunc from 0xe2a287e to 0xea1abd 2024-09-17T07:29:47.596 INFO:teuthology.orchestra.run.smithi045.stdout:3877 writesame 0x862138f thru 0x862f320 data_size 0x1(0xdf92 bytes) 2024-09-17T07:29:47.596 INFO:teuthology.orchestra.run.smithi045.stdout:3878 read 0xad08a0 thru 0xad9c2d (0x938e bytes) 2024-09-17T07:29:47.597 INFO:teuthology.orchestra.run.smithi045.stdout:3879 writesame 0x762c62a thru 0x763b15a data_size 0x1(0xeb31 bytes) 2024-09-17T07:29:47.599 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:47.599+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:47.624 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:47.624 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:47.626 INFO:teuthology.orchestra.run.smithi045.stdout:3881 punch from 0x3321924 to 0x3322f5b, (0x1637 bytes) 2024-09-17T07:29:47.626 INFO:teuthology.orchestra.run.smithi045.stdout:3882 writesame 0xdef94d thru 0xdf9c1e data_size 0x3(0xa2d2 bytes) 2024-09-17T07:29:47.633 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:47.646 INFO:teuthology.orchestra.run.smithi045.stdout:3883 write 0xdc1850d thru 0xdc271d0 (0xecc4 bytes) 2024-09-17T07:29:47.662 INFO:teuthology.orchestra.run.smithi045.stdout:3886 trunc from 0xdc271d1 to 0x217ddde 2024-09-17T07:29:47.689 INFO:teuthology.orchestra.run.smithi045.stdout:3888 writesame 0xb7e4fcd thru 0xb7ef784 data_size 0xc(0xa7b8 bytes) 2024-09-17T07:29:47.695 INFO:teuthology.orchestra.run.smithi045.stdout:3889 writesame 0xc856c97 thru 0xc857a11 data_size 0x11(0xd7b bytes) 2024-09-17T07:29:47.695 INFO:teuthology.orchestra.run.smithi045.stdout:3891 write 0x503b1ea thru 0x503b446 (0x25d bytes) 2024-09-17T07:29:47.700 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:47.700+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:47.705 INFO:teuthology.orchestra.run.smithi045.stdout:3892 write 0xe1a40a7 thru 0xe1b2a4b (0xe9a5 bytes) 2024-09-17T07:29:47.708 INFO:teuthology.orchestra.run.smithi045.stdout:3895 read 0x8892c72 thru 0x8895a29 (0x2db8 bytes) 2024-09-17T07:29:47.708 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.708 INFO:teuthology.orchestra.run.smithi045.stdout:3897 write 0xcae0e8a thru 0xcae8d29 (0x7ea0 bytes) 2024-09-17T07:29:47.712 INFO:teuthology.orchestra.run.smithi045.stdout:3898 read 0x269f1cf thru 0x26a25b9 (0x33eb bytes) 2024-09-17T07:29:47.712 INFO:teuthology.orchestra.run.smithi045.stdout:3900 read 0x13a8a0 thru 0x13bb52 (0x12b3 bytes) 2024-09-17T07:29:47.713 INFO:teuthology.orchestra.run.smithi045.stdout:3901 read 0x322efa2 thru 0x323bcd1 (0xcd30 bytes) 2024-09-17T07:29:47.713 INFO:teuthology.orchestra.run.smithi045.stdout:3903 write 0xdac0cff thru 0xdac71e1 (0x64e3 bytes) 2024-09-17T07:29:47.716 INFO:teuthology.orchestra.run.smithi045.stdout:3904 read 0x26ff930 thru 0x2702e32 (0x3503 bytes) 2024-09-17T07:29:47.716 INFO:teuthology.orchestra.run.smithi045.stdout:3906 write 0x6814d5c thru 0x6819007 (0x42ac bytes) 2024-09-17T07:29:47.719 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.719 INFO:teuthology.orchestra.run.smithi045.stdout:3908 read 0x68c657e thru 0x68cdb88 (0x760b bytes) 2024-09-17T07:29:47.720 INFO:teuthology.orchestra.run.smithi045.stdout:3910 write 0xcc9140d thru 0xcc9978f (0x8383 bytes) 2024-09-17T07:29:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:3911 read 0x3e43c6e thru 0x3e45243 (0x15d6 bytes) 2024-09-17T07:29:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:3913 read 0x40eb689 thru 0x40f97f9 (0xe171 bytes) 2024-09-17T07:29:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:3914 read 0xb2d7396 thru 0xb2db5cb (0x4236 bytes) 2024-09-17T07:29:47.722 INFO:teuthology.orchestra.run.smithi045.stdout:3915 writesame 0xde5983c thru 0xde620e2 data_size 0x27(0x88a7 bytes) 2024-09-17T07:29:47.722 INFO:teuthology.orchestra.run.smithi045.stdout:3916 write 0x5bd11a5 thru 0x5bd654c (0x53a8 bytes) 2024-09-17T07:29:47.724 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.724 INFO:teuthology.orchestra.run.smithi045.stdout:3919 write 0x9000374 thru 0x900b4c8 (0xb155 bytes) 2024-09-17T07:29:47.730 INFO:teuthology.orchestra.run.smithi045.stdout:3920 write 0x19fe54b thru 0x19fe5ed (0xa3 bytes) 2024-09-17T07:29:47.730 INFO:teuthology.orchestra.run.smithi045.stdout:3921 read 0x1acee3c thru 0x1ad3a6a (0x4c2f bytes) 2024-09-17T07:29:47.730 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.730 INFO:teuthology.orchestra.run.smithi045.stdout:3924 write 0x8f3a35c thru 0x8f47262 (0xcf07 bytes) 2024-09-17T07:29:47.734 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.735 INFO:teuthology.orchestra.run.smithi045.stdout:3927 read 0x6cc2cfb thru 0x6ccf98a (0xcc90 bytes) 2024-09-17T07:29:47.735 INFO:teuthology.orchestra.run.smithi045.stdout:3929 write 0x4f10894 thru 0x4f1e782 (0xdeef bytes) 2024-09-17T07:29:47.750 INFO:teuthology.orchestra.run.smithi045.stdout:3930 trunc from 0xe1b2a4c to 0x3f36734 2024-09-17T07:29:47.761 INFO:teuthology.orchestra.run.smithi045.stdout:3931 write 0x42d22f thru 0x43bff8 (0xedca bytes) 2024-09-17T07:29:47.763 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.767 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:47.767 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:47.772 INFO:teuthology.orchestra.run.smithi045.stdout:3933 writesame 0x731f2af thru 0x731f9e3 data_size 0xf(0x735 bytes) 2024-09-17T07:29:47.778 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:47.784 INFO:teuthology.orchestra.run.smithi045.stdout:3934 write 0xd336c0b thru 0xd3380fe (0x14f4 bytes) 2024-09-17T07:29:47.787 INFO:teuthology.orchestra.run.smithi045.stdout:3936 read 0xd001d92 thru 0xd00523a (0x34a9 bytes) 2024-09-17T07:29:47.788 INFO:teuthology.orchestra.run.smithi045.stdout:3937 punch from 0xc32583b to 0xc325e77, (0x63c bytes) 2024-09-17T07:29:47.788 INFO:teuthology.orchestra.run.smithi045.stdout:3939 read 0x9254067 thru 0x925e6fe (0xa698 bytes) 2024-09-17T07:29:47.788 INFO:teuthology.orchestra.run.smithi045.stdout:3940 read 0x8e57f01 thru 0x8e65d3a (0xde3a bytes) 2024-09-17T07:29:47.788 INFO:teuthology.orchestra.run.smithi045.stdout:3941 read 0x2f3df10 thru 0x2f41778 (0x3869 bytes) 2024-09-17T07:29:47.788 INFO:teuthology.orchestra.run.smithi045.stdout:3942 read 0x6a9f886 thru 0x6aab35e (0xbad9 bytes) 2024-09-17T07:29:47.788 INFO:teuthology.orchestra.run.smithi045.stdout:3943 read 0xb2ef931 thru 0xb2f7b5a (0x822a bytes) 2024-09-17T07:29:47.788 INFO:teuthology.orchestra.run.smithi045.stdout:3944 read 0x43a1f96 thru 0x43a8d16 (0x6d81 bytes) 2024-09-17T07:29:47.788 INFO:teuthology.orchestra.run.smithi045.stdout:3945 punch from 0x41b9963 to 0x41bd7df, (0x3e7c bytes) 2024-09-17T07:29:47.788 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.788 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.793 INFO:teuthology.orchestra.run.smithi045.stdout:3949 trunc from 0xd3380ff to 0x8164035 2024-09-17T07:29:47.801 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:47.801+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:47.802 INFO:teuthology.orchestra.run.smithi045.stdout:3950 writesame 0x56ab620 thru 0x56b4153 data_size 0x3b(0x8b34 bytes) 2024-09-17T07:29:47.804 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.804 INFO:teuthology.orchestra.run.smithi045.stdout:3952 write 0x4efe8e7 thru 0x4f0c087 (0xd7a1 bytes) 2024-09-17T07:29:47.808 INFO:teuthology.orchestra.run.smithi045.stdout:3953 punch from 0x49dd70e to 0x49e5483, (0x7d75 bytes) 2024-09-17T07:29:47.808 INFO:teuthology.orchestra.run.smithi045.stdout:3954 write 0x5bc03a1 thru 0x5bc3d28 (0x3988 bytes) 2024-09-17T07:29:47.817 INFO:teuthology.orchestra.run.smithi045.stdout:3956 write 0xab96810 thru 0xab9a971 (0x4162 bytes) 2024-09-17T07:29:47.820 INFO:teuthology.orchestra.run.smithi045.stdout:3957 read 0x5b4b128 thru 0x5b53ccd (0x8ba6 bytes) 2024-09-17T07:29:47.826 INFO:teuthology.orchestra.run.smithi045.stdout:3958 write 0xb8822d9 thru 0xb88b1a7 (0x8ecf bytes) 2024-09-17T07:29:47.829 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.830 INFO:teuthology.orchestra.run.smithi045.stdout:3962 write 0x2341c12 thru 0x2349a37 (0x7e26 bytes) 2024-09-17T07:29:47.831 INFO:teuthology.orchestra.run.smithi045.stdout:3963 writesame 0xb2e3dce thru 0xb2ebb03 data_size 0x3e(0x7d36 bytes) 2024-09-17T07:29:47.834 INFO:teuthology.orchestra.run.smithi045.stdout:3964 writesame 0x8955d1d thru 0x895e20d data_size 0x1(0x84f1 bytes) 2024-09-17T07:29:47.852 INFO:teuthology.orchestra.run.smithi045.stdout:3966 writesame 0x74a7ef0 thru 0x74ac5f3 data_size 0x14(0x4704 bytes) 2024-09-17T07:29:47.855 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.856 INFO:teuthology.orchestra.run.smithi045.stdout:3969 read 0x99f500 thru 0x9ac1eb (0xccec bytes) 2024-09-17T07:29:47.859 INFO:teuthology.orchestra.run.smithi045.stdout:3971 trunc from 0xb88b1a8 to 0x8b7d783 2024-09-17T07:29:47.870 INFO:teuthology.orchestra.run.smithi045.stdout:3973 read 0x1bebca7 thru 0x1bf6e8a (0xb1e4 bytes) 2024-09-17T07:29:47.870 INFO:teuthology.orchestra.run.smithi045.stdout:3974 read 0x18435cf thru 0x184e92b (0xb35d bytes) 2024-09-17T07:29:47.870 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.871 INFO:teuthology.orchestra.run.smithi045.stdout:3976 write 0x31d216e thru 0x31d334e (0x11e1 bytes) 2024-09-17T07:29:47.872 INFO:teuthology.orchestra.run.smithi045.stdout:3977 read 0x843c6c8 thru 0x8449a14 (0xd34d bytes) 2024-09-17T07:29:47.872 INFO:teuthology.orchestra.run.smithi045.stdout:3980 read 0x3661ab4 thru 0x3667c22 (0x616f bytes) 2024-09-17T07:29:47.873 INFO:teuthology.orchestra.run.smithi045.stdout:3981 write 0x2341d0c thru 0x23420d0 (0x3c5 bytes) 2024-09-17T07:29:47.879 INFO:teuthology.orchestra.run.smithi045.stdout:3982 trunc from 0x8b7d783 to 0xccdbc4b 2024-09-17T07:29:47.893 INFO:teuthology.orchestra.run.smithi045.stdout:3983 trunc from 0xccdbc4b to 0x5ade66f 2024-09-17T07:29:47.902 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:47.902 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:47.902 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:47.901+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:47.904 INFO:teuthology.orchestra.run.smithi045.stdout:3986 write 0x1e0775e thru 0x1e17694 (0xff37 bytes) 2024-09-17T07:29:47.906 INFO:teuthology.orchestra.run.smithi045.stdout:3987 read 0x2f67b62 thru 0x2f6e431 (0x68d0 bytes) 2024-09-17T07:29:47.907 INFO:teuthology.orchestra.run.smithi045.stdout:3988 flatten 2024-09-17T07:29:47.911 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:47.914 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.921 INFO:teuthology.orchestra.run.smithi045.stdout:3991 write 0x76ce8f4 thru 0x76dbe36 (0xd543 bytes) 2024-09-17T07:29:47.929 INFO:teuthology.orchestra.run.smithi045.stdout:3993 writesame 0x77f1b7f thru 0x77f2863 data_size 0x1(0xce5 bytes) 2024-09-17T07:29:47.932 INFO:teuthology.orchestra.run.smithi045.stdout:3994 trunc from 0x77f2864 to 0x4e2b4c2 2024-09-17T07:29:47.944 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.944 INFO:teuthology.orchestra.run.smithi045.stdout:3997 punch from 0xf6d8c8 to 0xf71ffe, (0x4736 bytes) 2024-09-17T07:29:47.944 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:47.944 INFO:teuthology.orchestra.run.smithi045.stdout:4000 read 0x2afb4 thru 0x3091d (0x596a bytes) 2024-09-17T07:29:47.960 INFO:teuthology.orchestra.run.smithi045.stdout:4001 write 0xc639f4b thru 0xc643b4d (0x9c03 bytes) 2024-09-17T07:29:47.964 INFO:teuthology.orchestra.run.smithi045.stdout:4002 read 0x9254757 thru 0x925e73d (0x9fe7 bytes) 2024-09-17T07:29:47.964 INFO:teuthology.orchestra.run.smithi045.stdout:4003 punch from 0xa3c59e7 to 0xa3cffdf, (0xa5f8 bytes) 2024-09-17T07:29:47.964 INFO:teuthology.orchestra.run.smithi045.stdout:4005 read 0x89ca728 thru 0x89d10b5 (0x698e bytes) 2024-09-17T07:29:47.964 INFO:teuthology.orchestra.run.smithi045.stdout:4006 writesame 0x98e7d9a thru 0x98f7297 data_size 0x3(0xf4fe bytes) 2024-09-17T07:29:47.977 INFO:teuthology.orchestra.run.smithi045.stdout:4010 read 0x10f6023 thru 0x10f8993 (0x2971 bytes) 2024-09-17T07:29:47.977 INFO:teuthology.orchestra.run.smithi045.stdout:4012 punch from 0x89e5ec8 to 0x89e67bd, (0x8f5 bytes) 2024-09-17T07:29:47.977 INFO:teuthology.orchestra.run.smithi045.stdout:4013 read 0x38d20d7 thru 0x38df5ea (0xd514 bytes) 2024-09-17T07:29:47.978 INFO:teuthology.orchestra.run.smithi045.stdout:4014 writesame 0xa1dae2f thru 0xa1e8648 data_size 0x2(0xd81a bytes) 2024-09-17T07:29:47.993 INFO:teuthology.orchestra.run.smithi045.stdout:4016 writesame 0xcdaea3 thru 0xcdf351 data_size 0x1(0x44af bytes) 2024-09-17T07:29:48.002 INFO:teuthology.orchestra.run.smithi045.stdout:4017 write 0xc13bbad thru 0xc13c67e (0xad2 bytes) 2024-09-17T07:29:48.002 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:48.002+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:48.012 INFO:teuthology.orchestra.run.smithi045.stdout:4018 write 0xe7ed34c thru 0xe7f9bdb (0xc890 bytes) 2024-09-17T07:29:48.015 INFO:teuthology.orchestra.run.smithi045.stdout:4019 punch from 0x92112e to 0x92df95, (0xce67 bytes) 2024-09-17T07:29:48.015 INFO:teuthology.orchestra.run.smithi045.stdout:4020 write 0x2b5998b thru 0x2b6641c (0xca92 bytes) 2024-09-17T07:29:48.017 INFO:teuthology.orchestra.run.smithi045.stdout:4021 writesame 0x5c14b2f thru 0x5c1c3d8 data_size 0x5(0x78aa bytes) 2024-09-17T07:29:48.022 INFO:teuthology.orchestra.run.smithi045.stdout:4022 write 0x6da3db5 thru 0x6da4ccb (0xf17 bytes) 2024-09-17T07:29:48.026 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:48.027 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:48.029 INFO:teuthology.orchestra.run.smithi045.stdout:4024 write 0xebe864a thru 0xebf0110 (0x7ac7 bytes) 2024-09-17T07:29:48.033 INFO:teuthology.orchestra.run.smithi045.stdout:4025 read 0x937a85 thru 0x93a310 (0x288c bytes) 2024-09-17T07:29:48.033 INFO:teuthology.orchestra.run.smithi045.stdout:4026 writesame 0xd415377 thru 0xd415f0e data_size 0x4(0xb98 bytes) 2024-09-17T07:29:48.036 INFO:teuthology.orchestra.run.smithi045.stdout:4027 writesame 0xb7ab761 thru 0xb7b090b data_size 0x45(0x51ab bytes) 2024-09-17T07:29:48.037 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:48.041 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:48.041 INFO:teuthology.orchestra.run.smithi045.stdout:4030 read 0xd3a9440 thru 0xd3b44d1 (0xb092 bytes) 2024-09-17T07:29:48.041 INFO:teuthology.orchestra.run.smithi045.stdout:4031 read 0x1d64bd thru 0x1e612c (0xfc70 bytes) 2024-09-17T07:29:48.042 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:48.042 INFO:teuthology.orchestra.run.smithi045.stdout:4033 write 0xc4b9466 thru 0xc4c7722 (0xe2bd bytes) 2024-09-17T07:29:48.060 INFO:teuthology.orchestra.run.smithi045.stdout:4034 trunc from 0xebf0111 to 0x1746299 2024-09-17T07:29:48.078 INFO:teuthology.orchestra.run.smithi045.stdout:4036 read 0x151a133 thru 0x152103e (0x6f0c bytes) 2024-09-17T07:29:48.085 INFO:teuthology.orchestra.run.smithi045.stdout:4037 writesame 0x2783331 thru 0x279221a data_size 0x35(0xeeea bytes) 2024-09-17T07:29:48.087 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:48.088 INFO:teuthology.orchestra.run.smithi045.stdout:4040 trunc from 0x279221b to 0x1bf38b7 2024-09-17T07:29:48.100 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T07:29:48.115 INFO:teuthology.orchestra.run.smithi045.stdout:4042 writesame 0xd2b89a0 thru 0xd2bc06a data_size 0xd(0x36cb bytes) 2024-09-17T07:29:48.115 INFO:teuthology.orchestra.run.smithi045.stdout:4043 punch from 0x548b17a to 0x548bd2a, (0xbb0 bytes) 2024-09-17T07:29:48.115 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:48.122 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:48.122+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:48.127 INFO:teuthology.orchestra.run.smithi045.stdout:4045 trunc from 0xd2bc06b to 0x2ab38f2 2024-09-17T07:29:48.140 INFO:teuthology.orchestra.run.smithi045.stdout:4046 write 0x2b0aa48 thru 0x2b0da5d (0x3016 bytes) 2024-09-17T07:29:48.141 INFO:teuthology.orchestra.run.smithi045.stdout:4047 write 0x11da90f thru 0x11ddb5b (0x324d bytes) 2024-09-17T07:29:48.143 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:48.143 INFO:teuthology.orchestra.run.smithi045.stdout:4050 read 0x1141818 thru 0x114cc03 (0xb3ec bytes) 2024-09-17T07:29:48.145 INFO:teuthology.orchestra.run.smithi045.stdout:4051 trunc from 0x2b0da5e to 0x4d7e71a 2024-09-17T07:29:48.151 INFO:teuthology.orchestra.run.smithi045.stdout:4052 write 0x3924aa2 thru 0x392cb8b (0x80ea bytes) 2024-09-17T07:29:48.170 INFO:teuthology.orchestra.run.smithi045.stdout:4053 write 0xdea62e3 thru 0xdeb2066 (0xbd84 bytes) 2024-09-17T07:29:48.173 INFO:teuthology.orchestra.run.smithi045.stdout:4054 write 0xc08bcff thru 0xc091292 (0x5594 bytes) 2024-09-17T07:29:48.176 INFO:teuthology.orchestra.run.smithi045.stdout:4055 write 0x138b6b5 thru 0x139008e (0x49da bytes) 2024-09-17T07:29:48.182 INFO:teuthology.orchestra.run.smithi045.stdout:4057 trunc from 0xdeb2067 to 0x95501f2 2024-09-17T07:29:48.186 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:48.186 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:48.192 INFO:teuthology.orchestra.run.smithi045.stdout:4058 write 0x5313fc9 thru 0x5321350 (0xd388 bytes) 2024-09-17T07:29:48.196 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:48.197 INFO:teuthology.orchestra.run.smithi045.stdout:4059 trunc from 0x95501f2 to 0xb4f94d2 2024-09-17T07:29:48.203 INFO:teuthology.orchestra.run.smithi045.stdout:4061 read 0x68b6470 thru 0x68bb82c (0x53bd bytes) 2024-09-17T07:29:48.203 INFO:teuthology.orchestra.run.smithi045.stdout:4063 read 0x821939b thru 0x82281f7 (0xee5d bytes) 2024-09-17T07:29:48.203 INFO:teuthology.orchestra.run.smithi045.stdout:4064 read 0x22b914 thru 0x2325ee (0x6cdb bytes) 2024-09-17T07:29:48.204 INFO:teuthology.orchestra.run.smithi045.stdout:4065 write 0x6268e85 thru 0x62763d9 (0xd555 bytes) 2024-09-17T07:29:48.207 INFO:teuthology.orchestra.run.smithi045.stdout:4066 read 0x65650c9 thru 0x6571f2d (0xce65 bytes) 2024-09-17T07:29:48.207 INFO:teuthology.orchestra.run.smithi045.stdout:4067 read 0xa00849b thru 0xa00f114 (0x6c7a bytes) 2024-09-17T07:29:48.207 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:48.207 INFO:teuthology.orchestra.run.smithi045.stdout:4069 write 0x5c9cd07 thru 0x5ca4900 (0x7bfa bytes) 2024-09-17T07:29:48.211 INFO:teuthology.orchestra.run.smithi045.stdout:4070 write 0x79a6b7f thru 0x79aa547 (0x39c9 bytes) 2024-09-17T07:29:48.214 INFO:teuthology.orchestra.run.smithi045.stdout:4071 read 0x170e932 thru 0x1714cb6 (0x6385 bytes) 2024-09-17T07:29:48.214 INFO:teuthology.orchestra.run.smithi045.stdout:4072 read 0x4992363 thru 0x4998b78 (0x6816 bytes) 2024-09-17T07:29:48.214 INFO:teuthology.orchestra.run.smithi045.stdout:4075 write 0x632a4e9 thru 0x632cbf6 (0x270e bytes) 2024-09-17T07:29:48.220 INFO:teuthology.orchestra.run.smithi045.stdout:4076 trunc from 0xb4f94d2 to 0xedbc212 2024-09-17T07:29:48.225 INFO:teuthology.orchestra.run.smithi045.stdout:4080 punch from 0xcb7571a to 0xcb8038e, (0xac74 bytes) 2024-09-17T07:29:48.225 INFO:teuthology.orchestra.run.smithi045.stdout:4081 read 0x589a35b thru 0x589d39d (0x3043 bytes) 2024-09-17T07:29:48.225 INFO:teuthology.orchestra.run.smithi045.stdout:4082 writesame 0xdd67d27 thru 0xdd6f65f data_size 0x1(0x7939 bytes) 2024-09-17T07:29:48.237 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:48.237+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:48.241 INFO:teuthology.orchestra.run.smithi045.stdout:4083 writesame 0x604d2b6 thru 0x6053be8 data_size 0x3(0x6933 bytes) 2024-09-17T07:29:48.248 INFO:teuthology.orchestra.run.smithi045.stdout:4086 writesame 0x2bc572c thru 0x2bc6d58 data_size 0x1(0x162d bytes) 2024-09-17T07:29:48.251 INFO:teuthology.orchestra.run.smithi045.stdout:4087 read 0xd600f9e thru 0xd60c8a6 (0xb909 bytes) 2024-09-17T07:29:48.251 INFO:teuthology.orchestra.run.smithi045.stdout:4089 write 0x63b65da thru 0x63c4a80 (0xe4a7 bytes) 2024-09-17T07:29:48.255 INFO:teuthology.orchestra.run.smithi045.stdout:4090 write 0x9c73e3f thru 0x9c77d8d (0x3f4f bytes) 2024-09-17T07:29:48.271 INFO:teuthology.orchestra.run.smithi045.stdout:4091 trunc from 0xedbc212 to 0x2d48bf0 2024-09-17T07:29:48.284 INFO:teuthology.orchestra.run.smithi045.stdout:4094 punch from 0x2bfb8d8 to 0x2c0506e, (0x9796 bytes) 2024-09-17T07:29:48.284 INFO:teuthology.orchestra.run.smithi045.stdout:4095 read 0x19daabb thru 0x19ea96e (0xfeb4 bytes) 2024-09-17T07:29:48.285 INFO:teuthology.orchestra.run.smithi045.stdout:4096 read 0x221f5f thru 0x22a434 (0x84d6 bytes) 2024-09-17T07:29:48.285 INFO:teuthology.orchestra.run.smithi045.stdout:4099 read 0x1b155a thru 0x1bc4f6 (0xaf9d bytes) 2024-09-17T07:29:48.299 INFO:teuthology.orchestra.run.smithi045.stdout:4100 write 0x9da3bda thru 0x9db11f7 (0xd61e bytes) 2024-09-17T07:29:48.304 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:48.304+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:48.310 INFO:teuthology.orchestra.run.smithi045.stdout:4103 write 0xbfac4da thru 0xbfaf16a (0x2c91 bytes) 2024-09-17T07:29:48.313 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:48.313 INFO:teuthology.orchestra.run.smithi045.stdout:4105 write 0xbd76e6a thru 0xbd81986 (0xab1d bytes) 2024-09-17T07:29:48.318 INFO:teuthology.orchestra.run.smithi045.stdout:4106 read 0x36a5321 thru 0x36a8959 (0x3639 bytes) 2024-09-17T07:29:48.318 INFO:teuthology.orchestra.run.smithi045.stdout:4107 read 0x567fa66 thru 0x56862bb (0x6856 bytes) 2024-09-17T07:29:48.325 INFO:teuthology.orchestra.run.smithi045.stdout:4110 write 0xceefa3c thru 0xcef2aea (0x30af bytes) 2024-09-17T07:29:48.326 INFO:teuthology.orchestra.run.smithi045.stdout:4111 writesame 0x227eba2 thru 0x228b49c data_size 0x1(0xc8fb bytes) 2024-09-17T07:29:48.347 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:48.347 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:48.356 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:48.356 INFO:teuthology.orchestra.run.smithi045.stdout:4112 trunc from 0xcef2aeb to 0x4b2dc34 2024-09-17T07:29:48.369 INFO:teuthology.orchestra.run.smithi045.stdout:4113 read 0x3142c03 thru 0x314d9db (0xadd9 bytes) 2024-09-17T07:29:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:4114 writesame 0xe8b8da4 thru 0xe8bbfc1 data_size 0x5(0x321e bytes) 2024-09-17T07:29:48.386 INFO:teuthology.orchestra.run.smithi045.stdout:4115 write 0x773ab6f thru 0x7742a23 (0x7eb5 bytes) 2024-09-17T07:29:48.391 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:48.392 INFO:teuthology.orchestra.run.smithi045.stdout:4117 writesame 0x705f32e thru 0x706a4ba data_size 0x6d(0xb18d bytes) 2024-09-17T07:29:48.393 INFO:teuthology.orchestra.run.smithi045.stdout:4118 read 0xb5bb4df thru 0xb5c4aea (0x960c bytes) 2024-09-17T07:29:48.393 INFO:teuthology.orchestra.run.smithi045.stdout:4119 read 0x4460489 thru 0x446b3e7 (0xaf5f bytes) 2024-09-17T07:29:48.393 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:48.393 INFO:teuthology.orchestra.run.smithi045.stdout:4122 write 0xcba03d6 thru 0xcba96f4 (0x931f bytes) 2024-09-17T07:29:48.396 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:48.396 INFO:teuthology.orchestra.run.smithi045.stdout:4127 punch from 0x6e35058 to 0x6e4345b, (0xe403 bytes) 2024-09-17T07:29:48.396 INFO:teuthology.orchestra.run.smithi045.stdout:4128 punch from 0x44904ad to 0x449125e, (0xdb1 bytes) 2024-09-17T07:29:48.396 INFO:teuthology.orchestra.run.smithi045.stdout:4129 write 0x2cc613 thru 0x2d2b94 (0x6582 bytes) 2024-09-17T07:29:48.398 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:48.398 INFO:teuthology.orchestra.run.smithi045.stdout:4132 write 0x5104da6 thru 0x5112cfd (0xdf58 bytes) 2024-09-17T07:29:48.401 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:48.401 INFO:teuthology.orchestra.run.smithi045.stdout:4134 write 0x9da69ae thru 0x9dae618 (0x7c6b bytes) 2024-09-17T07:29:48.402 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:48.403 INFO:teuthology.orchestra.run.smithi045.stdout:4137 punch from 0xbb41b0 to 0xbbde2a, (0x9c7a bytes) 2024-09-17T07:29:48.403 INFO:teuthology.orchestra.run.smithi045.stdout:4138 write 0x3ee5fb2 thru 0x3ee78d7 (0x1926 bytes) 2024-09-17T07:29:48.405 INFO:teuthology.orchestra.run.smithi045.stdout:4139 write 0xc8eb760 thru 0xc8fb6ae (0xff4f bytes) 2024-09-17T07:29:48.409 INFO:teuthology.orchestra.run.smithi045.stdout:4140 punch from 0x753f8b9 to 0x7544c44, (0x538b bytes) 2024-09-17T07:29:48.409 INFO:teuthology.orchestra.run.smithi045.stdout:4141 read 0x4523382 thru 0x452aff1 (0x7c70 bytes) 2024-09-17T07:29:48.409 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:48.409 INFO:teuthology.orchestra.run.smithi045.stdout:4148 read 0x5526d6 thru 0x561557 (0xee82 bytes) 2024-09-17T07:29:48.409 INFO:teuthology.orchestra.run.smithi045.stdout:4149 read 0x33c7e7d thru 0x33cd71f (0x58a3 bytes) 2024-09-17T07:29:48.410 INFO:teuthology.orchestra.run.smithi045.stdout:4151 punch from 0xa924bf5 to 0xa9348c3, (0xfcce bytes) 2024-09-17T07:29:48.417 INFO:teuthology.orchestra.run.smithi045.stdout:4152 trunc from 0xe8bbfc2 to 0x7e9b751 2024-09-17T07:29:48.429 INFO:teuthology.orchestra.run.smithi045.stdout:4154 writesame 0x1c0df3c thru 0x1c11580 data_size 0xb(0x3645 bytes) 2024-09-17T07:29:48.430 INFO:teuthology.orchestra.run.smithi045.stdout:4155 clone 35 order 22 su 131072 sc 15 2024-09-17T07:29:48.439 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:48.439+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:48.502 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:48.502 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:48.506 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:48.505+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:48.510 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:48.607 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:48.607+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:48.653 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:48.654 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:48.662 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:48.747 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:48.747+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:48.807 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:48.807 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:48.809 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:48.809+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:48.816 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:48.848 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:29:48.910 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:48.909+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:48.960 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:48.960 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:48.969 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:48.975 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:48.992 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:48.993 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:49.009 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:49.009 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:49.009 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:49.012 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:49.012+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:49.026 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:49.026 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:49.043 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:49.043 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:49.060 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:49.060 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:49.073 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:49.073 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:49.073 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:49.073 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:49.073 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:49.076 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:49.079 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:49.080 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:49.082 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:49.093 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-17T07:29:49.148 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:49.148+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:49.212 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:49.212+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:49.231 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:49.232 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:49.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:49.232 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:49.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.232 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:49.232 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:49.232 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.324711225999998, 2024-09-17T07:29:49.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:29:49.232 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.232 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.232 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.233 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.233 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.233 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:29:49.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.233 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:49.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:49.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:49.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:29:49.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:29:49.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:29:49.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:29:49.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:29:49.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:29:49.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:29:49.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:29:49.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:29:49.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:29:49.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:29:49.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:29:49.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:29:49.237 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.237 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.237 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.237 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:49.237 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:49.237 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.324692548999998, 2024-09-17T07:29:49.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:29:49.237 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.237 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.237 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.237 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.238 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.238 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.238 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:29:49.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.238 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:49.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:49.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:29:49.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:49.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:29:49.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:29:49.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:29:49.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:29:49.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:29:49.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:29:49.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:29:49.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:29:49.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:29:49.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:29:49.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:29:49.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:29:49.241 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.241 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.241 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.242 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:49.242 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:49.242 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.324674195999997, 2024-09-17T07:29:49.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:29:49.242 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.242 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.242 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.242 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:29:49.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.243 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:49.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:49.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:49.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:29:49.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:29:49.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:29:49.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:29:49.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:29:49.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:29:49.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:29:49.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:29:49.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:29:49.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:29:49.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:29:49.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:29:49.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:29:49.246 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.246 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.246 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.246 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:49.246 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:49.246 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.324654598000002, 2024-09-17T07:29:49.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:29:49.246 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.247 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.247 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.247 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.247 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.247 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.247 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:29:49.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.247 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:49.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:49.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:29:49.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:29:49.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:29:49.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:29:49.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:29:49.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:29:49.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:29:49.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:29:49.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:29:49.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:29:49.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:29:49.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:29:49.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:29:49.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:29:49.250 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.250 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.250 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.251 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:49.251 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:49.251 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.324634643000003, 2024-09-17T07:29:49.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:29:49.251 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.251 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.251 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.251 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.251 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.251 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.251 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:29:49.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.251 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:49.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:49.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:49.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:29:49.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:29:49.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:29:49.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:29:49.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:29:49.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:29:49.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:29:49.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:29:49.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:29:49.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:29:49.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:29:49.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:29:49.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:29:49.255 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.255 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.255 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.255 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:49.255 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:49.255 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.324615987999998, 2024-09-17T07:29:49.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:29:49.255 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.255 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.256 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.256 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:29:49.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.256 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:49.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:49.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:49.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:29:49.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:29:49.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:29:49.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:29:49.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:29:49.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:29:49.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:29:49.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:29:49.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:29:49.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:29:49.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:29:49.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:29:49.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:29:49.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.259 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.260 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:49.260 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:49.260 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.324597511999997, 2024-09-17T07:29:49.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:29:49.260 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.260 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.260 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.260 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:29:49.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.261 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:49.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:49.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:29:49.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:29:49.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:29:49.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:29:49.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:29:49.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:29:49.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:29:49.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:29:49.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:29:49.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:29:49.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:29:49.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:29:49.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:29:49.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:29:49.264 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.264 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.264 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.264 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:49.264 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:49.265 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.324577912999999, 2024-09-17T07:29:49.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:29:49.265 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.265 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.265 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.265 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:29:49.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.266 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:49.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:49.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:49.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:29:49.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:29:49.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:29:49.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:29:49.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:29:49.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:29:49.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:29:49.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:29:49.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:29:49.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:29:49.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:29:49.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:29:49.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:29:49.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.269 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.269 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:49.269 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:49.269 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.324559338, 2024-09-17T07:29:49.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:29:49.270 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.270 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.270 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.270 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:29:49.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.270 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:49.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:49.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:29:49.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:49.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:29:49.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:29:49.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:29:49.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:29:49.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:29:49.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:29:49.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:29:49.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:29:49.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:29:49.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:29:49.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:29:49.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:29:49.274 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.274 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.274 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.274 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:49.274 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:49.274 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.324536911000003, 2024-09-17T07:29:49.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:29:49.274 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.274 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.274 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.275 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.275 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.275 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:29:49.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.275 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:49.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:49.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:29:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:29:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:29:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:29:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:29:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:29:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:29:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:29:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:29:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:29:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:29:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:29:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.324509094, 2024-09-17T07:29:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:29:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:29:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:29:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:29:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:29:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:29:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:29:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:29:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:29:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:29:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:29:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:29:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:29:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:29:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:29:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.324487150000003, 2024-09-17T07:29:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:29:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:29:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:29:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:29:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:29:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:29:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:29:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:29:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:29:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:29:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:29:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:29:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:29:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:29:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:29:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.324466045999998, 2024-09-17T07:29:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:29:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:29:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:29:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:29:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:29:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:29:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:29:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:29:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:29:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:29:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:29:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:29:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:29:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:29:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.324446442999999, 2024-09-17T07:29:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:29:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:29:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:29:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:29:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:29:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:29:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:29:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:29:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:29:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:29:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:29:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:29:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:29:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:29:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:29:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:29:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.371505330000002, 2024-09-17T07:29:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:29:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:29:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:29:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:29:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:29:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:29:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:29:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:29:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:29:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:29:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:29:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:29:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:29:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:29:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:29:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:29:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.371491670000001, 2024-09-17T07:29:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:29:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:29:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:29:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:29:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:29:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:29:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:29:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:29:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:29:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:29:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:29:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:29:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:29:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:29:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:29:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.371476888, 2024-09-17T07:29:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:29:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:29:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:29:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:29:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:29:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:29:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:29:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:29:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:29:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:29:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:29:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:29:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:29:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:29:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:29:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.371465997000001, 2024-09-17T07:29:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:29:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:29:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:29:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:29:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:29:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:29:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:29:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:29:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:29:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:29:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:29:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:29:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:29:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:29:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.371456436999999, 2024-09-17T07:29:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:29:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:29:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:29:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:29:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:29:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:29:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:29:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:29:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:29:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:29:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:29:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:29:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:29:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:29:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:29:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.371445876999999, 2024-09-17T07:29:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:29:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:29:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:29:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:29:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:29:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:29:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:29:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:29:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:29:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:29:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:29:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:29:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:29:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:29:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:29:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:29:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.325 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:49.326 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-17T07:29:49.327 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:49.327 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:49.326+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:49.427 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:49.427+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:49.464 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:49.464 INFO:teuthology.orchestra.run.smithi045.stdout:4157 writesame 0x100614c thru 0x101191a data_size 0x5(0xb7cf bytes) 2024-09-17T07:29:49.473 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:49.473 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:49.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.473 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:94302 3.d 3:b7881231:::rbd_object_map.118311e0d6f8:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-17T07:29:49.473 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:49.471931+0000", 2024-09-17T07:29:49.473 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.00082734599999999996, 2024-09-17T07:29:49.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00084623800000000002, 2024-09-17T07:29:49.473 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.473 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.474 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "waiting for sub ops", 2024-09-17T07:29:49.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.474 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 94302 2024-09-17T07:29:49.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.474 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:49.471931+0000", 2024-09-17T07:29:49.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:49.471931+0000", 2024-09-17T07:29:49.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:49.471932+0000", 2024-09-17T07:29:49.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7899999999999997e-07 2024-09-17T07:29:49.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:49.471937+0000", 2024-09-17T07:29:49.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.502e-06 2024-09-17T07:29:49.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:49.471948+0000", 2024-09-17T07:29:49.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594999999999999e-05 2024-09-17T07:29:49.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:49.471953+0000", 2024-09-17T07:29:49.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3460000000000001e-06 2024-09-17T07:29:49.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:49.471993+0000", 2024-09-17T07:29:49.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9261999999999998e-05 2024-09-17T07:29:49.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:49.472046+0000", 2024-09-17T07:29:49.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3436999999999997e-05 2024-09-17T07:29:49.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 5", 2024-09-17T07:29:49.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:49.472238+0000", 2024-09-17T07:29:49.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00019207299999999999 2024-09-17T07:29:49.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.478 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.479 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.479 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-09-17T07:29:49.479 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 1 2024-09-17T07:29:49.479 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:49.479 INFO:teuthology.orchestra.run.smithi045.stdout:4158 punch from 0x49fe2ea to 0x4a0429c, (0x5fb2 bytes) 2024-09-17T07:29:49.483 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:49.483 INFO:teuthology.orchestra.run.smithi045.stdout:4159 trunc from 0x7e9b751 to 0x4805d6f 2024-09-17T07:29:49.513 INFO:teuthology.orchestra.run.smithi045.stdout:4160 read 0x25efc0 thru 0x26894e (0x998f bytes) 2024-09-17T07:29:49.517 INFO:teuthology.orchestra.run.smithi045.stdout:4161 trunc from 0x4805d6f to 0x7b3f8f7 2024-09-17T07:29:49.524 INFO:teuthology.orchestra.run.smithi045.stdout:4162 punch from 0x7abd111 to 0x7acbef1, (0xede0 bytes) 2024-09-17T07:29:49.524 INFO:teuthology.orchestra.run.smithi045.stdout:4163 read 0x347785f thru 0x348096b (0x910d bytes) 2024-09-17T07:29:49.524 INFO:teuthology.orchestra.run.smithi045.stdout:4164 punch from 0x40fc4ea to 0x4102fc8, (0x6ade bytes) 2024-09-17T07:29:49.529 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:49.528+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:49.531 INFO:teuthology.orchestra.run.smithi045.stdout:4165 write 0x98af02d thru 0x98b1a48 (0x2a1c bytes) 2024-09-17T07:29:49.534 INFO:teuthology.orchestra.run.smithi045.stdout:4169 read 0xc38b3 thru 0xc5b36 (0x2284 bytes) 2024-09-17T07:29:49.535 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:49.535 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:49.541 INFO:teuthology.orchestra.run.smithi045.stdout:4176 trunc from 0x98b1a49 to 0x3ec6d68 2024-09-17T07:29:49.563 INFO:teuthology.orchestra.run.smithi045.stdout:4177 read 0x139584f thru 0x139f270 (0x9a22 bytes) 2024-09-17T07:29:49.577 INFO:teuthology.orchestra.run.smithi045.stdout:4178 writesame 0x9c5dcd2 thru 0x9c6bb58 data_size 0x11(0xde87 bytes) 2024-09-17T07:29:49.586 INFO:teuthology.orchestra.run.smithi045.stdout:4179 trunc from 0x9c6bb59 to 0x106b445 2024-09-17T07:29:49.629 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:49.629+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:49.647 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:49.648 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:49.648 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:49.648 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:49.648 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:49.657 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:49.658 INFO:teuthology.orchestra.run.smithi045.stdout:4180 writesame 0xac4c5a8 thru 0xac591d7 data_size 0x6c(0xcc30 bytes) 2024-09-17T07:29:49.664 INFO:teuthology.orchestra.run.smithi045.stdout:4181 write 0xb18fd81 thru 0xb19acdd (0xaf5d bytes) 2024-09-17T07:29:49.668 INFO:teuthology.orchestra.run.smithi045.stdout:4184 writesame 0xb12dcac thru 0xb131b7f data_size 0x2(0x3ed4 bytes) 2024-09-17T07:29:49.674 INFO:teuthology.orchestra.run.smithi045.stdout:4185 writesame 0x96715e2 thru 0x96761ac data_size 0x1(0x4bcb bytes) 2024-09-17T07:29:49.687 INFO:teuthology.orchestra.run.smithi045.stdout:4186 write 0x1eb829f thru 0x1eba07c (0x1dde bytes) 2024-09-17T07:29:49.689 INFO:teuthology.orchestra.run.smithi045.stdout:4188 read 0x49fa9d0 thru 0x49fc200 (0x1831 bytes) 2024-09-17T07:29:49.689 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:49.697 INFO:teuthology.orchestra.run.smithi045.stdout:4191 trunc from 0xb19acde to 0x4320999 2024-09-17T07:29:49.721 INFO:teuthology.orchestra.run.smithi045.stdout:4193 write 0xa9cc93d thru 0xa9d3ee6 (0x75aa bytes) 2024-09-17T07:29:49.725 INFO:teuthology.orchestra.run.smithi045.stdout:4195 read 0xf63fd5 thru 0xf65407 (0x1433 bytes) 2024-09-17T07:29:49.725 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:49.730 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:49.730+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:49.732 INFO:teuthology.orchestra.run.smithi045.stdout:4198 write 0xb970881 thru 0xb971d1a (0x149a bytes) 2024-09-17T07:29:49.735 INFO:teuthology.orchestra.run.smithi045.stdout:4199 write 0x1790f53 thru 0x1796a76 (0x5b24 bytes) 2024-09-17T07:29:49.737 INFO:teuthology.orchestra.run.smithi045.stdout:4201 writesame 0x94967b5 thru 0x949f093 data_size 0x1(0x88df bytes) 2024-09-17T07:29:49.756 INFO:teuthology.orchestra.run.smithi045.stdout:4204 read 0xa10d40c thru 0xa112a78 (0x566d bytes) 2024-09-17T07:29:49.756 INFO:teuthology.orchestra.run.smithi045.stdout:4206 write 0xa96271 thru 0xaa3973 (0xd703 bytes) 2024-09-17T07:29:49.758 INFO:teuthology.orchestra.run.smithi045.stdout:4207 read 0xaa71b51 thru 0xaa72aa5 (0xf55 bytes) 2024-09-17T07:29:49.760 INFO:teuthology.orchestra.run.smithi045.stdout:4208 trunc from 0xb971d1b to 0xdc118a6 2024-09-17T07:29:49.766 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:49.767 INFO:teuthology.orchestra.run.smithi045.stdout:4210 write 0x6aa0dcb thru 0x6aa86b9 (0x78ef bytes) 2024-09-17T07:29:49.772 INFO:teuthology.orchestra.run.smithi045.stdout:4212 writesame 0x962198e thru 0x962e4e9 data_size 0x26(0xcb5c bytes) 2024-09-17T07:29:49.773 INFO:teuthology.orchestra.run.smithi045.stdout:4213 read 0x5f2b221 thru 0x5f3a67e (0xf45e bytes) 2024-09-17T07:29:49.773 INFO:teuthology.orchestra.run.smithi045.stdout:4216 write 0x9f2b704 thru 0x9f3615a (0xaa57 bytes) 2024-09-17T07:29:49.774 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:49.774 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:49.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:49.774 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:49.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.775 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:29:49.775 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:49.775 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.438186668, 2024-09-17T07:29:49.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:29:49.776 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.776 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.776 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:49.776 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.776 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:29:49.776 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:29:49.777 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:29:49.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.777 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:49.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:49.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:29:49.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:29:49.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:29:49.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:29:49.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:29:49.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:29:49.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:29:49.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:29:49.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:29:49.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:29:49.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:29:49.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:29:49.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:29:49.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:29:49.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:29:49.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:29:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:29:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:29:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:29:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:29:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:29:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:29:49.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:49.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:29:49.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:29:49.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:49.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:29:49.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:29:49.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:29:49.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:29:49.783 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.783 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.783 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.783 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7378 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7176192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.783 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:49.783 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.849144633999998, 2024-09-17T07:29:49.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073003980999999996, 2024-09-17T07:29:49.783 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.783 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.783 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.783 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.784 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.784 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.784 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7378 2024-09-17T07:29:49.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.784 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:49.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:49.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:49.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1799999999999999e-07 2024-09-17T07:29:49.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923713+0000", 2024-09-17T07:29:49.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.057e-06 2024-09-17T07:29:49.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923716+0000", 2024-09-17T07:29:49.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0599999999999999e-06 2024-09-17T07:29:49.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923719+0000", 2024-09-17T07:29:49.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8219999999999998e-06 2024-09-17T07:29:49.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996401+0000", 2024-09-17T07:29:49.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072681467999999999 2024-09-17T07:29:49.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996452+0000", 2024-09-17T07:29:49.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.134e-05 2024-09-17T07:29:49.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996716+0000", 2024-09-17T07:29:49.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000264016 2024-09-17T07:29:49.787 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.787 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.787 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.788 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.788 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:49.788 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.849134518, 2024-09-17T07:29:49.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:29:49.788 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.788 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.788 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.788 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.788 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.788 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.788 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:29:49.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.788 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:49.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:49.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:29:49.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:29:49.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:29:49.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:29:49.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:29:49.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:29:49.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:29:49.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:29:49.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:29:49.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:29:49.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:29:49.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:29:49.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:29:49.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:29:49.792 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.792 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.792 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.792 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7398 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6389760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.792 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:49.792 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.843865983999997, 2024-09-17T07:29:49.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072891580999999997, 2024-09-17T07:29:49.792 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.793 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.793 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.793 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.793 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.793 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.793 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7398 2024-09-17T07:29:49.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.793 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:49.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:49.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:49.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:49.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928995+0000", 2024-09-17T07:29:49.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0759999999999996e-06 2024-09-17T07:29:49.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928999+0000", 2024-09-17T07:29:49.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1609999999999996e-06 2024-09-17T07:29:49.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929002+0000", 2024-09-17T07:29:49.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8449999999999999e-06 2024-09-17T07:29:49.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001598+0000", 2024-09-17T07:29:49.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072596318000000007 2024-09-17T07:29:49.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001663+0000", 2024-09-17T07:29:49.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4995999999999999e-05 2024-09-17T07:29:49.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001882+0000", 2024-09-17T07:29:49.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021897299999999999 2024-09-17T07:29:49.797 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.797 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.797 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.797 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.797 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:49.797 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.843851401999999, 2024-09-17T07:29:49.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:29:49.797 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.797 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.797 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.798 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:29:49.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.798 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:49.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:49.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:49.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:29:49.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:29:49.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:29:49.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:29:49.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:29:49.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:29:49.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:29:49.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:29:49.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:29:49.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:29:49.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:29:49.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:29:49.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:29:49.801 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.801 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.801 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.802 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.802 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:49.802 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.843820422, 2024-09-17T07:29:49.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:29:49.802 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.802 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.802 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.802 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.802 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.802 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.802 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:29:49.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.803 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:49.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:49.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:29:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:29:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:29:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:29:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:29:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:29:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:29:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:29:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:29:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:29:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:29:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:29:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:29:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:29:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.843790372999997, 2024-09-17T07:29:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:29:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:29:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:29:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:29:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:29:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:29:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:29:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:29:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:29:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:29:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:29:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:29:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:29:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:29:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:29:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:29:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.843761966000002, 2024-09-17T07:29:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:29:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:29:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:29:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:29:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:29:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:29:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:29:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:29:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:29:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:29:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:29:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:29:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:29:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:29:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.843742145, 2024-09-17T07:29:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:29:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:29:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:29:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:29:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:29:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:29:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:29:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:29:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:29:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:29:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:29:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:29:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:29:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:29:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:29:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:29:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.843717892999997, 2024-09-17T07:29:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:29:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:29:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:29:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:29:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:29:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:29:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:29:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:29:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:29:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:29:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:29:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:29:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:29:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:29:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:29:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:29:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7418 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7700480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.838311869999998, 2024-09-17T07:29:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072752965000000003, 2024-09-17T07:29:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7418 2024-09-17T07:29:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3599999999999999e-07 2024-09-17T07:29:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934547+0000", 2024-09-17T07:29:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3350000000000001e-06 2024-09-17T07:29:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934554+0000", 2024-09-17T07:29:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9859999999999997e-06 2024-09-17T07:29:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934557+0000", 2024-09-17T07:29:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.818e-06 2024-09-17T07:29:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007104+0000", 2024-09-17T07:29:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072546779000000006 2024-09-17T07:29:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007147+0000", 2024-09-17T07:29:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2904000000000001e-05 2024-09-17T07:29:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007298+0000", 2024-09-17T07:29:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015080700000000001 2024-09-17T07:29:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.838295305000003, 2024-09-17T07:29:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:29:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:29:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:29:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:29:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:29:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:29:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:29:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:29:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:29:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:29:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:29:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:29:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:29:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:29:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:29:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:29:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.83824843, 2024-09-17T07:29:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:29:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:29:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:29:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:29:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:29:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:29:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:29:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:29:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:29:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:29:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:29:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:29:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:29:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:29:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.838233725999999, 2024-09-17T07:29:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:29:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:29:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:29:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:29:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:29:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:29:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:29:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:29:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:29:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:29:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:29:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:29:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:29:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:29:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.838210631000003, 2024-09-17T07:29:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:29:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:29:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:29:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:29:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:29:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:29:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:29:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:29:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:29:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:29:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:29:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:29:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:29:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:29:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:29:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.838179255999997, 2024-09-17T07:29:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:29:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:29:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:29:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:29:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:29:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:29:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:29:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:29:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:29:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:29:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:29:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:29:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:29:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:29:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:29:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:29:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.838163995999999, 2024-09-17T07:29:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:29:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:29:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:29:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:29:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:29:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:29:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:29:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:29:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:29:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:29:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:29:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:29:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:29:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:29:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:29:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.838143926000001, 2024-09-17T07:29:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:29:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:29:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:29:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:29:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:29:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:29:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:29:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:29:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:29:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:29:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:29:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:29:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:29:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:29:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:29:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.838131986, 2024-09-17T07:29:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:29:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:29:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:29:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:29:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:29:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:29:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:29:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:29:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:29:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:29:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:29:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:29:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:29:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:29:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.838108828000003, 2024-09-17T07:29:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:29:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:29:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:29:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:29:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:29:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:29:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:29:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:29:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:29:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:29:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:29:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:29:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:29:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:29:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:29:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:29:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:49.869 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:49.871 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:49.870+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:49.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:49.873 INFO:teuthology.orchestra.run.smithi045.stdout:4218 punch from 0x7c45a89 to 0x7c482a0, (0x2817 bytes) 2024-09-17T07:29:49.873 INFO:teuthology.orchestra.run.smithi045.stdout:4219 read 0x5694553 thru 0x56a1fcf (0xda7d bytes) 2024-09-17T07:29:49.873 INFO:teuthology.orchestra.run.smithi045.stdout:4220 read 0x8d58751 thru 0x8d62bdd (0xa48d bytes) 2024-09-17T07:29:49.873 INFO:teuthology.orchestra.run.smithi045.stdout:4221 read 0x7dd29ba thru 0x7de2352 (0xf999 bytes) 2024-09-17T07:29:49.874 INFO:teuthology.orchestra.run.smithi045.stdout:4222 write 0xc806be8 thru 0xc80d788 (0x6ba1 bytes) 2024-09-17T07:29:49.874 INFO:teuthology.orchestra.run.smithi045.stdout:4223 punch from 0x3e83ec4 to 0x3e92144, (0xe280 bytes) 2024-09-17T07:29:49.874 INFO:teuthology.orchestra.run.smithi045.stdout:4224 write 0xc97edb7 thru 0xc98b27d (0xc4c7 bytes) 2024-09-17T07:29:49.874 INFO:teuthology.orchestra.run.smithi045.stdout:4225 write 0x23f66b8 thru 0x23fbf82 (0x58cb bytes) 2024-09-17T07:29:49.874 INFO:teuthology.orchestra.run.smithi045.stdout:4227 flatten 2024-09-17T07:29:49.874 INFO:teuthology.orchestra.run.smithi045.stdout:4228 trunc from 0xdc118a6 to 0x9dd6c9d 2024-09-17T07:29:49.874 INFO:teuthology.orchestra.run.smithi045.stdout:4229 trunc from 0x9dd6c9d to 0x2f65f24 2024-09-17T07:29:49.874 INFO:teuthology.orchestra.run.smithi045.stdout:4230 writesame 0x34e56da thru 0x34f1a35 data_size 0x2(0xc35c bytes) 2024-09-17T07:29:49.874 INFO:teuthology.orchestra.run.smithi045.stdout:4233 writesame 0x4c6bd4e thru 0x4c78d0f data_size 0x1d(0xcfc2 bytes) 2024-09-17T07:29:49.874 INFO:teuthology.orchestra.run.smithi045.stdout:4234 read 0x32c22ba thru 0x32c76ef (0x5436 bytes) 2024-09-17T07:29:49.874 INFO:teuthology.orchestra.run.smithi045.stdout:4235 punch from 0x13eee7b to 0x13f3671, (0x47f6 bytes) 2024-09-17T07:29:49.874 INFO:teuthology.orchestra.run.smithi045.stdout:4236 write 0xa9dfad0 thru 0xa9e38fa (0x3e2b bytes) 2024-09-17T07:29:49.874 INFO:teuthology.orchestra.run.smithi045.stdout:4237 writesame 0x57dc099 thru 0x57e38a4 data_size 0x4(0x780c bytes) 2024-09-17T07:29:49.874 INFO:teuthology.orchestra.run.smithi045.stdout:4239 write 0xb2e4f0 thru 0xb30fda (0x2aeb bytes) 2024-09-17T07:29:49.875 INFO:teuthology.orchestra.run.smithi045.stdout:4241 writesame 0x9c7e8a1 thru 0x9c8d329 data_size 0x1(0xea89 bytes) 2024-09-17T07:29:49.912 INFO:teuthology.orchestra.run.smithi045.stdout:4242 writesame 0xc7033f7 thru 0xc708e31 data_size 0x1(0x5a3b bytes) 2024-09-17T07:29:49.912 INFO:teuthology.orchestra.run.smithi045.stdout:4243 read 0xc5ecb6b thru 0xc5fbf27 (0xf3bd bytes) 2024-09-17T07:29:49.912 INFO:teuthology.orchestra.run.smithi045.stdout:4244 writesame 0xda62da thru 0xdb11a0 data_size 0x1(0xaec7 bytes) 2024-09-17T07:29:49.929 INFO:teuthology.orchestra.run.smithi045.stdout:4245 punch from 0x601e671 to 0x6022f89, (0x4918 bytes) 2024-09-17T07:29:49.929 INFO:teuthology.orchestra.run.smithi045.stdout:4246 read 0x4f0eb1 thru 0x4fe1ce (0xd31e bytes) 2024-09-17T07:29:49.929 INFO:teuthology.orchestra.run.smithi045.stdout:4248 write 0x107ed25 thru 0x10851d4 (0x64b0 bytes) 2024-09-17T07:29:49.944 INFO:teuthology.orchestra.run.smithi045.stdout:4249 writesame 0xcadacc0 thru 0xcae9ab6 data_size 0x1(0xedf7 bytes) 2024-09-17T07:29:49.944 INFO:teuthology.orchestra.run.smithi045.stdout:4251 write 0x76cc388 thru 0x76cc9c0 (0x639 bytes) 2024-09-17T07:29:49.947 INFO:teuthology.orchestra.run.smithi045.stdout:4252 write 0xa57ac thru 0xb0716 (0xaf6b bytes) 2024-09-17T07:29:49.949 INFO:teuthology.orchestra.run.smithi045.stdout:4253 trunc from 0xcae9ab7 to 0xd7fd4f6 2024-09-17T07:29:49.962 INFO:teuthology.orchestra.run.smithi045.stdout:4254 trunc from 0xd7fd4f6 to 0x6c5a25f 2024-09-17T07:29:49.971 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:49.971+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:49.990 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:49.990 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:49.990 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:49.990 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:49.994 INFO:teuthology.orchestra.run.smithi045.stdout:4255 write 0xe6edd5a thru 0xe6f2c76 (0x4f1d bytes) 2024-09-17T07:29:49.997 INFO:teuthology.orchestra.run.smithi045.stdout:4256 write 0x1a4e312 thru 0x1a51fb4 (0x3ca3 bytes) 2024-09-17T07:29:49.999 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:50.000 INFO:teuthology.orchestra.run.smithi045.stdout:4258 write 0x3a4b47e thru 0x3a51d5e (0x68e1 bytes) 2024-09-17T07:29:50.001 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:50.001 INFO:teuthology.orchestra.run.smithi045.stdout:4261 write 0x1cdbebb thru 0x1ce1a40 (0x5b86 bytes) 2024-09-17T07:29:50.003 INFO:teuthology.orchestra.run.smithi045.stdout:4262 read 0x7865c04 thru 0x7867604 (0x1a01 bytes) 2024-09-17T07:29:50.003 INFO:teuthology.orchestra.run.smithi045.stdout:4264 write 0xcf49223 thru 0xcf52df8 (0x9bd6 bytes) 2024-09-17T07:29:50.006 INFO:teuthology.orchestra.run.smithi045.stdout:4265 writesame 0x3070c61 thru 0x3078f3d data_size 0xd(0x82dd bytes) 2024-09-17T07:29:50.009 INFO:teuthology.orchestra.run.smithi045.stdout:4267 punch from 0x61a8b92 to 0x61b5c32, (0xd0a0 bytes) 2024-09-17T07:29:50.009 INFO:teuthology.orchestra.run.smithi045.stdout:4270 clone 36 order 19 su 65536 sc 11 2024-09-17T07:29:50.073 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:50.073+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:50.110 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:50.110 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:50.110 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:50.110 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:50.110 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:50.119 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:50.174 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:50.174+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:50.273 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:50.273 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.139560422999999, 2024-09-17T07:29:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:29:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.274 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:29:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:29:50.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:29:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:29:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:29:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:29:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:29:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:29:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:29:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:29:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:29:50.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:29:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:29:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:29:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:29:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:29:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:29:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:29:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:29:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:29:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:29:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:29:50.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:29:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:29:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:29:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.833593719, 2024-09-17T07:29:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:29:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:29:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.281 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:29:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:29:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:29:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:29:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:29:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:29:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:29:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:29:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:29:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:29:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:29:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:29:50.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:29:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:29:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:29:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.833571603999999, 2024-09-17T07:29:50.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:29:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:29:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:29:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:29:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:29:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:29:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:29:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:29:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:29:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:29:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:29:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:29:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:29:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:29:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:29:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.289 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.290 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:50.290 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.833547500000002, 2024-09-17T07:29:50.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:29:50.290 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.290 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.290 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.290 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.290 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.290 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:29:50.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.290 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:50.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:29:50.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:29:50.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:29:50.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:29:50.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:29:50.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:29:50.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:29:50.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:29:50.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:29:50.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:29:50.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:29:50.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:29:50.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:29:50.294 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.294 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.294 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.294 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.294 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:50.294 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.833523411000002, 2024-09-17T07:29:50.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:29:50.294 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.294 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.294 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.295 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.295 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.295 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:29:50.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.295 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:50.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:50.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:29:50.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:29:50.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:29:50.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:29:50.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:29:50.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:29:50.298 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.298 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.298 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.298 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.298 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:50.298 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.833498992000003, 2024-09-17T07:29:50.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:29:50.298 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.298 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.298 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.298 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.298 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.299 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.299 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:29:50.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.299 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:50.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:50.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:29:50.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:29:50.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:29:50.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:29:50.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:29:50.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:29:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:29:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:29:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:29:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:29:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:29:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:29:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.833477291999998, 2024-09-17T07:29:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:29:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.302 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:29:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:50.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:29:50.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:29:50.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:29:50.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:29:50.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:29:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:29:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:29:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:29:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:29:50.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:29:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:29:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:29:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.833432559999999, 2024-09-17T07:29:50.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:29:50.307 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.307 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.307 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.307 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.307 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.307 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:29:50.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.307 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:50.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:29:50.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:29:50.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:29:50.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:29:50.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:29:50.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:29:50.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:29:50.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:29:50.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:29:50.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:29:50.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:29:50.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:29:50.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:29:50.310 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.310 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.310 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.310 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.311 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:50.311 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.833404094000002, 2024-09-17T07:29:50.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:29:50.311 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.311 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.311 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.311 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.311 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.311 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.311 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:29:50.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.311 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:50.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:50.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:29:50.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:29:50.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:29:50.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:29:50.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:29:50.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:29:50.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:29:50.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:29:50.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:29:50.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:29:50.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:29:50.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:29:50.314 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.314 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.314 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.315 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.315 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:50.315 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.833286842, 2024-09-17T07:29:50.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:29:50.315 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.315 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.315 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.315 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.315 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:29:50.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.316 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:50.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:29:50.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:29:50.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:29:50.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:29:50.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:29:50.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:29:50.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:29:50.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:29:50.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:29:50.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:29:50.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:29:50.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:29:50.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:29:50.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.319 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.319 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.319 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.319 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:50.319 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.832860740000001, 2024-09-17T07:29:50.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:29:50.319 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.319 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.319 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.320 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.320 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.320 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:29:50.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.320 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:50.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:29:50.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.321 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:29:50.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:50.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.321 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:29:50.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:29:50.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:29:50.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:29:50.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:29:50.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:29:50.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:29:50.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:29:50.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:29:50.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:29:50.323 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.323 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.323 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.323 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.323 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:50.323 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.832841068999997, 2024-09-17T07:29:50.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:29:50.323 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.324 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.324 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.324 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.324 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.324 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:29:50.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.324 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:50.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:50.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:29:50.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:29:50.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:29:50.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:29:50.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:29:50.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:29:50.327 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.327 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.327 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.327 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.327 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:50.327 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.832818205000002, 2024-09-17T07:29:50.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:29:50.327 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.328 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.328 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.328 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.328 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:29:50.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.328 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:50.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:50.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:29:50.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:29:50.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:29:50.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:29:50.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:29:50.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:29:50.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:29:50.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:29:50.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:29:50.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:29:50.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:29:50.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:29:50.331 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.331 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.331 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.331 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.331 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:50.331 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.832796903999999, 2024-09-17T07:29:50.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:29:50.331 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.332 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.332 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.332 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.332 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.332 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:29:50.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.332 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:50.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.333 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:29:50.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.333 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:29:50.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:29:50.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:29:50.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:29:50.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:29:50.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:29:50.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:29:50.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:29:50.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:29:50.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:29:50.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:29:50.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:29:50.335 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.335 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.336 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.336 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.336 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:50.336 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.832749116000002, 2024-09-17T07:29:50.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:29:50.336 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.336 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.336 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.336 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.336 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.336 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.336 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:29:50.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.337 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:50.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:29:50.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:29:50.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:29:50.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:29:50.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:29:50.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:29:50.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:29:50.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:29:50.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:29:50.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:29:50.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:29:50.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:29:50.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:29:50.340 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.340 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.340 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.340 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.340 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:50.340 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.832721894000002, 2024-09-17T07:29:50.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:29:50.340 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.340 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.340 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.341 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.341 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.341 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.341 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:29:50.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.341 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:50.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:50.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:29:50.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:29:50.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:29:50.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:29:50.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:29:50.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:29:50.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:29:50.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:29:50.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:29:50.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:29:50.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:29:50.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:29:50.344 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.344 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.344 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.344 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.344 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:50.344 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.832701507000003, 2024-09-17T07:29:50.345 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:29:50.345 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.345 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.345 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.345 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.345 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.345 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.345 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:29:50.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.345 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.345 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.345 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:50.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:29:50.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.346 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:29:50.346 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:29:50.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.346 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.346 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.347 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:29:50.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:29:50.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.347 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.347 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:29:50.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:29:50.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.347 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.347 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:29:50.347 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:29:50.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.347 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:29:50.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:29:50.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.348 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.348 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:29:50.348 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:29:50.348 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.348 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.348 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.348 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.349 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.349 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:50.349 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.832681802000003, 2024-09-17T07:29:50.349 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:29:50.349 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.349 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.349 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.349 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.349 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.349 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.349 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:29:50.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.349 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:50.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:29:50.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.350 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.350 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.350 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:29:50.350 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:29:50.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.351 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.351 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:29:50.351 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:29:50.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.351 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.351 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:29:50.351 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:29:50.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.351 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.351 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.351 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:29:50.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:29:50.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:29:50.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:29:50.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.352 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.352 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.352 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:29:50.352 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:29:50.352 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.352 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.353 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.353 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.353 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.353 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:50.353 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.832658723999998, 2024-09-17T07:29:50.353 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:29:50.353 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.353 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.353 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.353 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.353 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.354 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.354 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:29:50.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.354 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:50.354 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.354 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.354 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:29:50.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.355 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:29:50.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:29:50.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.355 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.355 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:29:50.355 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:29:50.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.355 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.356 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:29:50.356 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:29:50.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.356 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.356 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:29:50.356 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:29:50.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.356 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.356 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:29:50.356 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:29:50.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.356 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.357 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.357 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:29:50.357 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:29:50.357 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.357 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.357 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.357 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.357 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:29:50.357 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:50.357 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.832638162000002, 2024-09-17T07:29:50.357 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:29:50.357 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.358 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:29:50.358 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.358 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.358 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.358 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:29:50.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.358 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.358 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.358 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:50.358 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.358 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.359 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.359 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:50.359 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.359 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.359 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:29:50.359 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:29:50.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.359 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.359 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.359 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:29:50.359 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:29:50.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.360 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.360 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:29:50.360 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:29:50.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.360 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.360 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:29:50.360 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:29:50.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.360 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.360 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.360 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:29:50.361 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:29:50.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.361 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.361 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.361 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:29:50.361 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:29:50.361 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.361 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.361 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.361 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.361 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.361 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:50.363 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:50.364 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-17T07:29:50.364 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:50.362+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:50.435 INFO:teuthology.orchestra.run.smithi045.stdout:4272 read 0x9954317 thru 0x9962e48 (0xeb32 bytes) 2024-09-17T07:29:50.435 INFO:teuthology.orchestra.run.smithi045.stdout:4273 read 0x31a496d thru 0x31acb1c (0x81b0 bytes) 2024-09-17T07:29:50.436 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:50.436 INFO:teuthology.orchestra.run.smithi045.stdout:4275 read 0xdf9028d thru 0xdf9e0cf (0xde43 bytes) 2024-09-17T07:29:50.437 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:50.437 INFO:teuthology.orchestra.run.smithi045.stdout:4277 write 0x57831ed thru 0x57922c5 (0xf0d9 bytes) 2024-09-17T07:29:50.445 INFO:teuthology.orchestra.run.smithi045.stdout:4278 write 0x1dd759f thru 0x1ddaeee (0x3950 bytes) 2024-09-17T07:29:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:4279 read 0x755bab8 thru 0x7568b39 (0xd082 bytes) 2024-09-17T07:29:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:4280 write 0x63dacf4 thru 0x63e4e61 (0xa16e bytes) 2024-09-17T07:29:50.461 INFO:teuthology.orchestra.run.smithi045.stdout:4283 trunc from 0xe6f2c77 to 0x6080396 2024-09-17T07:29:50.463 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:50.463+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:50.485 INFO:teuthology.orchestra.run.smithi045.stdout:4284 writesame 0x453cd3f thru 0x454c4cd data_size 0x41(0xf78f bytes) 2024-09-17T07:29:50.489 INFO:teuthology.orchestra.run.smithi045.stdout:4287 punch from 0x573e4e4 to 0x5744c90, (0x67ac bytes) 2024-09-17T07:29:50.489 INFO:teuthology.orchestra.run.smithi045.stdout:4288 read 0x3288157 thru 0x32915e2 (0x948c bytes) 2024-09-17T07:29:50.490 INFO:teuthology.orchestra.run.smithi045.stdout:4290 read 0x252997a thru 0x2532740 (0x8dc7 bytes) 2024-09-17T07:29:50.490 INFO:teuthology.orchestra.run.smithi045.stdout:4292 write 0x3d530da thru 0x3d61e2d (0xed54 bytes) 2024-09-17T07:29:50.503 INFO:teuthology.orchestra.run.smithi045.stdout:4293 trunc from 0x6080396 to 0xdd5283f 2024-09-17T07:29:50.508 INFO:teuthology.orchestra.run.smithi045.stdout:4294 read 0xb479228 thru 0xb486dea (0xdbc3 bytes) 2024-09-17T07:29:50.508 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:50.509 INFO:teuthology.orchestra.run.smithi045.stdout:4297 trunc from 0xdd5283f to 0xe3dcfe6 2024-09-17T07:29:50.519 INFO:teuthology.orchestra.run.smithi045.stdout:4298 trunc from 0xe3dcfe6 to 0x96f5711 2024-09-17T07:29:50.524 INFO:teuthology.orchestra.run.smithi045.stdout:4301 write 0x3b6712f thru 0x3b73d91 (0xcc63 bytes) 2024-09-17T07:29:50.526 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:50.526 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:50.526 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:50.526 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:50.531 INFO:teuthology.orchestra.run.smithi045.stdout:4302 read 0x5ab933e thru 0x5ac7cc1 (0xe984 bytes) 2024-09-17T07:29:50.531 INFO:teuthology.orchestra.run.smithi045.stdout:4304 punch from 0x4b89949 to 0x4b8fdff, (0x64b6 bytes) 2024-09-17T07:29:50.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:50.540 INFO:teuthology.orchestra.run.smithi045.stdout:4305 write 0xbd1db2d thru 0xbd28daf (0xb283 bytes) 2024-09-17T07:29:50.544 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:50.553 INFO:teuthology.orchestra.run.smithi045.stdout:4307 write 0xede652a thru 0xedf0307 (0x9dde bytes) 2024-09-17T07:29:50.559 INFO:teuthology.orchestra.run.smithi045.stdout:4308 read 0x4a90964 thru 0x4a9d15f (0xc7fc bytes) 2024-09-17T07:29:50.560 INFO:teuthology.orchestra.run.smithi045.stdout:4310 read 0x688d8db thru 0x68931db (0x5901 bytes) 2024-09-17T07:29:50.565 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:50.564+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:50.571 INFO:teuthology.orchestra.run.smithi045.stdout:4311 trunc from 0xedf0308 to 0x4543d72 2024-09-17T07:29:50.605 INFO:teuthology.orchestra.run.smithi045.stdout:4312 writesame 0x58012f6 thru 0x580d33a data_size 0x3(0xc045 bytes) 2024-09-17T07:29:50.605 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:50.607 INFO:teuthology.orchestra.run.smithi045.stdout:4314 trunc from 0x580d33b to 0x7a983bb 2024-09-17T07:29:50.612 INFO:teuthology.orchestra.run.smithi045.stdout:4315 clone 37 order 22 su 16384 sc 8 2024-09-17T07:29:50.646 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-17T07:29:50.646 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:29:50.646 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-17T07:29:50.646 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T07:29:50.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T07:29:50.649 INFO:tasks.ceph.osd.3:Started 2024-09-17T07:29:50.649 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:29:50.659 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:50.659 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:50.659 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:50.659 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:50.660 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:50.666 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:50.666+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:50.670 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:50.752 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:50.753 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:50.753 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:50.753 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:50.753 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:50.753 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:50.753 INFO:teuthology.orchestra.run.smithi045.stderr:config diff 2024-09-17T07:29:50.753 INFO:teuthology.orchestra.run.smithi045.stderr:config diff get 2024-09-17T07:29:50.753 INFO:teuthology.orchestra.run.smithi045.stderr:config get 2024-09-17T07:29:50.753 INFO:teuthology.orchestra.run.smithi045.stderr:config help [] 2024-09-17T07:29:50.753 INFO:teuthology.orchestra.run.smithi045.stderr:config set ... 2024-09-17T07:29:50.753 INFO:teuthology.orchestra.run.smithi045.stderr:admin_socket: invalid command 2024-09-17T07:29:50.754 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:50.755 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-17T07:29:50.796 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:50.796+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:50.821 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:50.821 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:50.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:50.821 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:50.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.821 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.821 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:50.821 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.686261547000001, 2024-09-17T07:29:50.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:29:50.822 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.822 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.822 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.822 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.822 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:29:50.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.822 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:50.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:50.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:29:50.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:29:50.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:29:50.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:29:50.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:29:50.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:29:50.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:29:50.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:29:50.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:29:50.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:29:50.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:29:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:29:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:29:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:29:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:29:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:29:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:29:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:29:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:29:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:29:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:29:50.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:29:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.68622465, 2024-09-17T07:29:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:29:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.828 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:29:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:29:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:29:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:29:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:29:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:29:50.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:29:50.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:29:50.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:29:50.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:29:50.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:29:50.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:50.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:29:50.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:29:50.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:29:50.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:29:50.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:29:50.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:29:50.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:29:50.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:29:50.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:29:50.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:29:50.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:29:50.834 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.834 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.834 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.834 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.834 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:50.834 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.686198683000001, 2024-09-17T07:29:50.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:29:50.834 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.835 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.835 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.835 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.835 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.835 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:29:50.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.835 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:50.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:50.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:29:50.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:50.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:29:50.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:29:50.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:29:50.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:29:50.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:29:50.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:29:50.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:29:50.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:29:50.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:29:50.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:29:50.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:29:50.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:29:50.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:29:50.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:29:50.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:29:50.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:29:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:29:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:29:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:29:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:29:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.840 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.686174088000001, 2024-09-17T07:29:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:29:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.841 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:29:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:50.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:29:50.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:29:50.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:29:50.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:29:50.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:29:50.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:29:50.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:29:50.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:29:50.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:29:50.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:29:50.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:29:50.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:29:50.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:29:50.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:29:50.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:29:50.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:29:50.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:29:50.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:29:50.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:29:50.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:29:50.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:29:50.847 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.847 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.847 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.847 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.847 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:50.847 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.686150601000001, 2024-09-17T07:29:50.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:29:50.847 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.847 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.847 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.847 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.848 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.848 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.848 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:29:50.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.848 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:50.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:50.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:29:50.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:50.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:29:50.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:29:50.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:29:50.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:29:50.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:29:50.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:29:50.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:29:50.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:29:50.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:50.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:29:50.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:29:50.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:29:50.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:29:50.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:29:50.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:29:50.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:29:50.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:29:50.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:29:50.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:29:50.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:29:50.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.853 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.853 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.853 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:50.853 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.686136863000002, 2024-09-17T07:29:50.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:29:50.853 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.853 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.853 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.854 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.854 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.854 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.854 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:29:50.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.854 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:50.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:50.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:50.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:29:50.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:29:50.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:29:50.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:29:50.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:29:50.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:29:50.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:29:50.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:50.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:29:50.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:50.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:29:50.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:29:50.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:29:50.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:29:50.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:29:50.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:29:50.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:29:50.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:29:50.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:29:50.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:29:50.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:29:50.859 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.859 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.859 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.859 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.859 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:50.859 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.686106674000001, 2024-09-17T07:29:50.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:29:50.859 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:29:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:50.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:29:50.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:29:50.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:29:50.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:29:50.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:50.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:29:50.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:29:50.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:29:50.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:29:50.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:29:50.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:29:50.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:29:50.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:29:50.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:29:50.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:29:50.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:50.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:29:50.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:29:50.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:29:50.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:29:50.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:29:50.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:29:50.865 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.865 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.865 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.865 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.865 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:50.865 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.686070168999997, 2024-09-17T07:29:50.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:29:50.865 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.866 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.866 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.866 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.866 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.866 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:29:50.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.866 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:50.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:50.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:50.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:29:50.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:29:50.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:29:50.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:29:50.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:29:50.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:29:50.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:29:50.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:29:50.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:29:50.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:50.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:29:50.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:29:50.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:29:50.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:29:50.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:29:50.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:29:50.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:29:50.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:29:50.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:29:50.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:29:50.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:29:50.873 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.873 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.873 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.873 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.873 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:50.873 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.686042409999999, 2024-09-17T07:29:50.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:29:50.873 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.874 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.874 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.874 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.874 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:29:50.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.874 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:50.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:50.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:29:50.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:29:50.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:29:50.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:29:50.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:29:50.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:29:50.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:29:50.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:29:50.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:50.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:29:50.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:29:50.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:29:50.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:29:50.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:29:50.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:29:50.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:50.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:29:50.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:29:50.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:29:50.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:29:50.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:29:50.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:29:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.686020249999999, 2024-09-17T07:29:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:29:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.879 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:29:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:50.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:29:50.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:29:50.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:29:50.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:29:50.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:50.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:29:50.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:29:50.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:29:50.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:29:50.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:29:50.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:29:50.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:29:50.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:29:50.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:29:50.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:29:50.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:29:50.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:29:50.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:29:50.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:29:50.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:29:50.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:29:50.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.885 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.885 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.885 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:50.885 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.686007089, 2024-09-17T07:29:50.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:29:50.885 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.885 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.886 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.886 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.886 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.886 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:29:50.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.886 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:50.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:50.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:50.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:29:50.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:29:50.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:29:50.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:29:50.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:29:50.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:29:50.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:29:50.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:29:50.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:29:50.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:50.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:29:50.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:29:50.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:29:50.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:29:50.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:29:50.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:29:50.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:29:50.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:29:50.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:29:50.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.685988500999997, 2024-09-17T07:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.891 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:29:50.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:29:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:29:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:29:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:29:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:29:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:29:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:29:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:29:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.68595569, 2024-09-17T07:29:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:29:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.897 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:29:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:50.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.899 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:50.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:29:50.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.899 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:29:50.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:29:50.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:29:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:29:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:29:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:29:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:29:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:29:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:29:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:29:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:29:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:29:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:29:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:29:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:29:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:29:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:29:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.685936632999997, 2024-09-17T07:29:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:29:50.903 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:29:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:29:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:29:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:29:50.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:29:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:29:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:29:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:29:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:29:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:29:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:29:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:29:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:29:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:29:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:29:50.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:29:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:29:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:29:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:29:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:29:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:29:50.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:29:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:29:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.685914523999998, 2024-09-17T07:29:50.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:29:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:29:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:29:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:29:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:29:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:29:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:29:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:29:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:29:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.912 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:29:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:29:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:29:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:29:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:29:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:29:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.685888417000001, 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.915 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:29:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.916 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:29:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:29:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:29:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:29:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:29:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:29:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:29:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:29:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:29:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:29:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:29:50.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:29:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:29:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:29:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:29:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:29:50.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:29:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:29:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:29:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:29:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:29:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.685859272000002, 2024-09-17T07:29:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:29:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:29:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:50.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:29:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:29:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:29:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:29:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:29:50.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:29:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:29:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:29:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:29:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:29:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:29:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:29:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:29:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:29:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:29:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:29:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:29:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:29:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:29:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:29:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:50.926 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.685827469000003, 2024-09-17T07:29:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:29:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:29:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.927 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:29:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:29:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.928 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:29:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:29:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:29:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:29:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:29:50.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:29:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:29:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:29:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:29:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:29:50.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:29:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:29:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:29:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:29:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:29:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:29:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:29:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:29:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:29:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.932 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.685800522000001, 2024-09-17T07:29:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:29:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:29:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.933 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:29:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:29:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:29:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:29:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:29:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:29:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:29:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:29:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:29:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:29:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:29:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:29:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:29:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:29:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:29:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:29:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:29:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:29:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:29:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:29:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:50.938 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 52.685759572999999, 2024-09-17T07:29:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:29:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:29:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:50.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:29:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:29:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:50.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:29:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:29:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:29:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:29:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:29:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:29:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:29:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:29:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:29:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:29:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:29:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:29:50.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:29:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:29:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:29:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:29:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:29:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:29:50.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:29:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:29:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:50.944 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:50.944 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:50.946 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:50.946 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:50.945+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:51.046 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:51.046+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:51.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:29:51.094 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:51.094 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:51.094 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:51.094 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:51.106 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:51.107 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T07:29:51.148 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:51.147+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:51.169 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone36 from 0x7a983bb (overlap 0x4543d72) to 0x1bb4bc7 2024-09-17T07:29:51.249 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:51.249+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:51.255 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:51.255 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:51.278 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:51.293 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:51.293 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:51.293 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:51.293 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:51.293 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:51.298 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:51.298 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:51.300 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:51.304 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:51.319 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:51.319 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:51.325 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-17T07:29:51.337 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:51.337 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:51.352 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:51.351+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:51.356 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:51.356 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:51.373 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:51.374 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:51.377 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:51.378 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:51.428 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:51.428 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:51.438 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:51.481 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:51.481+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:51.541 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:51.541 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:51.549 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:51.554 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:51.553+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:51.611 INFO:teuthology.orchestra.run.smithi045.stdout:4316 writesame 0x87bac20 thru 0x87c4006 data_size 0x15(0x93e7 bytes) 2024-09-17T07:29:51.620 INFO:teuthology.orchestra.run.smithi045.stdout:4318 writesame 0xbcaef86 thru 0xbcb7b90 data_size 0x1(0x8c0b bytes) 2024-09-17T07:29:51.621 INFO:teuthology.orchestra.run.smithi045.stdout:4321 write 0x5097624 thru 0x50a53c7 (0xdda4 bytes) 2024-09-17T07:29:51.640 INFO:teuthology.orchestra.run.smithi045.stdout:4322 writesame 0x6396200 thru 0x639a5df data_size 0x8(0x43e0 bytes) 2024-09-17T07:29:51.650 INFO:teuthology.orchestra.run.smithi045.stdout:4324 trunc from 0xbcb7b91 to 0x69ed598 2024-09-17T07:29:51.655 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:51.654+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:51.679 INFO:teuthology.orchestra.run.smithi045.stdout:4325 write 0xc2b7134 thru 0xc2bb6e3 (0x45b0 bytes) 2024-09-17T07:29:51.683 INFO:teuthology.orchestra.run.smithi045.stdout:4326 write 0xb54a831 thru 0xb54b510 (0xce0 bytes) 2024-09-17T07:29:51.686 INFO:teuthology.orchestra.run.smithi045.stdout:4327 read 0x69647d8 thru 0x6968c00 (0x4429 bytes) 2024-09-17T07:29:51.686 INFO:teuthology.orchestra.run.smithi045.stdout:4328 read 0x77b1bed thru 0x77b8c40 (0x7054 bytes) 2024-09-17T07:29:51.686 INFO:teuthology.orchestra.run.smithi045.stdout:4329 writesame 0x1684c32 thru 0x1691b7f data_size 0xae(0xcf4e bytes) 2024-09-17T07:29:51.718 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:51.718 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:51.731 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:51.756 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:51.755+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:51.768 INFO:teuthology.orchestra.run.smithi045.stdout:4330 read 0x3a3fcc7 thru 0x3a440b7 (0x43f1 bytes) 2024-09-17T07:29:51.769 INFO:teuthology.orchestra.run.smithi045.stdout:4332 write 0x7dcde20 thru 0x7dd99d1 (0xbbb2 bytes) 2024-09-17T07:29:51.771 INFO:teuthology.orchestra.run.smithi045.stdout:4334 read 0x7c2830c thru 0x7c2bb9a (0x388f bytes) 2024-09-17T07:29:51.771 INFO:teuthology.orchestra.run.smithi045.stdout:4335 read 0x9fc838a thru 0x9fcd709 (0x5380 bytes) 2024-09-17T07:29:51.772 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:51.772 INFO:teuthology.orchestra.run.smithi045.stdout:4340 write 0x29c72ee thru 0x29d0583 (0x9296 bytes) 2024-09-17T07:29:51.839 INFO:teuthology.orchestra.run.smithi045.stdout:4341 write 0xb40d6f thru 0xb4c8c7 (0xbb59 bytes) 2024-09-17T07:29:51.857 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:51.856+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:51.869 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:51.869 INFO:teuthology.orchestra.run.smithi045.stdout:4343 writesame 0x939857b thru 0x93a427f data_size 0x35(0xbd05 bytes) 2024-09-17T07:29:51.878 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:51.884 INFO:teuthology.orchestra.run.smithi045.stdout:4348 trunc from 0xc2bb6e4 to 0x745334c 2024-09-17T07:29:51.900 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:51.900 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:51.902 INFO:teuthology.orchestra.run.smithi045.stdout:4349 read 0x3681d54 thru 0x3686362 (0x460f bytes) 2024-09-17T07:29:51.902 INFO:teuthology.orchestra.run.smithi045.stdout:4350 punch from 0x1efec10 to 0x1f04911, (0x5d01 bytes) 2024-09-17T07:29:51.910 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:51.911 INFO:teuthology.orchestra.run.smithi045.stdout:4353 writesame 0x93d64a8 thru 0x93e2246 data_size 0xb(0xbd9f bytes) 2024-09-17T07:29:51.919 INFO:teuthology.orchestra.run.smithi045.stdout:4354 write 0xaff8a23 thru 0xb0034a1 (0xaa7f bytes) 2024-09-17T07:29:51.931 INFO:teuthology.orchestra.run.smithi045.stdout:4355 write 0xcb9c0cd thru 0xcbaa9f5 (0xe929 bytes) 2024-09-17T07:29:51.943 INFO:teuthology.orchestra.run.smithi045.stdout:4356 trunc from 0xcbaa9f6 to 0x7ba457b 2024-09-17T07:29:51.957 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:51.957+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:51.963 INFO:teuthology.orchestra.run.smithi045.stdout:4359 read 0x1bacdbe thru 0x1badfee (0x1231 bytes) 2024-09-17T07:29:51.975 INFO:teuthology.orchestra.run.smithi045.stdout:4360 write 0xcd3f9e8 thru 0xcd4ec47 (0xf260 bytes) 2024-09-17T07:29:51.981 INFO:teuthology.orchestra.run.smithi045.stdout:4361 punch from 0x4e3844b to 0x4e39651, (0x1206 bytes) 2024-09-17T07:29:51.981 INFO:teuthology.orchestra.run.smithi045.stdout:4362 write 0x930cf8a thru 0x9313702 (0x6779 bytes) 2024-09-17T07:29:51.985 INFO:teuthology.orchestra.run.smithi045.stdout:4363 writesame 0x122cf8a thru 0x122dc4b data_size 0x17(0xcc2 bytes) 2024-09-17T07:29:51.987 INFO:teuthology.orchestra.run.smithi045.stdout:4365 write 0xbb80bd1 thru 0xbb86a03 (0x5e33 bytes) 2024-09-17T07:29:51.991 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:51.991 INFO:teuthology.orchestra.run.smithi045.stdout:4367 write 0x428fe3 thru 0x42f82d (0x684b bytes) 2024-09-17T07:29:51.992 INFO:teuthology.orchestra.run.smithi045.stdout:4368 read 0xab2cc3f thru 0xab38e4f (0xc211 bytes) 2024-09-17T07:29:51.992 INFO:teuthology.orchestra.run.smithi045.stdout:4370 punch from 0x85fd361 to 0x860205e, (0x4cfd bytes) 2024-09-17T07:29:51.992 INFO:teuthology.orchestra.run.smithi045.stdout:4372 writesame 0x6110910 thru 0x6116f6b data_size 0x2(0x665c bytes) 2024-09-17T07:29:51.996 INFO:teuthology.orchestra.run.smithi045.stdout:4373 read 0xb99fc84 thru 0xb9abe21 (0xc19e bytes) 2024-09-17T07:29:51.997 INFO:teuthology.orchestra.run.smithi045.stdout:4374 trunc from 0xcd4ec48 to 0xcf1c4a2 2024-09-17T07:29:52.003 INFO:teuthology.orchestra.run.smithi045.stdout:4375 read 0x2f0f4ae thru 0x2f13e32 (0x4985 bytes) 2024-09-17T07:29:52.003 INFO:teuthology.orchestra.run.smithi045.stdout:4376 read 0xaad02e0 thru 0xaad6806 (0x6527 bytes) 2024-09-17T07:29:52.010 INFO:teuthology.orchestra.run.smithi045.stdout:4377 write 0xea44ba4 thru 0xea481c9 (0x3626 bytes) 2024-09-17T07:29:52.020 INFO:teuthology.orchestra.run.smithi045.stdout:4378 trunc from 0xea481ca to 0x970a4bd 2024-09-17T07:29:52.021 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:52.021 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:52.031 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:52.043 INFO:teuthology.orchestra.run.smithi045.stdout:4379 trunc from 0x970a4bd to 0x1e01593 2024-09-17T07:29:52.072 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:52.072+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:52.103 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:29:52.102+0000 7fef8efa6640 -1 Falling back to public interface 2024-09-17T07:29:52.107 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-17T07:29:52.117 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:52.117 INFO:teuthology.orchestra.run.smithi045.stdout:4383 read 0x1794ee2 thru 0x179b067 (0x6186 bytes) 2024-09-17T07:29:52.118 INFO:teuthology.orchestra.run.smithi045.stdout:4384 punch from 0xf7a3ff to 0xf87fb4, (0xdbb5 bytes) 2024-09-17T07:29:52.118 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:52.124 INFO:teuthology.orchestra.run.smithi045.stdout:4387 writesame 0x29cd12a thru 0x29d60d0 data_size 0x5(0x8fa7 bytes) 2024-09-17T07:29:52.139 INFO:teuthology.orchestra.run.smithi045.stdout:4388 writesame 0xaf60bbb thru 0xaf6ea1d data_size 0x7(0xde63 bytes) 2024-09-17T07:29:52.150 INFO:teuthology.orchestra.run.smithi045.stdout:4389 writesame 0xea242c6 thru 0xea25cea data_size 0x17(0x1a25 bytes) 2024-09-17T07:29:52.150 INFO:teuthology.orchestra.run.smithi045.stdout:4390 read 0xa11c8c9 thru 0xa124e82 (0x85ba bytes) 2024-09-17T07:29:52.150 INFO:teuthology.orchestra.run.smithi045.stdout:4391 read 0x930bddf thru 0x9312813 (0x6a35 bytes) 2024-09-17T07:29:52.150 INFO:teuthology.orchestra.run.smithi045.stdout:4392 punch from 0x101904 to 0x109fd0, (0x86cc bytes) 2024-09-17T07:29:52.150 INFO:teuthology.orchestra.run.smithi045.stdout:4393 read 0xe046058 thru 0xe04d480 (0x7429 bytes) 2024-09-17T07:29:52.150 INFO:teuthology.orchestra.run.smithi045.stdout:4394 writesame 0xc3ce5c2 thru 0xc3d53e8 data_size 0x1(0x6e27 bytes) 2024-09-17T07:29:52.159 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:52.159+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:52.162 INFO:teuthology.orchestra.run.smithi045.stdout:4395 punch from 0xa349166 to 0xa34eb66, (0x5a00 bytes) 2024-09-17T07:29:52.162 INFO:teuthology.orchestra.run.smithi045.stdout:4400 writesame 0xa0963e2 thru 0xa096fea data_size 0xd(0xc09 bytes) 2024-09-17T07:29:52.165 INFO:teuthology.orchestra.run.smithi045.stdout:4402 punch from 0x6dbbcbe to 0x6dc4a99, (0x8ddb bytes) 2024-09-17T07:29:52.165 INFO:teuthology.orchestra.run.smithi045.stdout:4403 read 0x3b4b0e1 thru 0x3b4ce39 (0x1d59 bytes) 2024-09-17T07:29:52.165 INFO:teuthology.orchestra.run.smithi045.stdout:4404 writesame 0xc06590b thru 0xc066c11 data_size 0x1(0x1307 bytes) 2024-09-17T07:29:52.168 INFO:teuthology.orchestra.run.smithi045.stdout:4406 writesame 0x4800f4c thru 0x4804be1 data_size 0x6e(0x3c96 bytes) 2024-09-17T07:29:52.175 INFO:teuthology.orchestra.run.smithi045.stdout:4407 read 0xa79f7 thru 0xb008d (0x8697 bytes) 2024-09-17T07:29:52.176 INFO:teuthology.orchestra.run.smithi045.stdout:4408 write 0xccf5e98 thru 0xccfa952 (0x4abb bytes) 2024-09-17T07:29:52.178 INFO:teuthology.orchestra.run.smithi045.stdout:4409 read 0x3f5468 thru 0x40374c (0xe2e5 bytes) 2024-09-17T07:29:52.179 INFO:teuthology.orchestra.run.smithi045.stdout:4410 punch from 0x6ee7164 to 0x6eea0ba, (0x2f56 bytes) 2024-09-17T07:29:52.186 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:52.186 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:52.192 INFO:teuthology.orchestra.run.smithi045.stdout:4411 trunc from 0xea25ceb to 0x322efab 2024-09-17T07:29:52.195 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:52.214 INFO:teuthology.orchestra.run.smithi045.stdout:4414 write 0x6655909 thru 0x66589a1 (0x3099 bytes) 2024-09-17T07:29:52.217 INFO:teuthology.orchestra.run.smithi045.stdout:4415 read 0x77d896 thru 0x77ee74 (0x15df bytes) 2024-09-17T07:29:52.225 INFO:teuthology.orchestra.run.smithi045.stdout:4416 trunc from 0x66589a2 to 0x1dba202 2024-09-17T07:29:52.254 INFO:teuthology.orchestra.run.smithi045.stdout:4417 write 0x9debee3 thru 0x9df780e (0xb92c bytes) 2024-09-17T07:29:52.261 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:52.260+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:52.270 INFO:teuthology.orchestra.run.smithi045.stdout:4419 trunc from 0x9df780f to 0x1ee0dad 2024-09-17T07:29:52.294 INFO:teuthology.orchestra.run.smithi045.stdout:4420 writesame 0x5391421 thru 0x539bfd8 data_size 0x8c(0xabb8 bytes) 2024-09-17T07:29:52.294 INFO:teuthology.orchestra.run.smithi045.stdout:4421 write 0x458d217 thru 0x4596348 (0x9132 bytes) 2024-09-17T07:29:52.305 INFO:teuthology.orchestra.run.smithi045.stdout:4422 writesame 0x6a14953 thru 0x6a1bd78 data_size 0x1(0x7426 bytes) 2024-09-17T07:29:52.305 INFO:teuthology.orchestra.run.smithi045.stdout:4423 clone 38 order 18 su 32768 sc 5 2024-09-17T07:29:52.306 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:52.306 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:52.315 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:52.361 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:52.361+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:52.417 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:52.417 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:52.429 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:52.473 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:52.473+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:52.532 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:52.532 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:52.541 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:52.582 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:52.581+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:52.664 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:52.664+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:52.683 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:52.683 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:52.692 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:52.766 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:52.765+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:52.837 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:52.837 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:52.845 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:52.866 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:52.866+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:52.948 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:52.948 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:52.957 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:53.000 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:53.000+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:53.060 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:53.060 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:53.067 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:53.067+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:53.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:53.168 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:53.167+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:53.179 INFO:teuthology.orchestra.run.smithi045.stdout:flattening image image_client.0-clone37 2024-09-17T07:29:53.195 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:53.195 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:53.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:53.268 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:53.268+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:53.277 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-17T07:29:53.326 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:53.327 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:53.336 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:53.342 INFO:teuthology.orchestra.run.smithi045.stdout:4424 write 0x656f96d thru 0x6576f19 (0x75ad bytes) 2024-09-17T07:29:53.350 INFO:teuthology.orchestra.run.smithi045.stdout:4427 punch from 0x47f6226 to 0x47f7ea8, (0x1c82 bytes) 2024-09-17T07:29:53.350 INFO:teuthology.orchestra.run.smithi045.stdout:4429 read 0x2efbdc4 thru 0x2f08dee (0xd02b bytes) 2024-09-17T07:29:53.356 INFO:teuthology.orchestra.run.smithi045.stdout:4430 write 0x7355bda thru 0x7364771 (0xeb98 bytes) 2024-09-17T07:29:53.361 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.361 INFO:teuthology.orchestra.run.smithi045.stdout:4433 read 0x34f0a76 thru 0x34f7c88 (0x7213 bytes) 2024-09-17T07:29:53.362 INFO:teuthology.orchestra.run.smithi045.stdout:4435 punch from 0x418eb0f to 0x419bbca, (0xd0bb bytes) 2024-09-17T07:29:53.362 INFO:teuthology.orchestra.run.smithi045.stdout:4436 read 0x61348b6 thru 0x6138412 (0x3b5d bytes) 2024-09-17T07:29:53.362 INFO:teuthology.orchestra.run.smithi045.stdout:4437 punch from 0x12f11c9 to 0x12fd715, (0xc54c bytes) 2024-09-17T07:29:53.362 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.369 INFO:teuthology.orchestra.run.smithi045.stdout:4439 trunc from 0x7364772 to 0xe422021 2024-09-17T07:29:53.369 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:53.369+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:53.374 INFO:teuthology.orchestra.run.smithi045.stdout:4440 writesame 0x4246814 thru 0x4256644 data_size 0x6d(0xfe31 bytes) 2024-09-17T07:29:53.379 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:29:53.385 INFO:teuthology.orchestra.run.smithi045.stdout:4442 write 0x8531a26 thru 0x853d092 (0xb66d bytes) 2024-09-17T07:29:53.395 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.395 INFO:teuthology.orchestra.run.smithi045.stdout:4444 punch from 0x5380d79 to 0x5387c6c, (0x6ef3 bytes) 2024-09-17T07:29:53.395 INFO:teuthology.orchestra.run.smithi045.stdout:4445 punch from 0x668cc54 to 0x669b3fa, (0xe7a6 bytes) 2024-09-17T07:29:53.396 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.396 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.396 INFO:teuthology.orchestra.run.smithi045.stdout:4449 read 0xd5d349c thru 0xd5df837 (0xc39c bytes) 2024-09-17T07:29:53.396 INFO:teuthology.orchestra.run.smithi045.stdout:4451 write 0x35d05b thru 0x369b5a (0xcb00 bytes) 2024-09-17T07:29:53.405 INFO:teuthology.orchestra.run.smithi045.stdout:4452 writesame 0x9b6a7f6 thru 0x9b75028 data_size 0x1f(0xa833 bytes) 2024-09-17T07:29:53.410 INFO:teuthology.orchestra.run.smithi045.stdout:4453 punch from 0xd8847b to 0xd8c48f, (0x4014 bytes) 2024-09-17T07:29:53.410 INFO:teuthology.orchestra.run.smithi045.stdout:4454 read 0x5c32bb2 thru 0x5c3d95e (0xadad bytes) 2024-09-17T07:29:53.411 INFO:teuthology.orchestra.run.smithi045.stdout:4455 write 0xaf71717 thru 0xaf7acfe (0x95e8 bytes) 2024-09-17T07:29:53.415 INFO:teuthology.orchestra.run.smithi045.stdout:4456 write 0xdcf80a9 thru 0xdcffb92 (0x7aea bytes) 2024-09-17T07:29:53.419 INFO:teuthology.orchestra.run.smithi045.stdout:4457 write 0xb0e24dc thru 0xb0ede16 (0xb93b bytes) 2024-09-17T07:29:53.436 INFO:teuthology.orchestra.run.smithi045.stdout:4458 trunc from 0xe422021 to 0x1df7440 2024-09-17T07:29:53.459 INFO:teuthology.orchestra.run.smithi045.stdout:4459 punch from 0x958571 to 0x96755e, (0xefed bytes) 2024-09-17T07:29:53.459 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.459 INFO:teuthology.orchestra.run.smithi045.stdout:4463 punch from 0x52a225 to 0x531f63, (0x7d3e bytes) 2024-09-17T07:29:53.469 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:53.469+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:53.470 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:53.470 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:53.473 INFO:teuthology.orchestra.run.smithi045.stdout:4464 write 0x959ea8f thru 0x95a79e5 (0x8f57 bytes) 2024-09-17T07:29:53.477 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.477 INFO:teuthology.orchestra.run.smithi045.stdout:4466 punch from 0x201ab43 to 0x2028c0a, (0xe0c7 bytes) 2024-09-17T07:29:53.482 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:53.489 INFO:teuthology.orchestra.run.smithi045.stdout:4467 write 0xe2e5ddd thru 0xe2e85ff (0x2823 bytes) 2024-09-17T07:29:53.493 INFO:teuthology.orchestra.run.smithi045.stdout:4468 read 0x1909a88 thru 0x191234d (0x88c6 bytes) 2024-09-17T07:29:53.493 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.493 INFO:teuthology.orchestra.run.smithi045.stdout:4472 write 0x239d567 thru 0x23abc52 (0xe6ec bytes) 2024-09-17T07:29:53.498 INFO:teuthology.orchestra.run.smithi045.stdout:4474 write 0xf1d233 thru 0xf28b3e (0xb90c bytes) 2024-09-17T07:29:53.507 INFO:teuthology.orchestra.run.smithi045.stdout:4476 write 0x8e805e4 thru 0x8e8525a (0x4c77 bytes) 2024-09-17T07:29:53.510 INFO:teuthology.orchestra.run.smithi045.stdout:4477 read 0x3f141f5 thru 0x3f241a7 (0xffb3 bytes) 2024-09-17T07:29:53.510 INFO:teuthology.orchestra.run.smithi045.stdout:4478 read 0x6af6b4 thru 0x6b8c1b (0x9568 bytes) 2024-09-17T07:29:53.511 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.511 INFO:teuthology.orchestra.run.smithi045.stdout:4480 write 0x27dd231 thru 0x27ec2e2 (0xf0b2 bytes) 2024-09-17T07:29:53.531 INFO:teuthology.orchestra.run.smithi045.stdout:4481 trunc from 0xe2e8600 to 0x45a083f 2024-09-17T07:29:53.544 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:53.544 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:53.553 INFO:teuthology.orchestra.run.smithi045.stdout:4482 write 0x9bd3e39 thru 0x9bdb59c (0x7764 bytes) 2024-09-17T07:29:53.564 INFO:teuthology.orchestra.run.smithi045.stdout:4484 write 0xacfd9b3 thru 0xad0c63e (0xec8c bytes) 2024-09-17T07:29:53.569 INFO:teuthology.orchestra.run.smithi045.stdout:4485 read 0x2b3d7fb thru 0x2b420af (0x48b5 bytes) 2024-09-17T07:29:53.569 INFO:teuthology.orchestra.run.smithi045.stdout:4486 punch from 0x962028a to 0x9621496, (0x120c bytes) 2024-09-17T07:29:53.570 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:53.570+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:53.573 INFO:teuthology.orchestra.run.smithi045.stdout:4487 trunc from 0xad0c63f to 0xed4ad61 2024-09-17T07:29:53.579 INFO:teuthology.orchestra.run.smithi045.stdout:4488 writesame 0xe511939 thru 0xe514d17 data_size 0x7(0x33df bytes) 2024-09-17T07:29:53.580 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:53.580 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:53.583 INFO:teuthology.orchestra.run.smithi045.stdout:4489 write 0x6bacd4c thru 0x6bb3370 (0x6625 bytes) 2024-09-17T07:29:53.587 INFO:teuthology.orchestra.run.smithi045.stdout:4490 writesame 0xae0c986 thru 0xae0f94b data_size 0x2(0x2fc6 bytes) 2024-09-17T07:29:53.593 INFO:teuthology.orchestra.run.smithi045.stdout:4491 read 0xda6b7dd thru 0xda779e7 (0xc20b bytes) 2024-09-17T07:29:53.593 INFO:teuthology.orchestra.run.smithi045.stdout:4492 writesame 0x7ef988a thru 0x7f030dc data_size 0xb(0x9853 bytes) 2024-09-17T07:29:53.598 INFO:teuthology.orchestra.run.smithi045.stdout:4493 write 0xcca2c39 thru 0xccad672 (0xaa3a bytes) 2024-09-17T07:29:53.600 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:53.601 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:53.601 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:53.603 INFO:teuthology.orchestra.run.smithi045.stdout:4494 read 0x6b4f697 thru 0x6b59d29 (0xa693 bytes) 2024-09-17T07:29:53.603 INFO:teuthology.orchestra.run.smithi045.stdout:4495 write 0xdddeb5 thru 0xde9523 (0xb66f bytes) 2024-09-17T07:29:53.611 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.611 INFO:teuthology.orchestra.run.smithi045.stdout:4497 punch from 0x1ce9a to 0x2455e, (0x76c4 bytes) 2024-09-17T07:29:53.611 INFO:teuthology.orchestra.run.smithi045.stdout:4498 read 0x8922f8c thru 0x892717d (0x41f2 bytes) 2024-09-17T07:29:53.611 INFO:teuthology.orchestra.run.smithi045.stdout:4499 write 0xe6cfe4a thru 0xe6da556 (0xa70d bytes) 2024-09-17T07:29:53.617 INFO:teuthology.orchestra.run.smithi045.stdout:4500 write 0x1b30fa9 thru 0x1b3da31 (0xca89 bytes) 2024-09-17T07:29:53.623 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.623 INFO:teuthology.orchestra.run.smithi045.stdout:4502 writesame 0x9dfab1f thru 0x9e02213 data_size 0x1(0x76f5 bytes) 2024-09-17T07:29:53.626 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:53.626 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:53.637 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:53.637 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:53.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.637 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:101472 3.8 3:171b27c2:::rbd_data.11831ce727ee.0000000000000276:head [set-alloc-hint object_size 262144 write_size 262144,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-17T07:29:53.637 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:53.625878+0000", 2024-09-17T07:29:53.637 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.011166938, 2024-09-17T07:29:53.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0111877, 2024-09-17T07:29:53.637 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:53.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "waiting for sub ops", 2024-09-17T07:29:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 101472 2024-09-17T07:29:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:53.625878+0000", 2024-09-17T07:29:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:53.625878+0000", 2024-09-17T07:29:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:53.625879+0000", 2024-09-17T07:29:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4600000000000004e-07 2024-09-17T07:29:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:53.625883+0000", 2024-09-17T07:29:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.532e-06 2024-09-17T07:29:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:53.625896+0000", 2024-09-17T07:29:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.217e-05 2024-09-17T07:29:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:53.625902+0000", 2024-09-17T07:29:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:29:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:53.625963+0000", 2024-09-17T07:29:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0479999999999997e-05 2024-09-17T07:29:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:53.626043+0000", 2024-09-17T07:29:53.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0517000000000002e-05 2024-09-17T07:29:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:53.628234+0000", 2024-09-17T07:29:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021905240000000001 2024-09-17T07:29:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.642 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop_reply(client.4483.0:101472 3.8 e86/76)", 2024-09-17T07:29:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:53.634670+0000", 2024-09-17T07:29:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0023753070000000001, 2024-09-17T07:29:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024347959999999999, 2024-09-17T07:29:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "queued for pg", 2024-09-17T07:29:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:53.634670+0000", 2024-09-17T07:29:53.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:53.634670+0000", 2024-09-17T07:29:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:53.634671+0000", 2024-09-17T07:29:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.11e-06 2024-09-17T07:29:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:53.634676+0000", 2024-09-17T07:29:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.741e-06 2024-09-17T07:29:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:53.634692+0000", 2024-09-17T07:29:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5889999999999999e-05 2024-09-17T07:29:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:53.634705+0000", 2024-09-17T07:29:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3845e-05 2024-09-17T07:29:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:29:53.646 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 2 2024-09-17T07:29:53.646 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:53.647 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.647 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.647 INFO:teuthology.orchestra.run.smithi045.stdout:4508 read 0x9d51992 thru 0x9d5b2d0 (0x993f bytes) 2024-09-17T07:29:53.647 INFO:teuthology.orchestra.run.smithi045.stdout:4509 writesame 0x5f05446 thru 0x5f08b6e data_size 0x9(0x3729 bytes) 2024-09-17T07:29:53.647 INFO:teuthology.orchestra.run.smithi045.stdout:4510 write 0x2f9ae1c thru 0x2f9d4ae (0x2693 bytes) 2024-09-17T07:29:53.647 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.647 INFO:teuthology.orchestra.run.smithi045.stdout:4513 writesame 0x2cd1209 thru 0x2cd1ccb data_size 0x1(0xac3 bytes) 2024-09-17T07:29:53.648 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:53.648 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:53.649 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:53.650 INFO:teuthology.orchestra.run.smithi045.stdout:4514 writesame 0xb510e93 thru 0xb51985d data_size 0x53(0x89cb bytes) 2024-09-17T07:29:53.653 INFO:teuthology.orchestra.run.smithi045.stdout:4515 read 0x9f4d437 thru 0x9f59bc1 (0xc78b bytes) 2024-09-17T07:29:53.653 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.653 INFO:teuthology.orchestra.run.smithi045.stdout:4518 writesame 0x647ad5d thru 0x647cf5b data_size 0x3(0x21ff bytes) 2024-09-17T07:29:53.657 INFO:teuthology.orchestra.run.smithi045.stdout:4519 punch from 0x17154a8 to 0x17235c2, (0xe11a bytes) 2024-09-17T07:29:53.657 INFO:teuthology.orchestra.run.smithi045.stdout:4520 writesame 0x21bd6d0 thru 0x21c81d9 data_size 0x1(0xab0a bytes) 2024-09-17T07:29:53.669 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:53.671 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:53.671+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:53.677 INFO:teuthology.orchestra.run.smithi045.stdout:4521 write 0xcf4eedd thru 0xcf5c0c8 (0xd1ec bytes) 2024-09-17T07:29:53.684 INFO:teuthology.orchestra.run.smithi045.stdout:4523 writesame 0x39aa818 thru 0x39ada70 data_size 0x1(0x3259 bytes) 2024-09-17T07:29:53.691 INFO:teuthology.orchestra.run.smithi045.stdout:4527 writesame 0xe801015 thru 0xe801324 data_size 0x10(0x310 bytes) 2024-09-17T07:29:53.692 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:53.692 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:4529 write 0x42f0933 thru 0x42f6517 (0x5be5 bytes) 2024-09-17T07:29:53.696 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:53.696 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:53.699 INFO:teuthology.orchestra.run.smithi045.stdout:4532 read 0x6e9042 thru 0x6f3108 (0xa0c7 bytes) 2024-09-17T07:29:53.699 INFO:teuthology.orchestra.run.smithi045.stdout:4533 read 0xec6434 thru 0xec914d (0x2d1a bytes) 2024-09-17T07:29:53.701 INFO:teuthology.orchestra.run.smithi045.stdout:4535 punch from 0x34381d8 to 0x344309e, (0xaec6 bytes) 2024-09-17T07:29:53.702 INFO:teuthology.orchestra.run.smithi045.stdout:4536 read 0xe4400b8 thru 0xe440ac7 (0xa10 bytes) 2024-09-17T07:29:53.702 INFO:teuthology.orchestra.run.smithi045.stdout:4537 read 0x9933c19 thru 0x993a905 (0x6ced bytes) 2024-09-17T07:29:53.702 INFO:teuthology.orchestra.run.smithi045.stdout:4539 writesame 0x705aae5 thru 0x70648c8 data_size 0x2f(0x9de4 bytes) 2024-09-17T07:29:53.706 INFO:teuthology.orchestra.run.smithi045.stdout:4540 writesame 0x904e2f5 thru 0x905d820 data_size 0x47(0xf52c bytes) 2024-09-17T07:29:53.712 INFO:teuthology.orchestra.run.smithi045.stdout:4541 trunc from 0xed4ad61 to 0xdf5e90e 2024-09-17T07:29:53.722 INFO:teuthology.orchestra.run.smithi045.stdout:4542 write 0xdcac4c9 thru 0xdcb5159 (0x8c91 bytes) 2024-09-17T07:29:53.726 INFO:teuthology.orchestra.run.smithi045.stdout:4543 trunc from 0xdf5e90e to 0xd45f82a 2024-09-17T07:29:53.736 INFO:teuthology.orchestra.run.smithi045.stdout:4545 read 0x1745023 thru 0x174635e (0x133c bytes) 2024-09-17T07:29:53.736 INFO:teuthology.orchestra.run.smithi045.stdout:4546 read 0x7df37bd thru 0x7df7f00 (0x4744 bytes) 2024-09-17T07:29:53.736 INFO:teuthology.orchestra.run.smithi045.stdout:4547 write 0x9ba9ecf thru 0x9baf8d8 (0x5a0a bytes) 2024-09-17T07:29:53.738 INFO:teuthology.orchestra.run.smithi045.stdout:4548 write 0x1ba364b thru 0x1bade71 (0xa827 bytes) 2024-09-17T07:29:53.746 INFO:teuthology.orchestra.run.smithi045.stdout:4550 read 0xbd46c71 thru 0xbd54491 (0xd821 bytes) 2024-09-17T07:29:53.746 INFO:teuthology.orchestra.run.smithi045.stdout:4551 punch from 0x8403036 to 0x840a39f, (0x7369 bytes) 2024-09-17T07:29:53.746 INFO:teuthology.orchestra.run.smithi045.stdout:4552 write 0x832ff74 thru 0x833beb1 (0xbf3e bytes) 2024-09-17T07:29:53.754 INFO:teuthology.orchestra.run.smithi045.stdout:4553 read 0xc4dc3be thru 0xc4dd8ad (0x14f0 bytes) 2024-09-17T07:29:53.754 INFO:teuthology.orchestra.run.smithi045.stdout:4555 writesame 0xa5f6789 thru 0xa5faf51 data_size 0x1(0x47c9 bytes) 2024-09-17T07:29:53.762 INFO:teuthology.orchestra.run.smithi045.stdout:4557 write 0xb5a5559 thru 0xb5b0267 (0xad0f bytes) 2024-09-17T07:29:53.773 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:53.772+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:53.773 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.773 INFO:teuthology.orchestra.run.smithi045.stdout:4559 writesame 0xadac709 thru 0xadb3ace data_size 0x49(0x73c6 bytes) 2024-09-17T07:29:53.777 INFO:teuthology.orchestra.run.smithi045.stdout:4560 writesame 0xce13ffa thru 0xce1bd7d data_size 0x1d(0x7d84 bytes) 2024-09-17T07:29:53.779 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:53.780 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:53.780 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:53.781 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.792 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:53.793 INFO:teuthology.orchestra.run.smithi045.stdout:4563 trunc from 0xd45f82a to 0x3ca28dc 2024-09-17T07:29:53.835 INFO:teuthology.orchestra.run.smithi045.stdout:4564 write 0xcd33056 thru 0xcd425f1 (0xf59c bytes) 2024-09-17T07:29:53.840 INFO:teuthology.orchestra.run.smithi045.stdout:4565 write 0x23e8386 thru 0x23f3179 (0xadf4 bytes) 2024-09-17T07:29:53.845 INFO:teuthology.orchestra.run.smithi045.stdout:4566 read 0x48694a4 thru 0x486a330 (0xe8d bytes) 2024-09-17T07:29:53.845 INFO:teuthology.orchestra.run.smithi045.stdout:4567 writesame 0x4787c24 thru 0x478de76 data_size 0x1(0x6253 bytes) 2024-09-17T07:29:53.861 INFO:teuthology.orchestra.run.smithi045.stdout:4570 read 0xae4eb8c thru 0xae5a619 (0xba8e bytes) 2024-09-17T07:29:53.861 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:53.862 INFO:teuthology.orchestra.run.smithi045.stdout:4572 write 0x99778cf thru 0x997df12 (0x6644 bytes) 2024-09-17T07:29:53.865 INFO:teuthology.orchestra.run.smithi045.stdout:4573 punch from 0x355d669 to 0x3569a70, (0xc407 bytes) 2024-09-17T07:29:53.865 INFO:teuthology.orchestra.run.smithi045.stdout:4575 punch from 0x153dd74 to 0x154d729, (0xf9b5 bytes) 2024-09-17T07:29:53.866 INFO:teuthology.orchestra.run.smithi045.stdout:4577 punch from 0xbecbd82 to 0xbecfa11, (0x3c8f bytes) 2024-09-17T07:29:53.866 INFO:teuthology.orchestra.run.smithi045.stdout:4578 read 0x99a760d thru 0x99b0ff0 (0x99e4 bytes) 2024-09-17T07:29:53.866 INFO:teuthology.orchestra.run.smithi045.stdout:4579 write 0x1abe0c7 thru 0x1ac34ee (0x5428 bytes) 2024-09-17T07:29:53.867 INFO:teuthology.orchestra.run.smithi045.stdout:4580 write 0x7086458 thru 0x70953d0 (0xef79 bytes) 2024-09-17T07:29:53.874 INFO:teuthology.orchestra.run.smithi045.stdout:4581 read 0x7fc8917 thru 0x7fcb4b2 (0x2b9c bytes) 2024-09-17T07:29:53.874 INFO:teuthology.orchestra.run.smithi045.stdout:4582 writesame 0x8292ac9 thru 0x8299ff2 data_size 0x3(0x752a bytes) 2024-09-17T07:29:53.874 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:53.873+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:53.882 INFO:teuthology.orchestra.run.smithi045.stdout:4583 trunc from 0xcd425f2 to 0xb4d5ff4 2024-09-17T07:29:53.894 INFO:teuthology.orchestra.run.smithi045.stdout:4584 read 0x790a342 thru 0x790ae01 (0xac0 bytes) 2024-09-17T07:29:53.895 INFO:teuthology.orchestra.run.smithi045.stdout:4585 read 0x1d79e46 thru 0x1d7e3e5 (0x45a0 bytes) 2024-09-17T07:29:53.904 INFO:teuthology.orchestra.run.smithi045.stdout:4587 writesame 0xd683a44 thru 0xd68515a data_size 0x1(0x1717 bytes) 2024-09-17T07:29:53.904 INFO:teuthology.orchestra.run.smithi045.stdout:4588 write 0x1c73562 thru 0x1c7de28 (0xa8c7 bytes) 2024-09-17T07:29:53.909 INFO:teuthology.orchestra.run.smithi045.stdout:4590 read 0x7b18b5d thru 0x7b275d1 (0xea75 bytes) 2024-09-17T07:29:53.909 INFO:teuthology.orchestra.run.smithi045.stdout:4591 writesame 0xe993f thru 0xf1f47 data_size 0x1(0x8609 bytes) 2024-09-17T07:29:53.921 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:53.921 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786391408999997, 2024-09-17T07:29:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:29:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:29:53.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:29:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:29:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:29:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:29:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:29:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:29:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:29:53.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:29:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:29:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:29:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:29:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:29:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:29:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:29:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:29:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:29:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:29:53.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:29:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:29:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:29:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:29:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:29:53.927 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:29:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786354512000003, 2024-09-17T07:29:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:29:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:53.928 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:29:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:53.929 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:29:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:29:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:29:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:29:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:29:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:29:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:29:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:29:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:29:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:29:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:29:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:29:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:29:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:29:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:29:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:29:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:29:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:29:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:29:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:29:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786328545000003, 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:53.934 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:29:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:29:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:29:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:29:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:29:53.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:29:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:29:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:29:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:29:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:29:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:29:53.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:29:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:29:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:29:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:29:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:29:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:29:53.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:29:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:29:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:29:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:29:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:29:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:29:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786303949999997, 2024-09-17T07:29:53.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:29:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:29:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:53.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:29:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:29:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:53.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:29:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:29:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:29:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:29:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:29:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:29:53.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:29:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:29:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:29:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:29:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:29:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:29:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:29:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:29:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:29:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:29:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.945 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:29:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:29:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:29:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:29:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.946 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786280462999997, 2024-09-17T07:29:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:29:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:29:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:53.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:29:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:29:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:53.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:29:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:29:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:29:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:29:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:53.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:29:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:29:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:29:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:29:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:29:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:29:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:29:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:29:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:29:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:29:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:29:53.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:29:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:29:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:29:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:29:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786266724999997, 2024-09-17T07:29:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:29:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:29:53.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:29:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:29:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:29:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:29:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:29:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:29:53.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:29:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:29:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:29:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:53.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:29:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:29:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:29:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:29:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:29:53.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:29:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:29:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:29:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:29:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:29:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:29:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786236535999997, 2024-09-17T07:29:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:29:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:53.959 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:29:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:29:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:29:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:29:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:53.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:29:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:29:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:29:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:29:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:29:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:29:53.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:29:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:29:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:29:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:29:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:29:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:29:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:29:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:29:53.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:29:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:29:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:29:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:29:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:29:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.965 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786200031, 2024-09-17T07:29:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:29:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:53.966 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:29:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:53.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:29:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:29:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:29:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:29:53.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:29:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:29:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:29:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:29:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:29:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:29:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:29:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:29:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:29:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:29:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:29:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:29:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:29:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:29:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:29:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:29:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:29:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:53.972 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786172272000002, 2024-09-17T07:29:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:29:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:29:53.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:29:53.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:29:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:29:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:29:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:29:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:29:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:29:53.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:29:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:29:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:29:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:29:53.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:29:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:29:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:29:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:29:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:29:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:29:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:29:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:29:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:53.978 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786150112000001, 2024-09-17T07:29:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:29:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:29:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:53.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:29:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:29:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:29:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:29:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:29:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:29:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:29:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:29:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:29:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:29:53.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:53.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:29:53.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:29:53.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:29:53.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:29:53.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:53.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:29:53.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:29:53.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.984 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.984 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:29:53.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:29:53.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.984 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:53.984 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:29:53.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:29:53.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.984 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:53.984 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:29:53.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:29:53.985 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.985 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:53.985 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.985 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:53.985 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:53.985 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786136951000003, 2024-09-17T07:29:53.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:29:53.985 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:53.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:53.985 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:53.986 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:53.986 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:53.986 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:53.986 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:29:53.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.986 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:53.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:53.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:53.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:53.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:53.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:53.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:29:53.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:29:53.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:53.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:29:53.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:29:53.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:53.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:29:53.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:29:53.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:53.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:29:53.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:29:53.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:29:53.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:29:53.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:53.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:53.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:29:53.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:29:53.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:29:53.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:53.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:29:53.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:29:53.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:29:53.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:29:53.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:53.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:29:53.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:29:53.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:53.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:29:53.991 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:29:53.991 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.991 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:53.991 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.991 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:53.991 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:53.991 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786118363, 2024-09-17T07:29:53.991 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:29:53.991 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:53.991 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:53.991 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:53.991 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:53.992 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:53.992 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:53.992 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:29:53.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.992 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:53.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:53.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:53.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:53.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:29:53.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:53.993 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:29:53.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:29:53.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:53.993 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:29:53.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:29:53.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:53.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:29:53.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:29:53.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:53.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:29:53.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:29:53.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:53.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:29:53.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:53.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:53.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:29:53.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:29:53.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:29:53.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:53.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:29:53.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:29:53.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:53.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:29:53.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:29:53.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:53.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:29:53.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:29:53.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:53.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:29:53.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:29:53.997 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.997 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:53.997 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:53.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.997 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:53.997 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:53.997 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786085552000003, 2024-09-17T07:29:53.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:29:53.997 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:53.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:53.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:53.998 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:53.998 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:53.998 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:53.998 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:29:53.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.998 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:53.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:53.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:53.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:53.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:53.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:53.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:53.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:29:53.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:29:53.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:53.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:29:53.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:29:53.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:53.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:53.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:29:54.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:54.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:29:54.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:29:54.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:29:54.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:29:54.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:54.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:54.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:54.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:29:54.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:29:54.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:54.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:29:54.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:29:54.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:29:54.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:29:54.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:29:54.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:29:54.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.003 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:29:54.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:29:54.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.003 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.003 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:54.003 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:54.003 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786066495, 2024-09-17T07:29:54.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:29:54.003 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.003 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:54.004 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:54.004 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:54.004 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:54.004 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:29:54.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.004 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.004 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.004 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:54.004 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.004 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.004 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:29:54.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:29:54.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:29:54.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:29:54.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:29:54.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:29:54.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:29:54.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:29:54.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:29:54.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:29:54.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:29:54.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:54.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:29:54.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:29:54.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:29:54.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:29:54.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:54.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:29:54.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:29:54.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:29:54.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:29:54.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:29:54.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:29:54.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:29:54.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:29:54.009 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.009 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.009 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.009 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:54.009 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:54.009 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786044386, 2024-09-17T07:29:54.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:29:54.009 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.010 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.010 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:54.010 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:54.010 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:54.010 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:54.010 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:29:54.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.010 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.010 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.010 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:54.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:29:54.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:29:54.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:29:54.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:29:54.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:29:54.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:29:54.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:29:54.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:29:54.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:29:54.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:29:54.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:29:54.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:54.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:29:54.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:29:54.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:29:54.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:29:54.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:54.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:29:54.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:29:54.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:29:54.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:29:54.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:29:54.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:29:54.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.015 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.015 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:29:54.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:29:54.015 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.015 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.015 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.015 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:54.015 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:54.015 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.786018278999997, 2024-09-17T07:29:54.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:29:54.015 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.016 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:54.016 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:54.016 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:54.016 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:54.016 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:29:54.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.016 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:54.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:29:54.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:29:54.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:29:54.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:29:54.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:29:54.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:29:54.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:29:54.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:29:54.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:29:54.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:29:54.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:29:54.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:54.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:29:54.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:29:54.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:29:54.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:29:54.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:54.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:29:54.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:29:54.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:29:54.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:29:54.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:29:54.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:29:54.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:29:54.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:29:54.021 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.021 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.021 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.021 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:54.021 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:54.021 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.785989133999998, 2024-09-17T07:29:54.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:29:54.022 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.022 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:54.022 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:54.022 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:54.022 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:54.022 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:29:54.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.022 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:54.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:54.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:29:54.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:29:54.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:29:54.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:29:54.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:29:54.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:29:54.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:29:54.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:29:54.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:29:54.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:29:54.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:54.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:29:54.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:29:54.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:29:54.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:29:54.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.026 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:54.026 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:29:54.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:29:54.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.026 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.026 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:29:54.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:29:54.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.026 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.026 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:29:54.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:29:54.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:29:54.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:29:54.027 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.027 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.027 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.027 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:54.027 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:54.027 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.785957330999999, 2024-09-17T07:29:54.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:29:54.028 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.028 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.028 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:54.028 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:54.028 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:54.028 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:54.028 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:29:54.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.028 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:54.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.029 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.029 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:29:54.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.029 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.029 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:29:54.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:29:54.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.030 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.030 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:29:54.030 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:29:54.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.030 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.030 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:29:54.030 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:29:54.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.030 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.030 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:29:54.031 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:29:54.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.031 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.031 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:29:54.031 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:29:54.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.031 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:54.031 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:29:54.031 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:29:54.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.032 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.032 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:29:54.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:29:54.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.032 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:54.032 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:29:54.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:29:54.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:29:54.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:29:54.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:29:54.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:29:54.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:29:54.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:29:54.034 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.034 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.034 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.034 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:54.034 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:54.034 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.785930383999997, 2024-09-17T07:29:54.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:29:54.034 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.035 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:54.035 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:54.035 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:54.035 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:54.035 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:29:54.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.035 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.035 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.035 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:54.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.035 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:29:54.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:29:54.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:29:54.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:29:54.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:29:54.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:29:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:29:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:29:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:29:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:29:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:29:54.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:54.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:29:54.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:29:54.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:29:54.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:29:54.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:54.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:29:54.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:29:54.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.039 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.039 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:29:54.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:29:54.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.039 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.039 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:29:54.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:29:54.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.039 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:29:54.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:29:54.040 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.040 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.040 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.040 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:29:54.040 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:54.040 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.785889435000001, 2024-09-17T07:29:54.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:29:54.040 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.040 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:54.041 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:29:54.041 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:29:54.041 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:54.041 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:29:54.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.041 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.041 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.041 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:54.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.041 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.041 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:29:54.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.042 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:29:54.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:29:54.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.042 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:29:54.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:29:54.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:29:54.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:29:54.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:29:54.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:29:54.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:29:54.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:29:54.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:29:54.044 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:29:54.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:29:54.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.044 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:29:54.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:29:54.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:29:54.044 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:29:54.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:29:54.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.045 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:29:54.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:29:54.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.045 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:29:54.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:29:54.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:29:54.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:29:54.046 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.046 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.046 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.046 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.046 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.046 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:54.048 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:54.051 INFO:teuthology.orchestra.run.smithi045.stdout:4593 trunc from 0xd68515b to 0x4064c60 2024-09-17T07:29:54.051 INFO:teuthology.orchestra.run.smithi045.stdout:4594 trunc from 0x4064c60 to 0xa1919b4 2024-09-17T07:29:54.051 INFO:teuthology.orchestra.run.smithi045.stdout:4595 write 0x3f2a5f6 thru 0x3f39ae5 (0xf4f0 bytes) 2024-09-17T07:29:54.051 INFO:teuthology.orchestra.run.smithi045.stdout:4596 write 0x75fd94d thru 0x760b08b (0xd73f bytes) 2024-09-17T07:29:54.051 INFO:teuthology.orchestra.run.smithi045.stdout:4598 read 0x4c0d07e thru 0x4c13da2 (0x6d25 bytes) 2024-09-17T07:29:54.051 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.051 INFO:teuthology.orchestra.run.smithi045.stdout:4602 read 0x91f00ed thru 0x91f2fd7 (0x2eeb bytes) 2024-09-17T07:29:54.051 INFO:teuthology.orchestra.run.smithi045.stdout:4606 read 0x19e0a3f thru 0x19e813a (0x76fc bytes) 2024-09-17T07:29:54.052 INFO:teuthology.orchestra.run.smithi045.stdout:4607 write 0x490f771 thru 0x491eecc (0xf75c bytes) 2024-09-17T07:29:54.052 INFO:teuthology.orchestra.run.smithi045.stdout:4608 read 0x8fd7358 thru 0x8fe54a8 (0xe151 bytes) 2024-09-17T07:29:54.052 INFO:teuthology.orchestra.run.smithi045.stdout:4610 read 0x81a2935 thru 0x81ab609 (0x8cd5 bytes) 2024-09-17T07:29:54.052 INFO:teuthology.orchestra.run.smithi045.stdout:4611 read 0x79f5d2e thru 0x79f9703 (0x39d6 bytes) 2024-09-17T07:29:54.052 INFO:teuthology.orchestra.run.smithi045.stdout:4612 write 0x80723bc thru 0x8077f7b (0x5bc0 bytes) 2024-09-17T07:29:54.052 INFO:teuthology.orchestra.run.smithi045.stdout:4613 writesame 0x6e3725b thru 0x6e431ba data_size 0x10(0xbf60 bytes) 2024-09-17T07:29:54.052 INFO:teuthology.orchestra.run.smithi045.stdout:4616 punch from 0x4759771 to 0x475ee02, (0x5691 bytes) 2024-09-17T07:29:54.052 INFO:teuthology.orchestra.run.smithi045.stdout:4618 writesame 0xcec7e5c thru 0xcecec69 data_size 0x1(0x6e0e bytes) 2024-09-17T07:29:54.052 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.052 INFO:teuthology.orchestra.run.smithi045.stdout:4621 punch from 0x655e61b to 0x6565e54, (0x7839 bytes) 2024-09-17T07:29:54.052 INFO:teuthology.orchestra.run.smithi045.stdout:4622 read 0x8b24d0 thru 0x8c12b3 (0xede4 bytes) 2024-09-17T07:29:54.052 INFO:teuthology.orchestra.run.smithi045.stdout:4623 punch from 0xbe9d3fe to 0xbea98bf, (0xc4c1 bytes) 2024-09-17T07:29:54.053 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.053 INFO:teuthology.orchestra.run.smithi045.stdout:4626 writesame 0xd2191d thru 0xd28288 data_size 0x34(0x696c bytes) 2024-09-17T07:29:54.053 INFO:teuthology.orchestra.run.smithi045.stdout:4627 write 0x30ce3e thru 0x3105bc (0x377f bytes) 2024-09-17T07:29:54.053 INFO:teuthology.orchestra.run.smithi045.stdout:4628 read 0x3c220 thru 0x49575 (0xd356 bytes) 2024-09-17T07:29:54.053 INFO:teuthology.orchestra.run.smithi045.stdout:4629 punch from 0x96843a8 to 0x969360a, (0xf262 bytes) 2024-09-17T07:29:54.053 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.053 INFO:teuthology.orchestra.run.smithi045.stdout:4632 punch from 0x6edb20c to 0x6edc39e, (0x1192 bytes) 2024-09-17T07:29:54.053 INFO:teuthology.orchestra.run.smithi045.stdout:4633 trunc from 0xcecec6a to 0x76afeea 2024-09-17T07:29:54.053 INFO:teuthology.orchestra.run.smithi045.stdout:4634 write 0x6fdd43f thru 0x6fe793b (0xa4fd bytes) 2024-09-17T07:29:54.053 INFO:teuthology.orchestra.run.smithi045.stdout:4636 writesame 0x37bd8f5 thru 0x37c1f53 data_size 0x5(0x465f bytes) 2024-09-17T07:29:54.053 INFO:teuthology.orchestra.run.smithi045.stdout:4637 write 0x278000b thru 0x278b90b (0xb901 bytes) 2024-09-17T07:29:54.053 INFO:teuthology.orchestra.run.smithi045.stdout:4638 read 0x3371f39 thru 0x337f8d6 (0xd99e bytes) 2024-09-17T07:29:54.053 INFO:teuthology.orchestra.run.smithi045.stdout:4639 punch from 0x569b0f to 0x56bf73, (0x2464 bytes) 2024-09-17T07:29:54.053 INFO:teuthology.orchestra.run.smithi045.stdout:4642 punch from 0x2fa5547 to 0x2fa8fbf, (0x3a78 bytes) 2024-09-17T07:29:54.053 INFO:teuthology.orchestra.run.smithi045.stdout:4643 read 0x10a22bb thru 0x10b14f8 (0xf23e bytes) 2024-09-17T07:29:54.054 INFO:teuthology.orchestra.run.smithi045.stdout:4644 write 0x9ad74a1 thru 0x9ae64e9 (0xf049 bytes) 2024-09-17T07:29:54.054 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:54.047+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:54.060 INFO:teuthology.orchestra.run.smithi045.stdout:4646 write 0xcf18630 thru 0xcf284e0 (0xfeb1 bytes) 2024-09-17T07:29:54.066 INFO:teuthology.orchestra.run.smithi045.stdout:4647 read 0x9cbdf12 thru 0x9cc65aa (0x8699 bytes) 2024-09-17T07:29:54.067 INFO:teuthology.orchestra.run.smithi045.stdout:4650 trunc from 0xcf284e1 to 0xc36171c 2024-09-17T07:29:54.077 INFO:teuthology.orchestra.run.smithi045.stdout:4651 writesame 0x8f0e4ac thru 0x8f10b06 data_size 0x3(0x265b bytes) 2024-09-17T07:29:54.082 INFO:teuthology.orchestra.run.smithi045.stdout:4652 read 0x9cc86e5 thru 0x9cd2cc4 (0xa5e0 bytes) 2024-09-17T07:29:54.082 INFO:teuthology.orchestra.run.smithi045.stdout:4655 read 0x3a44361 thru 0x3a44df0 (0xa90 bytes) 2024-09-17T07:29:54.082 INFO:teuthology.orchestra.run.smithi045.stdout:4656 read 0x605def9 thru 0x6064966 (0x6a6e bytes) 2024-09-17T07:29:54.082 INFO:teuthology.orchestra.run.smithi045.stdout:4659 write 0x28cad6d thru 0x28d5a59 (0xaced bytes) 2024-09-17T07:29:54.085 INFO:teuthology.orchestra.run.smithi045.stdout:4662 writesame 0x514a958 thru 0x515a80c data_size 0x51(0xfeb5 bytes) 2024-09-17T07:29:54.093 INFO:teuthology.orchestra.run.smithi045.stdout:4663 trunc from 0xc36171c to 0x9e973e4 2024-09-17T07:29:54.099 INFO:teuthology.orchestra.run.smithi045.stdout:4664 read 0x2e9351d thru 0x2e9813c (0x4c20 bytes) 2024-09-17T07:29:54.100 INFO:teuthology.orchestra.run.smithi045.stdout:4665 punch from 0x61d39b0 to 0x61db271, (0x78c1 bytes) 2024-09-17T07:29:54.100 INFO:teuthology.orchestra.run.smithi045.stdout:4667 read 0x8a24431 thru 0x8a28238 (0x3e08 bytes) 2024-09-17T07:29:54.100 INFO:teuthology.orchestra.run.smithi045.stdout:4668 read 0x47c01b4 thru 0x47d010d (0xff5a bytes) 2024-09-17T07:29:54.100 INFO:teuthology.orchestra.run.smithi045.stdout:4669 read 0x8a96b8b thru 0x8a9fdbb (0x9231 bytes) 2024-09-17T07:29:54.100 INFO:teuthology.orchestra.run.smithi045.stdout:4671 writesame 0x2f1e371 thru 0x2f2e364 data_size 0x2(0xfff4 bytes) 2024-09-17T07:29:54.114 INFO:teuthology.orchestra.run.smithi045.stdout:4672 write 0x4ad5b33 thru 0x4adddd6 (0x82a4 bytes) 2024-09-17T07:29:54.120 INFO:teuthology.orchestra.run.smithi045.stdout:4674 read 0x74528f7 thru 0x745b14e (0x8858 bytes) 2024-09-17T07:29:54.120 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.120 INFO:teuthology.orchestra.run.smithi045.stdout:4676 write 0x98bfa99 thru 0x98c8663 (0x8bcb bytes) 2024-09-17T07:29:54.122 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T07:29:54.130 INFO:teuthology.orchestra.run.smithi045.stdout:4678 trunc from 0x9e973e4 to 0x5a5c7e6 2024-09-17T07:29:54.148 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:54.147+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:54.148 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.149 INFO:teuthology.orchestra.run.smithi045.stdout:4680 read 0x2ca74 thru 0x3008e (0x361b bytes) 2024-09-17T07:29:54.152 INFO:teuthology.orchestra.run.smithi045.stdout:4681 trunc from 0x5a5c7e6 to 0x8a13ede 2024-09-17T07:29:54.160 INFO:teuthology.orchestra.run.smithi045.stdout:4682 trunc from 0x8a13ede to 0x7566bae 2024-09-17T07:29:54.165 INFO:teuthology.orchestra.run.smithi045.stdout:4683 write 0x4936a17 thru 0x4938974 (0x1f5e bytes) 2024-09-17T07:29:54.169 INFO:teuthology.orchestra.run.smithi045.stdout:4684 trunc from 0x7566bae to 0x94f4435 2024-09-17T07:29:54.175 INFO:teuthology.orchestra.run.smithi045.stdout:4685 write 0x3462928 thru 0x346f094 (0xc76d bytes) 2024-09-17T07:29:54.180 INFO:teuthology.orchestra.run.smithi045.stdout:4687 read 0x7b89183 thru 0x7b898fc (0x77a bytes) 2024-09-17T07:29:54.180 INFO:teuthology.orchestra.run.smithi045.stdout:4690 write 0x40e747e thru 0x40ee1a4 (0x6d27 bytes) 2024-09-17T07:29:54.187 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:54.188 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:54.188 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:54.188 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:54.190 INFO:teuthology.orchestra.run.smithi045.stdout:4691 writesame 0xa3d3744 thru 0xa3d6e62 data_size 0x1(0x371f bytes) 2024-09-17T07:29:54.190 INFO:teuthology.orchestra.run.smithi045.stdout:4692 punch from 0x6f65d23 to 0x6f68389, (0x2666 bytes) 2024-09-17T07:29:54.191 INFO:teuthology.orchestra.run.smithi045.stdout:4693 writesame 0x667d323 thru 0x667dc85 data_size 0x9(0x963 bytes) 2024-09-17T07:29:54.194 INFO:teuthology.orchestra.run.smithi045.stdout:4695 write 0x5e9fc26 thru 0x5ea09a5 (0xd80 bytes) 2024-09-17T07:29:54.198 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:54.199 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.199 INFO:teuthology.orchestra.run.smithi045.stdout:4698 write 0x82cf294 thru 0x82d523c (0x5fa9 bytes) 2024-09-17T07:29:54.202 INFO:teuthology.orchestra.run.smithi045.stdout:4699 writesame 0x5bad5e1 thru 0x5bb5d02 data_size 0xe(0x8722 bytes) 2024-09-17T07:29:54.206 INFO:teuthology.orchestra.run.smithi045.stdout:4701 read 0x5249e7 thru 0x530814 (0xbe2e bytes) 2024-09-17T07:29:54.209 INFO:teuthology.orchestra.run.smithi045.stdout:4702 write 0x710b66b thru 0x711661e (0xafb4 bytes) 2024-09-17T07:29:54.213 INFO:teuthology.orchestra.run.smithi045.stdout:4704 write 0x996d89d thru 0x997365c (0x5dc0 bytes) 2024-09-17T07:29:54.218 INFO:teuthology.orchestra.run.smithi045.stdout:4705 write 0x9fc194e thru 0x9fc490b (0x2fbe bytes) 2024-09-17T07:29:54.221 INFO:teuthology.orchestra.run.smithi045.stdout:4706 read 0x13344cb thru 0x133b821 (0x7357 bytes) 2024-09-17T07:29:54.227 INFO:teuthology.orchestra.run.smithi045.stdout:4707 write 0xa839097 thru 0xa846961 (0xd8cb bytes) 2024-09-17T07:29:54.235 INFO:teuthology.orchestra.run.smithi045.stdout:4709 trunc from 0xa846962 to 0x8288547 2024-09-17T07:29:54.247 INFO:teuthology.orchestra.run.smithi045.stdout:4710 read 0x3bcf325 thru 0x3bdbf88 (0xcc64 bytes) 2024-09-17T07:29:54.248 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:54.248+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:54.253 INFO:teuthology.orchestra.run.smithi045.stdout:4712 trunc from 0x8288547 to 0xcb66ac2 2024-09-17T07:29:54.259 INFO:teuthology.orchestra.run.smithi045.stdout:4714 punch from 0x541895c to 0x542020f, (0x78b3 bytes) 2024-09-17T07:29:54.259 INFO:teuthology.orchestra.run.smithi045.stdout:4716 read 0x2828b77 thru 0x2830cac (0x8136 bytes) 2024-09-17T07:29:54.259 INFO:teuthology.orchestra.run.smithi045.stdout:4717 punch from 0x383afb7 to 0x38413dc, (0x6425 bytes) 2024-09-17T07:29:54.266 INFO:teuthology.orchestra.run.smithi045.stdout:4718 write 0xd827ef3 thru 0xd82f9c6 (0x7ad4 bytes) 2024-09-17T07:29:54.272 INFO:teuthology.orchestra.run.smithi045.stdout:4719 writesame 0x5a2745d thru 0x5a2b3ad data_size 0x3(0x3f51 bytes) 2024-09-17T07:29:54.277 INFO:teuthology.orchestra.run.smithi045.stdout:4721 writesame 0x5412640 thru 0x5418ea7 data_size 0x34(0x6868 bytes) 2024-09-17T07:29:54.282 INFO:teuthology.orchestra.run.smithi045.stdout:4722 read 0x97818c2 thru 0x978dbb9 (0xc2f8 bytes) 2024-09-17T07:29:54.283 INFO:teuthology.orchestra.run.smithi045.stdout:4723 punch from 0xcdc179 to 0xcdc4d9, (0x360 bytes) 2024-09-17T07:29:54.283 INFO:teuthology.orchestra.run.smithi045.stdout:4725 write 0x5557338 thru 0x5558e7a (0x1b43 bytes) 2024-09-17T07:29:54.289 INFO:teuthology.orchestra.run.smithi045.stdout:4727 punch from 0x363ab66 to 0x364239d, (0x7837 bytes) 2024-09-17T07:29:54.289 INFO:teuthology.orchestra.run.smithi045.stdout:4728 write 0x954121d thru 0x954db68 (0xc94c bytes) 2024-09-17T07:29:54.294 INFO:teuthology.orchestra.run.smithi045.stdout:4729 trunc from 0xd82f9c7 to 0xce5d47a 2024-09-17T07:29:54.304 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.305 INFO:teuthology.orchestra.run.smithi045.stdout:4731 write 0x9e62896 thru 0x9e65450 (0x2bbb bytes) 2024-09-17T07:29:54.307 INFO:teuthology.orchestra.run.smithi045.stdout:4732 write 0xca349ab thru 0xca38413 (0x3a69 bytes) 2024-09-17T07:29:54.322 INFO:teuthology.orchestra.run.smithi045.stdout:4733 write 0xe1d398c thru 0xe1d7fd0 (0x4645 bytes) 2024-09-17T07:29:54.326 INFO:teuthology.orchestra.run.smithi045.stdout:4734 write 0x21b869d thru 0x21c2ea5 (0xa809 bytes) 2024-09-17T07:29:54.328 INFO:teuthology.orchestra.run.smithi045.stdout:4735 read 0x1ac860d thru 0x1ad7d6c (0xf760 bytes) 2024-09-17T07:29:54.328 INFO:teuthology.orchestra.run.smithi045.stdout:4736 read 0x788d92 thru 0x78fe90 (0x70ff bytes) 2024-09-17T07:29:54.329 INFO:teuthology.orchestra.run.smithi045.stdout:4737 punch from 0x77af806 to 0x77b6620, (0x6e1a bytes) 2024-09-17T07:29:54.329 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.329 INFO:teuthology.orchestra.run.smithi045.stdout:4740 read 0xe1dd3b thru 0xe24d67 (0x702d bytes) 2024-09-17T07:29:54.329 INFO:teuthology.orchestra.run.smithi045.stdout:4741 writesame 0x52954ca thru 0x529cc84 data_size 0x11(0x77bb bytes) 2024-09-17T07:29:54.335 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:54.335 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:54.335 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:29:54.335 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:29:54.335 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:54.337 INFO:teuthology.orchestra.run.smithi045.stdout:4743 writesame 0xcc30346 thru 0xcc3af7b data_size 0x5e(0xac36 bytes) 2024-09-17T07:29:54.341 INFO:teuthology.orchestra.run.smithi045.stdout:4744 writesame 0xcdd13b2 thru 0xcdd8494 data_size 0x75(0x70e3 bytes) 2024-09-17T07:29:54.346 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:54.349 INFO:teuthology.orchestra.run.smithi045.stdout:4745 trunc from 0xe1d7fd1 to 0xc313235 2024-09-17T07:29:54.349 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:54.349+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:54.361 INFO:teuthology.orchestra.run.smithi045.stdout:4750 trunc from 0xc313235 to 0xb4b1578 2024-09-17T07:29:54.368 INFO:teuthology.orchestra.run.smithi045.stdout:4752 punch from 0x2a5ef04 to 0x2a5faf6, (0xbf2 bytes) 2024-09-17T07:29:54.368 INFO:teuthology.orchestra.run.smithi045.stdout:4754 writesame 0x69a9b0a thru 0x69b3c8f data_size 0x19(0xa186 bytes) 2024-09-17T07:29:54.375 INFO:teuthology.orchestra.run.smithi045.stdout:4755 read 0x244818e thru 0x244c15b (0x3fce bytes) 2024-09-17T07:29:54.375 INFO:teuthology.orchestra.run.smithi045.stdout:4756 read 0x58e6d6a thru 0x58ecaf7 (0x5d8e bytes) 2024-09-17T07:29:54.383 INFO:teuthology.orchestra.run.smithi045.stdout:4757 writesame 0xc72e799 thru 0xc734088 data_size 0x8(0x58f0 bytes) 2024-09-17T07:29:54.383 INFO:teuthology.orchestra.run.smithi045.stdout:4758 read 0x2627d13 thru 0x2636b21 (0xee0f bytes) 2024-09-17T07:29:54.383 INFO:teuthology.orchestra.run.smithi045.stdout:4759 write 0x35848dc thru 0x358e730 (0x9e55 bytes) 2024-09-17T07:29:54.387 INFO:teuthology.orchestra.run.smithi045.stdout:4760 punch from 0x6273b9a to 0x627c6c9, (0x8b2f bytes) 2024-09-17T07:29:54.387 INFO:teuthology.orchestra.run.smithi045.stdout:4761 punch from 0x5d2f7d5 to 0x5d33647, (0x3e72 bytes) 2024-09-17T07:29:54.388 INFO:teuthology.orchestra.run.smithi045.stdout:4762 writesame 0xf37023 thru 0xf440cd data_size 0x1(0xd0ab bytes) 2024-09-17T07:29:54.414 INFO:teuthology.orchestra.run.smithi045.stdout:4763 read 0xc234cc7 thru 0xc24479e (0xfad8 bytes) 2024-09-17T07:29:54.420 INFO:teuthology.orchestra.run.smithi045.stdout:4764 trunc from 0xc734089 to 0x7227274 2024-09-17T07:29:54.432 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.432 INFO:teuthology.orchestra.run.smithi045.stdout:4767 read 0x6c30bfe thru 0x6c36cfc (0x60ff bytes) 2024-09-17T07:29:54.432 INFO:teuthology.orchestra.run.smithi045.stdout:4768 punch from 0x2ee911d to 0x2eedc38, (0x4b1b bytes) 2024-09-17T07:29:54.432 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.432 INFO:teuthology.orchestra.run.smithi045.stdout:4770 read 0x12cdf98 thru 0x12d6927 (0x8990 bytes) 2024-09-17T07:29:54.432 INFO:teuthology.orchestra.run.smithi045.stdout:4771 writesame 0x10030d1 thru 0x100c109 data_size 0x1f(0x9039 bytes) 2024-09-17T07:29:54.433 INFO:teuthology.orchestra.run.smithi045.stdout:4772 punch from 0x44f6de to 0x4509bf, (0x12e1 bytes) 2024-09-17T07:29:54.433 INFO:teuthology.orchestra.run.smithi045.stdout:4776 punch from 0xec2b40 to 0xec401a, (0x14da bytes) 2024-09-17T07:29:54.439 INFO:teuthology.orchestra.run.smithi045.stdout:4777 trunc from 0x7227274 to 0x251fea2 2024-09-17T07:29:54.450 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:54.449+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:54.466 INFO:teuthology.orchestra.run.smithi045.stdout:4778 read 0x1905f57 thru 0x190a149 (0x41f3 bytes) 2024-09-17T07:29:54.475 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:29:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:29:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:29:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.475 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17069 3.0 e41/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 41'110, mlcod=40'108)", 2024-09-17T07:29:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.733848109, 2024-09-17T07:29:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011408908000000001, 2024-09-17T07:29:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:29:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739753+0000", 2024-09-17T07:29:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:29:54.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743532+0000", 2024-09-17T07:29:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037794510000000001 2024-09-17T07:29:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743557+0000", 2024-09-17T07:29:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5253e-05 2024-09-17T07:29:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743569+0000", 2024-09-17T07:29:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1161999999999999e-05 2024-09-17T07:29:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743650+0000", 2024-09-17T07:29:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1087999999999993e-05 2024-09-17T07:29:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743674+0000", 2024-09-17T07:29:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4562e-05 2024-09-17T07:29:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751095+0000", 2024-09-17T07:29:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074211140000000004 2024-09-17T07:29:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751161+0000", 2024-09-17T07:29:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5707e-05 2024-09-17T07:29:54.480 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17240 3.0 e42/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 42'113, mlcod=41'110)", 2024-09-17T07:29:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.248189625000002, 2024-09-17T07:29:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064392440000000002, 2024-09-17T07:29:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:29:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:29:54.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225418+0000", 2024-09-17T07:29:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8360000000000003e-06 2024-09-17T07:29:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225426+0000", 2024-09-17T07:29:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2670000000000006e-06 2024-09-17T07:29:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225434+0000", 2024-09-17T07:29:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0560000000000005e-06 2024-09-17T07:29:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225514+0000", 2024-09-17T07:29:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9979000000000001e-05 2024-09-17T07:29:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225535+0000", 2024-09-17T07:29:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0965e-05 2024-09-17T07:29:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231797+0000", 2024-09-17T07:29:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062621129999999997 2024-09-17T07:29:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231850+0000", 2024-09-17T07:29:54.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2611999999999997e-05 2024-09-17T07:29:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20054 3.9 e44/23 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:head v 44'131, mlcod=44'129)", 2024-09-17T07:29:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 49.206038509000003, 2024-09-17T07:29:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028510570000000002, 2024-09-17T07:29:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.486 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:29:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267563+0000", 2024-09-17T07:29:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.584e-06 2024-09-17T07:29:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268269+0000", 2024-09-17T07:29:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00070567399999999999 2024-09-17T07:29:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268287+0000", 2024-09-17T07:29:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.84e-05 2024-09-17T07:29:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268297+0000", 2024-09-17T07:29:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9059999999999997e-06 2024-09-17T07:29:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268368+0000", 2024-09-17T07:29:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0562999999999995e-05 2024-09-17T07:29:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268393+0000", 2024-09-17T07:29:54.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4958999999999999e-05 2024-09-17T07:29:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270330+0000", 2024-09-17T07:29:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001936644 2024-09-17T07:29:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270413+0000", 2024-09-17T07:29:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3326999999999997e-05 2024-09-17T07:29:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:32543 3.0 e55/23 3:03bfd97d:::rbd_data.11839a97b753.0000000000000007:head v 55'150, mlcod=54'148)", 2024-09-17T07:29:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 37.018963372000002, 2024-09-17T07:29:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031813480000000001, 2024-09-17T07:29:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.491 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:29:54.492 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5199999999999998e-07 2024-09-17T07:29:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455083+0000", 2024-09-17T07:29:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044629199999999999 2024-09-17T07:29:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455098+0000", 2024-09-17T07:29:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4183e-05 2024-09-17T07:29:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455106+0000", 2024-09-17T07:29:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:29:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455173+0000", 2024-09-17T07:29:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6346000000000005e-05 2024-09-17T07:29:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455200+0000", 2024-09-17T07:29:54.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7319000000000001e-05 2024-09-17T07:29:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457774+0000", 2024-09-17T07:29:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025742650000000001 2024-09-17T07:29:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457818+0000", 2024-09-17T07:29:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3891999999999999e-05 2024-09-17T07:29:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.495 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33735 3.1f e55/52 3:fea54097:::rbd_data.11839a97b753.0000000000000000:head v 55'260, mlcod=55'258)", 2024-09-17T07:29:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:29:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 36.649030744000001, 2024-09-17T07:29:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035215200000000002, 2024-09-17T07:29:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.496 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:29:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:29:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824571+0000", 2024-09-17T07:29:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.164e-06 2024-09-17T07:29:54.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825118+0000", 2024-09-17T07:29:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054696400000000004 2024-09-17T07:29:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825134+0000", 2024-09-17T07:29:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6342999999999999e-05 2024-09-17T07:29:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825142+0000", 2024-09-17T07:29:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5270000000000001e-06 2024-09-17T07:29:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825200+0000", 2024-09-17T07:29:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.893e-05 2024-09-17T07:29:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825218+0000", 2024-09-17T07:29:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7935999999999999e-05 2024-09-17T07:29:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828037+0000", 2024-09-17T07:29:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002818961 2024-09-17T07:29:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828091+0000", 2024-09-17T07:29:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3695e-05 2024-09-17T07:29:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.501 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33869 3.9 e55/23 3:907b81b5:::rbd_data.11839a97b753.0000000000000001:head v 55'171, mlcod=55'169)", 2024-09-17T07:29:54.501 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:29:54.501 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 36.631974921000001, 2024-09-17T07:29:54.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033990859999999999, 2024-09-17T07:29:54.501 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.501 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.501 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.501 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:29:54.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.502 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.502 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:29:54.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.502 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.502 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841626+0000", 2024-09-17T07:29:54.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.023e-06 2024-09-17T07:29:54.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842067+0000", 2024-09-17T07:29:54.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000440612 2024-09-17T07:29:54.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842080+0000", 2024-09-17T07:29:54.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.29e-05 2024-09-17T07:29:54.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842086+0000", 2024-09-17T07:29:54.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2090000000000002e-06 2024-09-17T07:29:54.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.504 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842144+0000", 2024-09-17T07:29:54.504 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7627999999999998e-05 2024-09-17T07:29:54.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.504 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842171+0000", 2024-09-17T07:29:54.504 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7214000000000001e-05 2024-09-17T07:29:54.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.505 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.844973+0000", 2024-09-17T07:29:54.505 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002802232 2024-09-17T07:29:54.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.505 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.505 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.845024+0000", 2024-09-17T07:29:54.505 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1267999999999998e-05 2024-09-17T07:29:54.505 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.505 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.505 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.505 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51432 3.0 e59/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 59'177, mlcod=58'174)", 2024-09-17T07:29:54.506 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:29:54.506 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.652377904000002, 2024-09-17T07:29:54.506 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010790086000000001, 2024-09-17T07:29:54.506 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.506 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.506 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.506 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.506 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.506 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.506 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:29:54.506 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.507 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.507 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:29:54.507 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.507 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.507 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821224+0000", 2024-09-17T07:29:54.507 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.511e-06 2024-09-17T07:29:54.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.507 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.507 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825174+0000", 2024-09-17T07:29:54.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0039497949999999999 2024-09-17T07:29:54.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.508 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.508 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825196+0000", 2024-09-17T07:29:54.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2643999999999999e-05 2024-09-17T07:29:54.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.508 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.508 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825206+0000", 2024-09-17T07:29:54.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4320000000000005e-06 2024-09-17T07:29:54.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.509 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.509 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825272+0000", 2024-09-17T07:29:54.509 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5829999999999998e-05 2024-09-17T07:29:54.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.509 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.509 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825299+0000", 2024-09-17T07:29:54.509 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7107e-05 2024-09-17T07:29:54.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.509 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.509 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.831939+0000", 2024-09-17T07:29:54.510 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0066398510000000004 2024-09-17T07:29:54.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.510 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.510 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.832012+0000", 2024-09-17T07:29:54.510 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3916000000000002e-05 2024-09-17T07:29:54.510 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.510 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.510 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.511 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51969 3.0 e59/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 59'179, mlcod=59'177)", 2024-09-17T07:29:54.511 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:29:54.511 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.543509526000001, 2024-09-17T07:29:54.511 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060297249999999997, 2024-09-17T07:29:54.511 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.511 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.511 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.511 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.511 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.511 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:29:54.511 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.512 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.512 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:29:54.512 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.512 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.512 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930093+0000", 2024-09-17T07:29:54.512 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.764e-06 2024-09-17T07:29:54.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.513 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.513 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931779+0000", 2024-09-17T07:29:54.513 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001686379 2024-09-17T07:29:54.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.513 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.513 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931802+0000", 2024-09-17T07:29:54.513 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2682000000000001e-05 2024-09-17T07:29:54.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.513 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.513 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931811+0000", 2024-09-17T07:29:54.513 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5640000000000002e-06 2024-09-17T07:29:54.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.514 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.514 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931877+0000", 2024-09-17T07:29:54.514 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6155999999999995e-05 2024-09-17T07:29:54.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.514 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.514 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931896+0000", 2024-09-17T07:29:54.514 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9111e-05 2024-09-17T07:29:54.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.515 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.515 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936063+0000", 2024-09-17T07:29:54.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004166711 2024-09-17T07:29:54.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.515 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.515 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936121+0000", 2024-09-17T07:29:54.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7358000000000002e-05 2024-09-17T07:29:54.515 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.515 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.515 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.516 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54417 3.0 e60/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 60'182, mlcod=59'179)", 2024-09-17T07:29:54.516 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:29:54.516 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.035992534999998, 2024-09-17T07:29:54.516 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0059885219999999996, 2024-09-17T07:29:54.516 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.516 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.516 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.516 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.516 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.517 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:29:54.517 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.517 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.517 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:29:54.517 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.517 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.517 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437609+0000", 2024-09-17T07:29:54.517 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5829999999999999e-06 2024-09-17T07:29:54.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.518 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.518 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.518 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437623+0000", 2024-09-17T07:29:54.518 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3348e-05 2024-09-17T07:29:54.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.518 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.518 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.518 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437643+0000", 2024-09-17T07:29:54.519 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0463999999999999e-05 2024-09-17T07:29:54.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.519 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.519 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437653+0000", 2024-09-17T07:29:54.519 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6290000000000005e-06 2024-09-17T07:29:54.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.519 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.520 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437719+0000", 2024-09-17T07:29:54.520 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6428999999999997e-05 2024-09-17T07:29:54.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.520 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.520 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.520 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437745+0000", 2024-09-17T07:29:54.520 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5690999999999998e-05 2024-09-17T07:29:54.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.520 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.521 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.521 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443549+0000", 2024-09-17T07:29:54.521 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0058038259999999998 2024-09-17T07:29:54.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.521 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.521 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443596+0000", 2024-09-17T07:29:54.521 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7552000000000001e-05 2024-09-17T07:29:54.521 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.521 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.521 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.522 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54419 3.0 e60/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 60'184, mlcod=59'179)", 2024-09-17T07:29:54.522 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:29:54.522 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 31.029640178000001, 2024-09-17T07:29:54.522 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004706844, 2024-09-17T07:29:54.522 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.522 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.522 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.522 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.522 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.522 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:29:54.523 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.523 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.523 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:29:54.523 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.523 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.523 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443961+0000", 2024-09-17T07:29:54.523 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:29:54.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.524 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.524 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443967+0000", 2024-09-17T07:29:54.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.268e-06 2024-09-17T07:29:54.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.524 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.524 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443983+0000", 2024-09-17T07:29:54.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5248999999999999e-05 2024-09-17T07:29:54.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.524 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.525 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443990+0000", 2024-09-17T07:29:54.525 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5120000000000002e-06 2024-09-17T07:29:54.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.525 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.525 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444056+0000", 2024-09-17T07:29:54.525 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6197000000000003e-05 2024-09-17T07:29:54.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.525 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.525 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444081+0000", 2024-09-17T07:29:54.525 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4791e-05 2024-09-17T07:29:54.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.526 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.526 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448616+0000", 2024-09-17T07:29:54.526 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045346129999999998 2024-09-17T07:29:54.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.526 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.526 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448667+0000", 2024-09-17T07:29:54.526 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-17T07:29:54.526 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.526 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.526 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.527 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54895 3.1f e60/52 3:fa1c1177:::rbd_data.1183ad87379e.0000000000000001:head v 60'283, mlcod=60'281)", 2024-09-17T07:29:54.527 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:29:54.527 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 30.682159181999999, 2024-09-17T07:29:54.527 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0029495210000000001, 2024-09-17T07:29:54.527 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.527 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.527 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.527 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.527 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.527 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:29:54.528 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.528 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.528 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:29:54.528 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.528 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.528 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791443+0000", 2024-09-17T07:29:54.528 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.153e-06 2024-09-17T07:29:54.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.529 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792619+0000", 2024-09-17T07:29:54.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001176011 2024-09-17T07:29:54.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.529 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792637+0000", 2024-09-17T07:29:54.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-05 2024-09-17T07:29:54.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.529 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792649+0000", 2024-09-17T07:29:54.530 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1712000000000001e-05 2024-09-17T07:29:54.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.530 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.530 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792722+0000", 2024-09-17T07:29:54.530 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3545000000000001e-05 2024-09-17T07:29:54.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.530 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.530 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792738+0000", 2024-09-17T07:29:54.530 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5223e-05 2024-09-17T07:29:54.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.530 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.531 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.794330+0000", 2024-09-17T07:29:54.531 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015922740000000001 2024-09-17T07:29:54.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.531 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.531 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.794391+0000", 2024-09-17T07:29:54.531 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0803e-05 2024-09-17T07:29:54.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.531 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.531 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:80794 3.9 e72/23 3:90311a61:::rbd_data.1183cb746bb8.0000000000000007:head v 72'222, mlcod=71'220)", 2024-09-17T07:29:54.532 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:29:54.532 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.041651294000001, 2024-09-17T07:29:54.532 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004371733, 2024-09-17T07:29:54.532 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.532 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.532 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.532 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.532 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.532 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:29:54.532 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:29:54.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:29:54.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0800000000000001e-07 2024-09-17T07:29:54.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434108+0000", 2024-09-17T07:29:54.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021587569999999999 2024-09-17T07:29:54.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434120+0000", 2024-09-17T07:29:54.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1821000000000001e-05 2024-09-17T07:29:54.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434128+0000", 2024-09-17T07:29:54.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7579999999999992e-06 2024-09-17T07:29:54.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.535 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434191+0000", 2024-09-17T07:29:54.535 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3202999999999998e-05 2024-09-17T07:29:54.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.535 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.535 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434218+0000", 2024-09-17T07:29:54.535 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6771000000000002e-05 2024-09-17T07:29:54.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.535 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.535 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436289+0000", 2024-09-17T07:29:54.535 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020717700000000001 2024-09-17T07:29:54.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.536 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.536 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436321+0000", 2024-09-17T07:29:54.536 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1445000000000001e-05 2024-09-17T07:29:54.536 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.536 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.536 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.536 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 75/74 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9, version: 36'125, data_included: [5478708~26316,5767168~524288,6553600~497972], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9@36'125, size: 10246384, copy_subset: [(0, 10246384)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7051572, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5478708, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:29:54.536 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:41.719003+0000", 2024-09-17T07:29:54.536 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 12.754597131000001, 2024-09-17T07:29:54.536 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025840260000000001, 2024-09-17T07:29:54.537 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.537 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.537 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:54.537 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.537 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.537 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.719003+0000", 2024-09-17T07:29:54.537 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.537 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.537 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.719003+0000", 2024-09-17T07:29:54.537 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.538 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.538 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.538 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.719004+0000", 2024-09-17T07:29:54.538 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0699999999999996e-07 2024-09-17T07:29:54.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.538 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.538 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.538 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.720057+0000", 2024-09-17T07:29:54.538 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010526800000000001 2024-09-17T07:29:54.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.538 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.538 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.539 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.720083+0000", 2024-09-17T07:29:54.539 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6237e-05 2024-09-17T07:29:54.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.539 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.539 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.720102+0000", 2024-09-17T07:29:54.539 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8918000000000001e-05 2024-09-17T07:29:54.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.539 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.539 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.720207+0000", 2024-09-17T07:29:54.539 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010558800000000001 2024-09-17T07:29:54.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.540 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.540 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.720224+0000", 2024-09-17T07:29:54.540 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6426000000000002e-05 2024-09-17T07:29:54.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.540 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.540 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.721587+0000", 2024-09-17T07:29:54.540 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00136337 2024-09-17T07:29:54.540 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.540 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.540 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.541 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 75/74 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head, version: 36'126, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head@36'126, size: 9431460, copy_subset: [], clone_subset: {3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9: [(0, 9431460)]}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:29:54.541 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:29:54.541 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 12.732743701, 2024-09-17T07:29:54.541 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041288289999999997, 2024-09-17T07:29:54.541 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.541 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.541 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:29:54.541 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.541 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.541 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:29:54.542 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.542 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.542 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:29:54.542 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.542 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.542 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740858+0000", 2024-09-17T07:29:54.542 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1599999999999999e-06 2024-09-17T07:29:54.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.543 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.543 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740885+0000", 2024-09-17T07:29:54.543 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.707e-05 2024-09-17T07:29:54.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.543 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.543 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740925+0000", 2024-09-17T07:29:54.543 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0649000000000003e-05 2024-09-17T07:29:54.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.543 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.543 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740957+0000", 2024-09-17T07:29:54.544 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1585999999999999e-05 2024-09-17T07:29:54.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.544 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.544 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741185+0000", 2024-09-17T07:29:54.544 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022759700000000001 2024-09-17T07:29:54.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.544 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.544 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741228+0000", 2024-09-17T07:29:54.544 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3705999999999998e-05 2024-09-17T07:29:54.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.545 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.744985+0000", 2024-09-17T07:29:54.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037570609999999999 2024-09-17T07:29:54.545 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.545 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.545 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.545 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85061 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'257, mlcod=79'255)", 2024-09-17T07:29:54.545 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:29:54.546 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.3249127450000007, 2024-09-17T07:29:54.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033850550000000001, 2024-09-17T07:29:54.546 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.546 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.546 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.546 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.546 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.546 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:29:54.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:29:54.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:29:54.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9699999999999995e-07 2024-09-17T07:29:54.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148835+0000", 2024-09-17T07:29:54.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000146852 2024-09-17T07:29:54.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.548 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.548 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148845+0000", 2024-09-17T07:29:54.548 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0232e-05 2024-09-17T07:29:54.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.548 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.548 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148852+0000", 2024-09-17T07:29:54.548 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6490000000000002e-06 2024-09-17T07:29:54.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.548 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.549 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148913+0000", 2024-09-17T07:29:54.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1099999999999994e-05 2024-09-17T07:29:54.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.549 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.549 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148931+0000", 2024-09-17T07:29:54.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7938e-05 2024-09-17T07:29:54.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.549 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.549 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152025+0000", 2024-09-17T07:29:54.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0030942729999999998 2024-09-17T07:29:54.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.550 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.550 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152073+0000", 2024-09-17T07:29:54.550 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7213999999999999e-05 2024-09-17T07:29:54.550 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.550 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.550 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.550 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85063 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'259, mlcod=79'255)", 2024-09-17T07:29:54.550 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:29:54.550 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.3237657909999996, 2024-09-17T07:29:54.550 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037722659999999998, 2024-09-17T07:29:54.550 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.551 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.551 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.551 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.551 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.551 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:29:54.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.551 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.551 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:29:54.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:29:54.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5799999999999999e-07 2024-09-17T07:29:54.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149841+0000", 2024-09-17T07:29:54.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4840000000000003e-06 2024-09-17T07:29:54.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149849+0000", 2024-09-17T07:29:54.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4670000000000004e-06 2024-09-17T07:29:54.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149854+0000", 2024-09-17T07:29:54.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4779999999999999e-06 2024-09-17T07:29:54.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149920+0000", 2024-09-17T07:29:54.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5760999999999996e-05 2024-09-17T07:29:54.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149949+0000", 2024-09-17T07:29:54.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8838000000000001e-05 2024-09-17T07:29:54.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153558+0000", 2024-09-17T07:29:54.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036092939999999999 2024-09-17T07:29:54.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153607+0000", 2024-09-17T07:29:54.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8386000000000001e-05 2024-09-17T07:29:54.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.555 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.555 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85085 3.5 e79/76 3:a50fe76b:::rbd_data.1183c869c667.0000000000000035:head v 79'401, mlcod=78'399)", 2024-09-17T07:29:54.555 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.151174+0000", 2024-09-17T07:29:54.555 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.3224259939999996, 2024-09-17T07:29:54.555 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025105449999999999, 2024-09-17T07:29:54.555 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.555 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.555 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.555 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.556 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.556 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151174+0000", 2024-09-17T07:29:54.556 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.556 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.556 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151174+0000", 2024-09-17T07:29:54.556 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.556 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.556 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151175+0000", 2024-09-17T07:29:54.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9999999999999997e-07 2024-09-17T07:29:54.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151180+0000", 2024-09-17T07:29:54.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2650000000000004e-06 2024-09-17T07:29:54.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151189+0000", 2024-09-17T07:29:54.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0780000000000002e-06 2024-09-17T07:29:54.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.558 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151196+0000", 2024-09-17T07:29:54.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1600000000000003e-06 2024-09-17T07:29:54.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.558 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151267+0000", 2024-09-17T07:29:54.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1916999999999996e-05 2024-09-17T07:29:54.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151300+0000", 2024-09-17T07:29:54.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2975000000000002e-05 2024-09-17T07:29:54.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153644+0000", 2024-09-17T07:29:54.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00234329 2024-09-17T07:29:54.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153685+0000", 2024-09-17T07:29:54.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1159999999999999e-05 2024-09-17T07:29:54.559 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.560 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.560 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.560 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85234 3.9 e79/23 3:967c906c:::rbd_data.1183c869c667.0000000000000031:head v 79'260, mlcod=79'257)", 2024-09-17T07:29:54.560 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.165738+0000", 2024-09-17T07:29:54.560 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.3078624340000005, 2024-09-17T07:29:54.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002411519, 2024-09-17T07:29:54.560 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.560 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.560 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.560 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.561 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.165738+0000", 2024-09-17T07:29:54.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.561 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.165738+0000", 2024-09-17T07:29:54.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.561 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.165739+0000", 2024-09-17T07:29:54.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0380000000000001e-06 2024-09-17T07:29:54.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.562 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.165852+0000", 2024-09-17T07:29:54.562 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112841 2024-09-17T07:29:54.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.562 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.165868+0000", 2024-09-17T07:29:54.562 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6093e-05 2024-09-17T07:29:54.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.165876+0000", 2024-09-17T07:29:54.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6440000000000006e-06 2024-09-17T07:29:54.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.165932+0000", 2024-09-17T07:29:54.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5858e-05 2024-09-17T07:29:54.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.165959+0000", 2024-09-17T07:29:54.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6900999999999999e-05 2024-09-17T07:29:54.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.168109+0000", 2024-09-17T07:29:54.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021500109999999998 2024-09-17T07:29:54.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.168149+0000", 2024-09-17T07:29:54.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0132999999999997e-05 2024-09-17T07:29:54.564 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.564 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.564 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.564 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:94996 3.0 e82/23 3:074b4f56:::rbd_data.118311e0d6f8.0000000000000001:head v 82'715, mlcod=80'713)", 2024-09-17T07:29:54.564 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:29:54.565 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.857860198, 2024-09-17T07:29:54.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031499179999999998, 2024-09-17T07:29:54.565 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.565 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.565 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.565 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:29:54.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:29:54.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615741+0000", 2024-09-17T07:29:54.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999996e-07 2024-09-17T07:29:54.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616917+0000", 2024-09-17T07:29:54.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011763310000000001 2024-09-17T07:29:54.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.567 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.567 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616936+0000", 2024-09-17T07:29:54.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8624e-05 2024-09-17T07:29:54.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.567 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.567 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616948+0000", 2024-09-17T07:29:54.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.238e-05 2024-09-17T07:29:54.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.567 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.567 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617054+0000", 2024-09-17T07:29:54.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000106556 2024-09-17T07:29:54.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617081+0000", 2024-09-17T07:29:54.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.622e-05 2024-09-17T07:29:54.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618836+0000", 2024-09-17T07:29:54.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001755412 2024-09-17T07:29:54.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618890+0000", 2024-09-17T07:29:54.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3915000000000001e-05 2024-09-17T07:29:54.569 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.569 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.569 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.569 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:95007 3.5 e82/76 3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head v 82'422, mlcod=82'420)", 2024-09-17T07:29:54.569 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:29:54.569 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.849937497, 2024-09-17T07:29:54.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003939901, 2024-09-17T07:29:54.569 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:29:54.569 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:29:54.570 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.570 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:29:54.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:29:54.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:29:54.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:29:54.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:29:54.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:29:54.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:29:54.571 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623664+0000", 2024-09-17T07:29:54.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3799999999999999e-06 2024-09-17T07:29:54.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:29:54.571 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626104+0000", 2024-09-17T07:29:54.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002440149 2024-09-17T07:29:54.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:29:54.571 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626120+0000", 2024-09-17T07:29:54.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5962000000000001e-05 2024-09-17T07:29:54.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626127+0000", 2024-09-17T07:29:54.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8340000000000001e-06 2024-09-17T07:29:54.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:29:54.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626186+0000", 2024-09-17T07:29:54.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9234999999999998e-05 2024-09-17T07:29:54.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:29:54.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626208+0000", 2024-09-17T07:29:54.573 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1719e-05 2024-09-17T07:29:54.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.573 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:29:54.573 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627567+0000", 2024-09-17T07:29:54.573 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013587919999999999 2024-09-17T07:29:54.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:29:54.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:29:54.573 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:29:54.573 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627603+0000", 2024-09-17T07:29:54.573 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5830000000000001e-05 2024-09-17T07:29:54.573 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.573 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.573 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.574 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:29:54.574 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:29:54.574 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:54.575 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:54.574+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:54.575 INFO:teuthology.orchestra.run.smithi045.stdout:4779 writesame 0xd8ab6b6 thru 0xd8b4fae data_size 0x1(0x98f9 bytes) 2024-09-17T07:29:54.575 INFO:teuthology.orchestra.run.smithi045.stdout:4782 read 0x94c03eb thru 0x94cc588 (0xc19e bytes) 2024-09-17T07:29:54.575 INFO:teuthology.orchestra.run.smithi045.stdout:4784 read 0x7f053ec thru 0x7f11cef (0xc904 bytes) 2024-09-17T07:29:54.575 INFO:teuthology.orchestra.run.smithi045.stdout:4785 write 0x552ea63 thru 0x5536d0b (0x82a9 bytes) 2024-09-17T07:29:54.576 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:54.579 INFO:teuthology.orchestra.run.smithi045.stdout:4786 writesame 0x815c1b thru 0x81d08c data_size 0x37(0x7472 bytes) 2024-09-17T07:29:54.579 INFO:teuthology.orchestra.run.smithi045.stdout:4788 writesame 0x5e675bb thru 0x5e71dcf data_size 0x15(0xa815 bytes) 2024-09-17T07:29:54.579 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.579 INFO:teuthology.orchestra.run.smithi045.stdout:4790 punch from 0xc05be41 to 0xc061a60, (0x5c1f bytes) 2024-09-17T07:29:54.580 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.580 INFO:teuthology.orchestra.run.smithi045.stdout:4796 write 0xcc9b033 thru 0xcca4410 (0x93de bytes) 2024-09-17T07:29:54.580 INFO:teuthology.orchestra.run.smithi045.stdout:4797 read 0x3310412 thru 0x3310fd4 (0xbc3 bytes) 2024-09-17T07:29:54.580 INFO:teuthology.orchestra.run.smithi045.stdout:4798 write 0xad9c7b thru 0xae40c1 (0xa447 bytes) 2024-09-17T07:29:54.580 INFO:teuthology.orchestra.run.smithi045.stdout:4799 punch from 0x89e8a85 to 0x89ee7b8, (0x5d33 bytes) 2024-09-17T07:29:54.580 INFO:teuthology.orchestra.run.smithi045.stdout:4801 writesame 0x1cb2754 thru 0x1cb8606 data_size 0x1(0x5eb3 bytes) 2024-09-17T07:29:54.580 INFO:teuthology.orchestra.run.smithi045.stdout:4802 write 0x2e29500 thru 0x2e339a5 (0xa4a6 bytes) 2024-09-17T07:29:54.580 INFO:teuthology.orchestra.run.smithi045.stdout:4803 write 0x2371389 thru 0x23727db (0x1453 bytes) 2024-09-17T07:29:54.580 INFO:teuthology.orchestra.run.smithi045.stdout:4806 write 0xc0fe22f thru 0xc108cb6 (0xaa88 bytes) 2024-09-17T07:29:54.580 INFO:teuthology.orchestra.run.smithi045.stdout:4807 read 0x3b9dfba thru 0x3ba5a88 (0x7acf bytes) 2024-09-17T07:29:54.580 INFO:teuthology.orchestra.run.smithi045.stdout:4808 read 0xa56316f thru 0xa56731f (0x41b1 bytes) 2024-09-17T07:29:54.580 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.580 INFO:teuthology.orchestra.run.smithi045.stdout:4810 write 0x2285916 thru 0x228bdc4 (0x64af bytes) 2024-09-17T07:29:54.581 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.581 INFO:teuthology.orchestra.run.smithi045.stdout:4814 write 0xd02bb84 thru 0xd030b07 (0x4f84 bytes) 2024-09-17T07:29:54.581 INFO:teuthology.orchestra.run.smithi045.stdout:4815 read 0x5420b75 thru 0x542bae7 (0xaf73 bytes) 2024-09-17T07:29:54.581 INFO:teuthology.orchestra.run.smithi045.stdout:4816 write 0xd2abc52 thru 0xd2b932c (0xd6db bytes) 2024-09-17T07:29:54.581 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.581 INFO:teuthology.orchestra.run.smithi045.stdout:4819 write 0x2364ebf thru 0x236533c (0x47e bytes) 2024-09-17T07:29:54.581 INFO:teuthology.orchestra.run.smithi045.stdout:4820 punch from 0xad38dff to 0xad4167b, (0x887c bytes) 2024-09-17T07:29:54.581 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.581 INFO:teuthology.orchestra.run.smithi045.stdout:4822 punch from 0x9ea0acf to 0x9eac122, (0xb653 bytes) 2024-09-17T07:29:54.581 INFO:teuthology.orchestra.run.smithi045.stdout:4825 punch from 0x1f6522c to 0x1f73636, (0xe40a bytes) 2024-09-17T07:29:54.581 INFO:teuthology.orchestra.run.smithi045.stdout:4826 trunc from 0xd8b4faf to 0x288606e 2024-09-17T07:29:54.581 INFO:teuthology.orchestra.run.smithi045.stdout:4827 punch from 0x1a15e07 to 0x1a16efd, (0x10f6 bytes) 2024-09-17T07:29:54.581 INFO:teuthology.orchestra.run.smithi045.stdout:4828 punch from 0x1af0647 to 0x1afcabb, (0xc474 bytes) 2024-09-17T07:29:54.582 INFO:teuthology.orchestra.run.smithi045.stdout:4829 write 0x2cd89da thru 0x2cda176 (0x179d bytes) 2024-09-17T07:29:54.593 INFO:teuthology.orchestra.run.smithi045.stdout:4832 trunc from 0x2cda177 to 0x96f334c 2024-09-17T07:29:54.598 INFO:teuthology.orchestra.run.smithi045.stdout:4833 punch from 0x84f51f4 to 0x84f5430, (0x23c bytes) 2024-09-17T07:29:54.604 INFO:teuthology.orchestra.run.smithi045.stdout:4834 write 0x9b9b587 thru 0x9b9c451 (0xecb bytes) 2024-09-17T07:29:54.612 INFO:teuthology.orchestra.run.smithi045.stdout:4835 trunc from 0x9b9c452 to 0x4f5bf8c 2024-09-17T07:29:54.640 INFO:teuthology.orchestra.run.smithi045.stdout:4836 write 0xe0d6fba thru 0xe0e1574 (0xa5bb bytes) 2024-09-17T07:29:54.652 INFO:teuthology.orchestra.run.smithi045.stdout:4837 trunc from 0xe0e1575 to 0x6bf8214 2024-09-17T07:29:54.663 INFO:teuthology.orchestra.run.smithi045.stdout:4838 read 0x6b16301 thru 0x6b238c0 (0xd5c0 bytes) 2024-09-17T07:29:54.663 INFO:teuthology.orchestra.run.smithi045.stdout:4839 read 0x6ab0dc1 thru 0x6ab3ebe (0x30fe bytes) 2024-09-17T07:29:54.664 INFO:teuthology.orchestra.run.smithi045.stdout:4840 punch from 0x860449 to 0x863e61, (0x3a18 bytes) 2024-09-17T07:29:54.672 INFO:teuthology.orchestra.run.smithi045.stdout:4842 trunc from 0x6bf8214 to 0xe6af542 2024-09-17T07:29:54.678 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:54.677+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:54.678 INFO:teuthology.orchestra.run.smithi045.stdout:4843 read 0x18c231d thru 0x18cf682 (0xd366 bytes) 2024-09-17T07:29:54.679 INFO:teuthology.orchestra.run.smithi045.stdout:4844 writesame 0x770c71e thru 0x7717a37 data_size 0x32(0xb31a bytes) 2024-09-17T07:29:54.686 INFO:teuthology.orchestra.run.smithi045.stdout:4845 writesame 0x34adef0 thru 0x34b8039 data_size 0x5(0xa14a bytes) 2024-09-17T07:29:54.687 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:54.687 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:54.688 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:54.692 INFO:teuthology.orchestra.run.smithi045.stdout:4847 read 0x2df42a0 thru 0x2dfa5b7 (0x6318 bytes) 2024-09-17T07:29:54.692 INFO:teuthology.orchestra.run.smithi045.stdout:4850 read 0x9c5f014 thru 0x9c6e3e8 (0xf3d5 bytes) 2024-09-17T07:29:54.693 INFO:teuthology.orchestra.run.smithi045.stdout:4852 read 0x15ebc23 thru 0x15f50c8 (0x94a6 bytes) 2024-09-17T07:29:54.693 INFO:teuthology.orchestra.run.smithi045.stdout:4853 write 0xbd11791 thru 0xbd15633 (0x3ea3 bytes) 2024-09-17T07:29:54.697 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:54.697 INFO:teuthology.orchestra.run.smithi045.stdout:4856 read 0x43d9114 thru 0x43dffa3 (0x6e90 bytes) 2024-09-17T07:29:54.697 INFO:teuthology.orchestra.run.smithi045.stdout:4858 writesame 0x1863c4e thru 0x187281a data_size 0x1(0xebcd bytes) 2024-09-17T07:29:54.698 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:54.723 INFO:teuthology.orchestra.run.smithi045.stdout:4859 writesame 0xb52a710 thru 0xb537d73 data_size 0x2(0xd664 bytes) 2024-09-17T07:29:54.736 INFO:teuthology.orchestra.run.smithi045.stdout:4860 read 0x9fb58f6 thru 0x9fb9eae (0x45b9 bytes) 2024-09-17T07:29:54.736 INFO:teuthology.orchestra.run.smithi045.stdout:4862 punch from 0x9f432d1 to 0x9f463f6, (0x3125 bytes) 2024-09-17T07:29:54.736 INFO:teuthology.orchestra.run.smithi045.stdout:4865 write 0xb4b924 thru 0xb568b4 (0xaf91 bytes) 2024-09-17T07:29:54.742 INFO:teuthology.orchestra.run.smithi045.stdout:4866 read 0xad33574 thru 0xad41561 (0xdfee bytes) 2024-09-17T07:29:54.742 INFO:teuthology.orchestra.run.smithi045.stdout:4867 clone 39 order 24 su 131072 sc 12 2024-09-17T07:29:54.779 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:54.778+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:54.807 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:54.807 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:54.807 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:54.807 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:54.807 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:54.816 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:54.899 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:54.899+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:54.960 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:54.960 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:54.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:54.960 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:54.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.961 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:30525 3.1b 3:def7c8f0:::rbd_data.118381ffbbab.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T07:29:54.961 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:54.961 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.436504456999998, 2024-09-17T07:29:54.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55515850300000003, 2024-09-17T07:29:54.961 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:54.961 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:54.961 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:54.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:54.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:54.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:54.961 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30525 2024-09-17T07:29:54.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.961 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:54.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:54.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:54.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:54.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:29:54.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:54.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522300+0000", 2024-09-17T07:29:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:29:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522305+0000", 2024-09-17T07:29:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-17T07:29:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522318+0000", 2024-09-17T07:29:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3030999999999999e-05 2024-09-17T07:29:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522326+0000", 2024-09-17T07:29:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4200000000000001e-06 2024-09-17T07:29:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522385+0000", 2024-09-17T07:29:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9005000000000002e-05 2024-09-17T07:29:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:29:54.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522393+0000", 2024-09-17T07:29:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-06 2024-09-17T07:29:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075347+0000", 2024-09-17T07:29:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55295417800000002 2024-09-17T07:29:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075502+0000", 2024-09-17T07:29:54.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154272 2024-09-17T07:29:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:29:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075630+0000", 2024-09-17T07:29:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012816300000000001 2024-09-17T07:29:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077221+0000", 2024-09-17T07:29:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015912929999999999 2024-09-17T07:29:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:54.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077353+0000", 2024-09-17T07:29:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013215699999999999 2024-09-17T07:29:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077413+0000", 2024-09-17T07:29:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0034999999999997e-05 2024-09-17T07:29:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077457+0000", 2024-09-17T07:29:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4017999999999998e-05 2024-09-17T07:29:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:54.967 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44181 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13922304~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:29:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.321325809000001, 2024-09-17T07:29:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046010502000000002, 2024-09-17T07:29:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44181 2024-09-17T07:29:54.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:29:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:29:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637478+0000", 2024-09-17T07:29:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:29:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637480+0000", 2024-09-17T07:29:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.843e-06 2024-09-17T07:29:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637487+0000", 2024-09-17T07:29:54.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6159999999999996e-06 2024-09-17T07:29:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637492+0000", 2024-09-17T07:29:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5360000000000003e-06 2024-09-17T07:29:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683351+0000", 2024-09-17T07:29:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045858758999999999 2024-09-17T07:29:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683422+0000", 2024-09-17T07:29:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1705999999999994e-05 2024-09-17T07:29:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683488+0000", 2024-09-17T07:29:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5612999999999997e-05 2024-09-17T07:29:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:54.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44182 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13938688~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.321305138, 2024-09-17T07:29:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046153118999999999, 2024-09-17T07:29:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44182 2024-09-17T07:29:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.973 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:29:54.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:29:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637506+0000", 2024-09-17T07:29:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1520000000000003e-06 2024-09-17T07:29:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637512+0000", 2024-09-17T07:29:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-09-17T07:29:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637517+0000", 2024-09-17T07:29:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-09-17T07:29:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683514+0000", 2024-09-17T07:29:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045997616999999998 2024-09-17T07:29:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683584+0000", 2024-09-17T07:29:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9942000000000002e-05 2024-09-17T07:29:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683651+0000", 2024-09-17T07:29:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6951000000000003e-05 2024-09-17T07:29:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44183 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13955072~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:29:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.321279906000001, 2024-09-17T07:29:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046339793999999997, 2024-09-17T07:29:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:54.977 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44183 2024-09-17T07:29:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:29:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:29:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639583+0000", 2024-09-17T07:29:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002059412 2024-09-17T07:29:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639597+0000", 2024-09-17T07:29:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3832000000000001e-05 2024-09-17T07:29:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639607+0000", 2024-09-17T07:29:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485e-05 2024-09-17T07:29:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639614+0000", 2024-09-17T07:29:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0849999999999999e-06 2024-09-17T07:29:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683705+0000", 2024-09-17T07:29:54.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044091261 2024-09-17T07:29:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683792+0000", 2024-09-17T07:29:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6197000000000002e-05 2024-09-17T07:29:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683863+0000", 2024-09-17T07:29:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1521999999999997e-05 2024-09-17T07:29:54.981 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44199 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14217216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.317762354000003, 2024-09-17T07:29:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045995623999999999, 2024-09-17T07:29:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:54.982 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44199 2024-09-17T07:29:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:29:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:29:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641043+0000", 2024-09-17T07:29:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.624e-06 2024-09-17T07:29:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641048+0000", 2024-09-17T07:29:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4029999999999998e-06 2024-09-17T07:29:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641053+0000", 2024-09-17T07:29:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6670000000000004e-06 2024-09-17T07:29:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686918+0000", 2024-09-17T07:29:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045865131000000003 2024-09-17T07:29:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686986+0000", 2024-09-17T07:29:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8299000000000005e-05 2024-09-17T07:29:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687036+0000", 2024-09-17T07:29:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0164999999999998e-05 2024-09-17T07:29:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44200 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14233600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:54.986 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.317744525000002, 2024-09-17T07:29:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046183984999999997, 2024-09-17T07:29:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44200 2024-09-17T07:29:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:54.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:29:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641061+0000", 2024-09-17T07:29:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.767e-06 2024-09-17T07:29:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641067+0000", 2024-09-17T07:29:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7379999999999999e-06 2024-09-17T07:29:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641071+0000", 2024-09-17T07:29:54.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-09-17T07:29:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687078+0000", 2024-09-17T07:29:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046007095999999997 2024-09-17T07:29:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687158+0000", 2024-09-17T07:29:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0374e-05 2024-09-17T07:29:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687243+0000", 2024-09-17T07:29:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4221999999999994e-05 2024-09-17T07:29:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44201 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14249984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:29:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.317726878999999, 2024-09-17T07:29:54.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046303182999999998, 2024-09-17T07:29:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44201 2024-09-17T07:29:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:29:54.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:29:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641120+0000", 2024-09-17T07:29:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3933000000000001e-05 2024-09-17T07:29:54.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641123+0000", 2024-09-17T07:29:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.191e-06 2024-09-17T07:29:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641133+0000", 2024-09-17T07:29:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4020000000000008e-06 2024-09-17T07:29:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641138+0000", 2024-09-17T07:29:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6559999999999997e-06 2024-09-17T07:29:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687279+0000", 2024-09-17T07:29:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046140278999999999 2024-09-17T07:29:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687338+0000", 2024-09-17T07:29:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9268999999999998e-05 2024-09-17T07:29:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687379+0000", 2024-09-17T07:29:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1452999999999998e-05 2024-09-17T07:29:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44202 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14266368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:29:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.317656112999998, 2024-09-17T07:29:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046484299, 2024-09-17T07:29:54.996 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44202 2024-09-17T07:29:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:29:54.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:29:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642810+0000", 2024-09-17T07:29:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016627980000000001 2024-09-17T07:29:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642815+0000", 2024-09-17T07:29:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.091e-06 2024-09-17T07:29:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642829+0000", 2024-09-17T07:29:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4047e-05 2024-09-17T07:29:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642844+0000", 2024-09-17T07:29:54.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4515000000000001e-05 2024-09-17T07:29:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687426+0000", 2024-09-17T07:29:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044582201000000002 2024-09-17T07:29:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687529+0000", 2024-09-17T07:29:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103078 2024-09-17T07:29:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687631+0000", 2024-09-17T07:29:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102569 2024-09-17T07:29:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44209 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14381056~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:29:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.315757961000003, 2024-09-17T07:29:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045944444000000001, 2024-09-17T07:29:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.001 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44209 2024-09-17T07:29:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:29:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:29:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643046+0000", 2024-09-17T07:29:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:29:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643048+0000", 2024-09-17T07:29:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:29:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643054+0000", 2024-09-17T07:29:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.032e-06 2024-09-17T07:29:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643065+0000", 2024-09-17T07:29:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1861999999999999e-05 2024-09-17T07:29:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688806+0000", 2024-09-17T07:29:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045740617999999997 2024-09-17T07:29:55.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688927+0000", 2024-09-17T07:29:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000121327 2024-09-17T07:29:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688990+0000", 2024-09-17T07:29:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2293999999999997e-05 2024-09-17T07:29:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44210 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14397440~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.315731034999999, 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046104635999999997, 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44210 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:29:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:29:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643073+0000", 2024-09-17T07:29:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.77e-07 2024-09-17T07:29:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643075+0000", 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643082+0000", 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7789999999999996e-06 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643087+0000", 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.967e-06 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689022+0000", 2024-09-17T07:29:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045934349999999999 2024-09-17T07:29:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689112+0000", 2024-09-17T07:29:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0039999999999999e-05 2024-09-17T07:29:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689177+0000", 2024-09-17T07:29:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5041000000000003e-05 2024-09-17T07:29:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.009 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44211 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14413824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:29:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.315709902000002, 2024-09-17T07:29:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046309003000000001, 2024-09-17T07:29:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44211 2024-09-17T07:29:55.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643094+0000", 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:29:55.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643095+0000", 2024-09-17T07:29:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-17T07:29:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643101+0000", 2024-09-17T07:29:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8730000000000004e-06 2024-09-17T07:29:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643106+0000", 2024-09-17T07:29:55.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6859999999999997e-06 2024-09-17T07:29:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689220+0000", 2024-09-17T07:29:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046113945000000003 2024-09-17T07:29:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689309+0000", 2024-09-17T07:29:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9560000000000003e-05 2024-09-17T07:29:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689402+0000", 2024-09-17T07:29:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2786000000000002e-05 2024-09-17T07:29:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44212 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14430208~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.315665340000002, 2024-09-17T07:29:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046448912000000002, 2024-09-17T07:29:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.014 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44212 2024-09-17T07:29:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:55.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2200000000000004e-07 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643141+0000", 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5210000000000001e-06 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643149+0000", 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6490000000000008e-06 2024-09-17T07:29:55.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643161+0000", 2024-09-17T07:29:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2241e-05 2024-09-17T07:29:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689423+0000", 2024-09-17T07:29:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046262323000000001 2024-09-17T07:29:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689505+0000", 2024-09-17T07:29:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2103999999999999e-05 2024-09-17T07:29:55.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689587+0000", 2024-09-17T07:29:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1452000000000006e-05 2024-09-17T07:29:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44213 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14446592~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:29:55.018 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.315633841, 2024-09-17T07:29:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046601780000000002, 2024-09-17T07:29:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44213 2024-09-17T07:29:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:29:55.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:29:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643170+0000", 2024-09-17T07:29:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-17T07:29:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643172+0000", 2024-09-17T07:29:55.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1660000000000001e-06 2024-09-17T07:29:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643180+0000", 2024-09-17T07:29:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-09-17T07:29:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643185+0000", 2024-09-17T07:29:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-09-17T07:29:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689622+0000", 2024-09-17T07:29:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046436742000000003 2024-09-17T07:29:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689697+0000", 2024-09-17T07:29:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5179999999999995e-05 2024-09-17T07:29:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689771+0000", 2024-09-17T07:29:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3928000000000001e-05 2024-09-17T07:29:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.022 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44214 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14462976~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:29:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.315610780999997, 2024-09-17T07:29:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046736038000000001, 2024-09-17T07:29:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.023 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44214 2024-09-17T07:29:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:29:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:29:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643193+0000", 2024-09-17T07:29:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:29:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643195+0000", 2024-09-17T07:29:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0640000000000001e-06 2024-09-17T07:29:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643201+0000", 2024-09-17T07:29:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2859999999999996e-06 2024-09-17T07:29:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643206+0000", 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689814+0000", 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046607742000000001 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689886+0000", 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2020999999999994e-05 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689928+0000", 2024-09-17T07:29:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2771000000000001e-05 2024-09-17T07:29:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44215 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14479360~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:29:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.315566883999999, 2024-09-17T07:29:55.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046868543999999998, 2024-09-17T07:29:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44215 2024-09-17T07:29:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:29:55.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:29:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643237+0000", 2024-09-17T07:29:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:29:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643245+0000", 2024-09-17T07:29:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3270000000000001e-06 2024-09-17T07:29:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643254+0000", 2024-09-17T07:29:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6049999999999998e-06 2024-09-17T07:29:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643260+0000", 2024-09-17T07:29:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7849999999999997e-06 2024-09-17T07:29:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689961+0000", 2024-09-17T07:29:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046701196 2024-09-17T07:29:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690036+0000", 2024-09-17T07:29:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5153000000000004e-05 2024-09-17T07:29:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690105+0000", 2024-09-17T07:29:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8874000000000005e-05 2024-09-17T07:29:55.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44216 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14495744~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:29:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.315535171000001, 2024-09-17T07:29:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047003047999999999, 2024-09-17T07:29:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.032 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44216 2024-09-17T07:29:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:29:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:29:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643269+0000", 2024-09-17T07:29:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:29:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643271+0000", 2024-09-17T07:29:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-09-17T07:29:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643278+0000", 2024-09-17T07:29:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8850000000000001e-06 2024-09-17T07:29:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643282+0000", 2024-09-17T07:29:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5979999999999999e-06 2024-09-17T07:29:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690133+0000", 2024-09-17T07:29:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850470999999998 2024-09-17T07:29:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690205+0000", 2024-09-17T07:29:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2442999999999997e-05 2024-09-17T07:29:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690271+0000", 2024-09-17T07:29:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5874e-05 2024-09-17T07:29:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44217 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14512128~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.315514159000003, 2024-09-17T07:29:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047166485000000001, 2024-09-17T07:29:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44217 2024-09-17T07:29:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:29:55.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-17T07:29:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643291+0000", 2024-09-17T07:29:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.962e-06 2024-09-17T07:29:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643297+0000", 2024-09-17T07:29:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0700000000000003e-06 2024-09-17T07:29:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643302+0000", 2024-09-17T07:29:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:29:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690308+0000", 2024-09-17T07:29:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047005354999999999 2024-09-17T07:29:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690385+0000", 2024-09-17T07:29:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7541999999999997e-05 2024-09-17T07:29:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690455+0000", 2024-09-17T07:29:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0328999999999997e-05 2024-09-17T07:29:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44218 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14528512~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.315494481999998, 2024-09-17T07:29:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047321015000000001, 2024-09-17T07:29:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.041 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44218 2024-09-17T07:29:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:55.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:29:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-17T07:29:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643311+0000", 2024-09-17T07:29:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.933e-06 2024-09-17T07:29:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643318+0000", 2024-09-17T07:29:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3980000000000004e-06 2024-09-17T07:29:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643322+0000", 2024-09-17T07:29:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7609999999999998e-06 2024-09-17T07:29:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690484+0000", 2024-09-17T07:29:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047161428999999998 2024-09-17T07:29:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690564+0000", 2024-09-17T07:29:55.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9845000000000005e-05 2024-09-17T07:29:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690630+0000", 2024-09-17T07:29:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6021999999999999e-05 2024-09-17T07:29:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44219 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14544896~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.315474115000001, 2024-09-17T07:29:55.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048128708999999999, 2024-09-17T07:29:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44219 2024-09-17T07:29:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:29:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:29:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643337+0000", 2024-09-17T07:29:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.109e-06 2024-09-17T07:29:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643350+0000", 2024-09-17T07:29:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3046e-05 2024-09-17T07:29:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643356+0000", 2024-09-17T07:29:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8930000000000002e-06 2024-09-17T07:29:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690660+0000", 2024-09-17T07:29:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047304061000000001 2024-09-17T07:29:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691372+0000", 2024-09-17T07:29:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00071281299999999997 2024-09-17T07:29:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691458+0000", 2024-09-17T07:29:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5352000000000005e-05 2024-09-17T07:29:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44220 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14561280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:55.049 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:29:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.315440066999997, 2024-09-17T07:29:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048309849000000002, 2024-09-17T07:29:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44220 2024-09-17T07:29:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:29:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:29:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646869+0000", 2024-09-17T07:29:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003506267 2024-09-17T07:29:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646874+0000", 2024-09-17T07:29:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7500000000000003e-06 2024-09-17T07:29:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646887+0000", 2024-09-17T07:29:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3115999999999999e-05 2024-09-17T07:29:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646894+0000", 2024-09-17T07:29:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5289999999999997e-06 2024-09-17T07:29:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691547+0000", 2024-09-17T07:29:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044652743000000002 2024-09-17T07:29:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691622+0000", 2024-09-17T07:29:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5037999999999995e-05 2024-09-17T07:29:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691673+0000", 2024-09-17T07:29:55.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-17T07:29:55.054 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.054 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.054 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.054 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.054 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.054 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:55.055 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:55.056 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:55.055+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:55.115 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-17T07:29:55.170 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:55.169+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:55.187 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone38 from 0xe6af542 (overlap 0x1df7440) to 0x1af8239 2024-09-17T07:29:55.214 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:55.214 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:55.214 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:55.225 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:55.268 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:55.268+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:55.287 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-17T07:29:55.334 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:55.334 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:55.335 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:55.335 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:55.335 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:55.343 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:55.358 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:55.356+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:55.428 INFO:teuthology.orchestra.run.smithi045.stdout:4868 write 0x25bdff1 thru 0x25bfeb5 (0x1ec5 bytes) 2024-09-17T07:29:55.452 INFO:teuthology.orchestra.run.smithi045.stdout:4869 read 0xcd2e032 thru 0xcd395b9 (0xb588 bytes) 2024-09-17T07:29:55.452 INFO:teuthology.orchestra.run.smithi045.stdout:4870 read 0x3fb6d09 thru 0x3fc1dbb (0xb0b3 bytes) 2024-09-17T07:29:55.452 INFO:teuthology.orchestra.run.smithi045.stdout:4871 write 0x4af161c thru 0x4b01405 (0xfdea bytes) 2024-09-17T07:29:55.459 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:55.459+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:55.481 INFO:teuthology.orchestra.run.smithi045.stdout:4872 punch from 0xe0d5d35 to 0xe0d9532, (0x37fd bytes) 2024-09-17T07:29:55.484 INFO:teuthology.orchestra.run.smithi045.stdout:4874 trunc from 0xe6af542 to 0xbe120a6 2024-09-17T07:29:55.519 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19985 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5275648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.264623534999998, 2024-09-17T07:29:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071819998999999995, 2024-09-17T07:29:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19985 2024-09-17T07:29:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.520 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:55.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252493+0000", 2024-09-17T07:29:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6300000000000004e-07 2024-09-17T07:29:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252496+0000", 2024-09-17T07:29:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-17T07:29:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252499+0000", 2024-09-17T07:29:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:29:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324036+0000", 2024-09-17T07:29:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071536587999999998 2024-09-17T07:29:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324118+0000", 2024-09-17T07:29:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2336999999999995e-05 2024-09-17T07:29:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324312+0000", 2024-09-17T07:29:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000193709 2024-09-17T07:29:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19986 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5505024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.264612382999999, 2024-09-17T07:29:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072083761999999996, 2024-09-17T07:29:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.524 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19986 2024-09-17T07:29:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:29:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252508+0000", 2024-09-17T07:29:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:29:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252511+0000", 2024-09-17T07:29:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.145e-06 2024-09-17T07:29:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252514+0000", 2024-09-17T07:29:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5569999999999998e-06 2024-09-17T07:29:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324326+0000", 2024-09-17T07:29:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071812681000000003 2024-09-17T07:29:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324372+0000", 2024-09-17T07:29:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5893000000000003e-05 2024-09-17T07:29:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324587+0000", 2024-09-17T07:29:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214608 2024-09-17T07:29:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19987 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5734400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.264598765000002, 2024-09-17T07:29:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072276676999999998, 2024-09-17T07:29:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.529 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19987 2024-09-17T07:29:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252518+0000", 2024-09-17T07:29:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-17T07:29:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252522+0000", 2024-09-17T07:29:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2440000000000004e-06 2024-09-17T07:29:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252525+0000", 2024-09-17T07:29:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5919999999999999e-06 2024-09-17T07:29:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324621+0000", 2024-09-17T07:29:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072096442999999996 2024-09-17T07:29:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324699+0000", 2024-09-17T07:29:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8061999999999999e-05 2024-09-17T07:29:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324793+0000", 2024-09-17T07:29:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4207000000000001e-05 2024-09-17T07:29:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.533 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19988 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.264587794000001, 2024-09-17T07:29:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072585798000000007, 2024-09-17T07:29:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19988 2024-09-17T07:29:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252529+0000", 2024-09-17T07:29:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:29:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252532+0000", 2024-09-17T07:29:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9579999999999999e-06 2024-09-17T07:29:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252534+0000", 2024-09-17T07:29:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5730000000000002e-06 2024-09-17T07:29:55.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324825+0000", 2024-09-17T07:29:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072290757999999997 2024-09-17T07:29:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324874+0000", 2024-09-17T07:29:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9360000000000002e-05 2024-09-17T07:29:55.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325113+0000", 2024-09-17T07:29:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000239025 2024-09-17T07:29:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19989 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.264578172999997, 2024-09-17T07:29:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072963494000000004, 2024-09-17T07:29:55.538 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19989 2024-09-17T07:29:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:29:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:29:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:29:55.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2900000000000002e-07 2024-09-17T07:29:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252542+0000", 2024-09-17T07:29:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8600000000000003e-06 2024-09-17T07:29:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252545+0000", 2024-09-17T07:29:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.605e-06 2024-09-17T07:29:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325144+0000", 2024-09-17T07:29:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072599460000000005 2024-09-17T07:29:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325223+0000", 2024-09-17T07:29:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8473999999999994e-05 2024-09-17T07:29:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325501+0000", 2024-09-17T07:29:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000278063 2024-09-17T07:29:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.542 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19990 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6422528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.264567702000001, 2024-09-17T07:29:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073241127000000003, 2024-09-17T07:29:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19990 2024-09-17T07:29:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.543 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:55.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252552+0000", 2024-09-17T07:29:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4719999999999997e-06 2024-09-17T07:29:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252557+0000", 2024-09-17T07:29:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3039999999999998e-06 2024-09-17T07:29:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252560+0000", 2024-09-17T07:29:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8830000000000002e-06 2024-09-17T07:29:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325529+0000", 2024-09-17T07:29:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072968880999999999 2024-09-17T07:29:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325574+0000", 2024-09-17T07:29:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466000000000001e-05 2024-09-17T07:29:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325789+0000", 2024-09-17T07:29:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021490999999999999 2024-09-17T07:29:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19991 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6651904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.264548400000002, 2024-09-17T07:29:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073720552999999994, 2024-09-17T07:29:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.547 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19991 2024-09-17T07:29:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4699999999999998e-07 2024-09-17T07:29:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252568+0000", 2024-09-17T07:29:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5300000000000002e-07 2024-09-17T07:29:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252572+0000", 2024-09-17T07:29:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9160000000000003e-06 2024-09-17T07:29:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252575+0000", 2024-09-17T07:29:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:29:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325820+0000", 2024-09-17T07:29:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073245597999999995 2024-09-17T07:29:55.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325901+0000", 2024-09-17T07:29:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0913000000000003e-05 2024-09-17T07:29:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326288+0000", 2024-09-17T07:29:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038646900000000002 2024-09-17T07:29:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19992 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6881280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.264538047000002, 2024-09-17T07:29:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074054683999999996, 2024-09-17T07:29:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19992 2024-09-17T07:29:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.552 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252578+0000", 2024-09-17T07:29:55.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:29:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252579+0000", 2024-09-17T07:29:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2500000000000004e-07 2024-09-17T07:29:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252582+0000", 2024-09-17T07:29:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-17T07:29:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252585+0000", 2024-09-17T07:29:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.542e-06 2024-09-17T07:29:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326315+0000", 2024-09-17T07:29:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073729921000000004 2024-09-17T07:29:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326359+0000", 2024-09-17T07:29:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4095999999999999e-05 2024-09-17T07:29:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326632+0000", 2024-09-17T07:29:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027314499999999997 2024-09-17T07:29:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19993 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7110656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.264527743000002, 2024-09-17T07:29:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074425142999999999, 2024-09-17T07:29:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.556 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19993 2024-09-17T07:29:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:29:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252589+0000", 2024-09-17T07:29:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.33e-07 2024-09-17T07:29:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252593+0000", 2024-09-17T07:29:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:29:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252595+0000", 2024-09-17T07:29:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.621e-06 2024-09-17T07:29:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326666+0000", 2024-09-17T07:29:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074070787999999999 2024-09-17T07:29:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326752+0000", 2024-09-17T07:29:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5748000000000006e-05 2024-09-17T07:29:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327013+0000", 2024-09-17T07:29:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026098400000000002 2024-09-17T07:29:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.560 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19994 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7340032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.264517284, 2024-09-17T07:29:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074765920999999999, 2024-09-17T07:29:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19994 2024-09-17T07:29:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:29:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252599+0000", 2024-09-17T07:29:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2099999999999995e-07 2024-09-17T07:29:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252603+0000", 2024-09-17T07:29:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:29:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252606+0000", 2024-09-17T07:29:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-09-17T07:29:55.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327047+0000", 2024-09-17T07:29:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074441050999999994 2024-09-17T07:29:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327121+0000", 2024-09-17T07:29:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4283999999999996e-05 2024-09-17T07:29:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327364+0000", 2024-09-17T07:29:55.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000243155 2024-09-17T07:29:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19995 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7569408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.264506587, 2024-09-17T07:29:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075172694999999998, 2024-09-17T07:29:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19995 2024-09-17T07:29:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252610+0000", 2024-09-17T07:29:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6799999999999999e-07 2024-09-17T07:29:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252614+0000", 2024-09-17T07:29:55.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000002e-06 2024-09-17T07:29:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252616+0000", 2024-09-17T07:29:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.419e-06 2024-09-17T07:29:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327407+0000", 2024-09-17T07:29:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074790617000000004 2024-09-17T07:29:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327498+0000", 2024-09-17T07:29:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0758000000000001e-05 2024-09-17T07:29:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327782+0000", 2024-09-17T07:29:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028395599999999998 2024-09-17T07:29:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7798784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:55.569 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.264496270999999, 2024-09-17T07:29:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075605832999999997, 2024-09-17T07:29:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:29:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:55.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:29:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:29:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:29:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:29:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:29:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:29:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327844+0000", 2024-09-17T07:29:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075217709999999993 2024-09-17T07:29:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327927+0000", 2024-09-17T07:29:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2628999999999998e-05 2024-09-17T07:29:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328225+0000", 2024-09-17T07:29:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029800600000000002 2024-09-17T07:29:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.573 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8028160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.264485917999998, 2024-09-17T07:29:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076061237000000004, 2024-09-17T07:29:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.574 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:29:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:29:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:29:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:29:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:29:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:29:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:29:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328268+0000", 2024-09-17T07:29:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075626323999999995 2024-09-17T07:29:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328352+0000", 2024-09-17T07:29:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3578999999999994e-05 2024-09-17T07:29:55.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328691+0000", 2024-09-17T07:29:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033915099999999999 2024-09-17T07:29:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8257536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.264470467000002, 2024-09-17T07:29:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076441903000000005, 2024-09-17T07:29:55.578 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:29:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:29:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:29:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:29:55.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:29:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:29:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:29:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:29:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328747+0000", 2024-09-17T07:29:55.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076094192000000005 2024-09-17T07:29:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328824+0000", 2024-09-17T07:29:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7096999999999997e-05 2024-09-17T07:29:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.329087+0000", 2024-09-17T07:29:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026249999999999998 2024-09-17T07:29:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20029 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.254385479, 2024-09-17T07:29:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071783696999999994, 2024-09-17T07:29:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20029 2024-09-17T07:29:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.583 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262735+0000", 2024-09-17T07:29:55.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.955e-06 2024-09-17T07:29:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262737+0000", 2024-09-17T07:29:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0090000000000002e-06 2024-09-17T07:29:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262743+0000", 2024-09-17T07:29:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2890000000000003e-06 2024-09-17T07:29:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262748+0000", 2024-09-17T07:29:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334163+0000", 2024-09-17T07:29:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071415336999999995 2024-09-17T07:29:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334243+0000", 2024-09-17T07:29:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9548000000000005e-05 2024-09-17T07:29:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334514+0000", 2024-09-17T07:29:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027086800000000001 2024-09-17T07:29:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7356416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.254354308000003, 2024-09-17T07:29:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072172616999999994, 2024-09-17T07:29:55.587 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:29:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.588 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:29:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:29:55.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:29:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:29:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:29:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:29:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:29:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:29:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334558+0000", 2024-09-17T07:29:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071780790999999997 2024-09-17T07:29:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334647+0000", 2024-09-17T07:29:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9448000000000001e-05 2024-09-17T07:29:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334934+0000", 2024-09-17T07:29:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028654599999999999 2024-09-17T07:29:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7585792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.254332595999998, 2024-09-17T07:29:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072606135000000002, 2024-09-17T07:29:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.592 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:29:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:29:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:29:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:29:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:29:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:29:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:29:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:29:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335116+0000", 2024-09-17T07:29:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072323897999999998 2024-09-17T07:29:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335166+0000", 2024-09-17T07:29:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9882000000000001e-05 2024-09-17T07:29:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335389+0000", 2024-09-17T07:29:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022334300000000001 2024-09-17T07:29:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.596 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7815168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.254320397000001, 2024-09-17T07:29:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073051230999999994, 2024-09-17T07:29:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:29:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:29:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:29:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:29:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:29:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:29:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:29:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335431+0000", 2024-09-17T07:29:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072627852000000007 2024-09-17T07:29:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335525+0000", 2024-09-17T07:29:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3708000000000005e-05 2024-09-17T07:29:55.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335846+0000", 2024-09-17T07:29:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032182000000000001 2024-09-17T07:29:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.601 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.254309538999998, 2024-09-17T07:29:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073284199999999994, 2024-09-17T07:29:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:29:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:29:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:29:55.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:29:55.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:29:55.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:29:55.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:29:55.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:29:55.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335863+0000", 2024-09-17T07:29:55.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073048921000000003 2024-09-17T07:29:55.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335904+0000", 2024-09-17T07:29:55.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1545000000000003e-05 2024-09-17T07:29:55.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336090+0000", 2024-09-17T07:29:55.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185787 2024-09-17T07:29:55.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.606 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.606 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.606 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8273920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:55.606 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:55.606 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.254295278999997, 2024-09-17T07:29:55.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073670478999999997, 2024-09-17T07:29:55.606 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:55.606 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:55.606 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:55.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:55.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:55.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:55.607 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:29:55.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.607 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:55.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:55.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:55.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:55.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:55.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:55.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:55.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:55.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:29:55.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:55.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:29:55.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:29:55.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:55.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:29:55.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:29:55.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:55.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:29:55.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:29:55.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:55.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336126+0000", 2024-09-17T07:29:55.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073297254000000006 2024-09-17T07:29:55.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:55.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336211+0000", 2024-09-17T07:29:55.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4728000000000006e-05 2024-09-17T07:29:55.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:55.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:55.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:55.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336491+0000", 2024-09-17T07:29:55.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027979399999999998 2024-09-17T07:29:55.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.610 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:55.610 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:55.610 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:55.612 INFO:teuthology.orchestra.run.smithi045.stdout:4875 write 0x43f15f0 thru 0x43f4583 (0x2f94 bytes) 2024-09-17T07:29:55.612 INFO:teuthology.orchestra.run.smithi045.stdout:4877 trunc from 0xbe120a6 to 0x3846bc5 2024-09-17T07:29:55.612 INFO:teuthology.orchestra.run.smithi045.stdout:4878 write 0x9f2dece thru 0x9f3d2f3 (0xf426 bytes) 2024-09-17T07:29:55.612 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:55.612 INFO:teuthology.orchestra.run.smithi045.stdout:4882 writesame 0xeccad5 thru 0xed6d6c data_size 0x16(0xa298 bytes) 2024-09-17T07:29:55.613 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:55.613 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:55.611+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:55.613 INFO:teuthology.orchestra.run.smithi045.stdout:4887 trunc from 0x9f3d2f4 to 0x7fffd5c 2024-09-17T07:29:55.625 INFO:teuthology.orchestra.run.smithi045.stdout:4888 punch from 0x5d12c04 to 0x5d1bbec, (0x8fe8 bytes) 2024-09-17T07:29:55.625 INFO:teuthology.orchestra.run.smithi045.stdout:4889 read 0x14005ca thru 0x140c579 (0xbfb0 bytes) 2024-09-17T07:29:55.626 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:55.626 INFO:teuthology.orchestra.run.smithi045.stdout:4891 clone 40 order 25 su 1048576 sc 7 2024-09-17T07:29:55.698 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:29:55.712 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:55.711+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:55.756 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:29:55.757 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:55.758 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:55.758 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:55.758 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:55.764 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:55.826 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:55.826 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:55.843 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:55.848 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:55.848+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:55.860 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:55.861 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:55.861 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:55.861 INFO:teuthology.orchestra.run.smithi045.stderr:no valid command found; 10 closest matches: 2024-09-17T07:29:55.861 INFO:teuthology.orchestra.run.smithi045.stderr:0 2024-09-17T07:29:55.862 INFO:teuthology.orchestra.run.smithi045.stderr:1 2024-09-17T07:29:55.862 INFO:teuthology.orchestra.run.smithi045.stderr:2 2024-09-17T07:29:55.862 INFO:teuthology.orchestra.run.smithi045.stderr:abort 2024-09-17T07:29:55.862 INFO:teuthology.orchestra.run.smithi045.stderr:assert 2024-09-17T07:29:55.862 INFO:teuthology.orchestra.run.smithi045.stderr:config diff 2024-09-17T07:29:55.862 INFO:teuthology.orchestra.run.smithi045.stderr:config diff get 2024-09-17T07:29:55.862 INFO:teuthology.orchestra.run.smithi045.stderr:config get 2024-09-17T07:29:55.862 INFO:teuthology.orchestra.run.smithi045.stderr:config help [] 2024-09-17T07:29:55.862 INFO:teuthology.orchestra.run.smithi045.stderr:config set ... 2024-09-17T07:29:55.862 INFO:teuthology.orchestra.run.smithi045.stderr:admin_socket: invalid command 2024-09-17T07:29:55.863 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T07:29:55.864 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-17T07:29:55.879 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:55.879 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:55.895 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:55.895 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:55.909 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:55.910 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:55.910 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:55.910 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:55.910 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:55.914 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:55.914 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:55.914 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:55.913+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:55.920 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:55.932 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:55.932 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:55.934 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:55.934 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:56.015 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:56.014+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:56.062 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:56.062 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:56.071 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:56.118 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:56.116+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:56.178 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:56.178 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:56.187 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:56.189 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone39 from 0x7fffd5c (overlap 0x3846bc5) to 0x1682b1b 2024-09-17T07:29:56.218 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:56.217+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:56.285 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-17T07:29:56.307 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:56.307 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:56.321 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:56.323 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:56.322+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:56.392 INFO:teuthology.orchestra.run.smithi045.stdout:4892 write 0xb5e22dc thru 0xb5ea188 (0x7ead bytes) 2024-09-17T07:29:56.413 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:29:56.411+0000 7fef8efa6640 -1 osd.3 36 log_to_monitors true 2024-09-17T07:29:56.418 INFO:teuthology.orchestra.run.smithi045.stdout:4893 punch from 0xb273704 to 0xb27858b, (0x4e87 bytes) 2024-09-17T07:29:56.421 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:56.421+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:56.425 INFO:teuthology.orchestra.run.smithi045.stdout:4895 write 0xce47a59 thru 0xce5613c (0xe6e4 bytes) 2024-09-17T07:29:56.445 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:56.445 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:56.450 INFO:teuthology.orchestra.run.smithi045.stdout:4896 read 0x47ae2d thru 0x485441 (0xa615 bytes) 2024-09-17T07:29:56.451 INFO:teuthology.orchestra.run.smithi045.stdout:4900 read 0x9492b7d thru 0x9494b11 (0x1f95 bytes) 2024-09-17T07:29:56.454 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:56.460 INFO:teuthology.orchestra.run.smithi045.stdout:4901 trunc from 0xce5613d to 0x512c70c 2024-09-17T07:29:56.526 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:56.524+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:56.545 INFO:teuthology.orchestra.run.smithi045.stdout:4902 write 0xbd0fc94 thru 0xbd16c88 (0x6ff5 bytes) 2024-09-17T07:29:56.547 INFO:teuthology.orchestra.run.smithi045.stdout:4903 punch from 0xadf17e9 to 0xadf744f, (0x5c66 bytes) 2024-09-17T07:29:56.554 INFO:teuthology.orchestra.run.smithi045.stdout:4905 write 0xce63935 thru 0xce685e3 (0x4caf bytes) 2024-09-17T07:29:56.557 INFO:teuthology.orchestra.run.smithi045.stdout:4907 trunc from 0xce685e4 to 0xbb0dfcc 2024-09-17T07:29:56.569 INFO:teuthology.orchestra.run.smithi045.stdout:4908 write 0x3818138 thru 0x381b2dc (0x31a5 bytes) 2024-09-17T07:29:56.571 INFO:teuthology.orchestra.run.smithi045.stdout:4909 read 0x393c973 thru 0x393e1a2 (0x1830 bytes) 2024-09-17T07:29:56.571 INFO:teuthology.orchestra.run.smithi045.stdout:4911 write 0xba8824b thru 0xba8c93f (0x46f5 bytes) 2024-09-17T07:29:56.573 INFO:teuthology.orchestra.run.smithi045.stdout:4912 read 0xa4ed8ee thru 0xa4f5c0a (0x831d bytes) 2024-09-17T07:29:56.573 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:56.573 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:56.575 INFO:teuthology.orchestra.run.smithi045.stdout:4916 trunc from 0xbb0dfcc to 0x9fa20f4 2024-09-17T07:29:56.585 INFO:teuthology.orchestra.run.smithi045.stdout:4917 write 0x41fe0e3 thru 0x42084fa (0xa418 bytes) 2024-09-17T07:29:56.586 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:56.586 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:56.589 INFO:teuthology.orchestra.run.smithi045.stdout:4919 writesame 0x2e21a31 thru 0x2e296bc data_size 0x6(0x7c8c bytes) 2024-09-17T07:29:56.593 INFO:teuthology.orchestra.run.smithi045.stdout:4924 read 0x30182be thru 0x301c303 (0x4046 bytes) 2024-09-17T07:29:56.593 INFO:teuthology.orchestra.run.smithi045.stdout:4925 punch from 0x5e81326 to 0x5e91012, (0xfcec bytes) 2024-09-17T07:29:56.595 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:56.599 INFO:teuthology.orchestra.run.smithi045.stdout:4926 writesame 0xa51643a thru 0xa516c43 data_size 0x15(0x80a bytes) 2024-09-17T07:29:56.601 INFO:teuthology.orchestra.run.smithi045.stdout:4927 read 0xa418a5b thru 0xa420f89 (0x852f bytes) 2024-09-17T07:29:56.604 INFO:teuthology.orchestra.run.smithi045.stdout:4928 trunc from 0xa516c44 to 0xd53ce7d 2024-09-17T07:29:56.611 INFO:teuthology.orchestra.run.smithi045.stdout:4929 read 0x2ffbad8 thru 0x2fffa74 (0x3f9d bytes) 2024-09-17T07:29:56.611 INFO:teuthology.orchestra.run.smithi045.stdout:4931 write 0x3b1edac thru 0x3b23c58 (0x4ead bytes) 2024-09-17T07:29:56.612 INFO:teuthology.orchestra.run.smithi045.stdout:4933 writesame 0x9c6c65a thru 0x9c70b9d data_size 0x1f(0x4544 bytes) 2024-09-17T07:29:56.615 INFO:teuthology.orchestra.run.smithi045.stdout:4934 read 0x57363ae thru 0x573d2f6 (0x6f49 bytes) 2024-09-17T07:29:56.615 INFO:teuthology.orchestra.run.smithi045.stdout:4935 read 0xa384030 thru 0xa393a5b (0xfa2c bytes) 2024-09-17T07:29:56.616 INFO:teuthology.orchestra.run.smithi045.stdout:4936 write 0x1f57b1b thru 0x1f63e06 (0xc2ec bytes) 2024-09-17T07:29:56.618 INFO:teuthology.orchestra.run.smithi045.stdout:4937 write 0x5a38577 thru 0x5a41cb6 (0x9740 bytes) 2024-09-17T07:29:56.621 INFO:teuthology.orchestra.run.smithi045.stdout:4939 write 0x3c8a2cc thru 0x3c8e558 (0x428d bytes) 2024-09-17T07:29:56.622 INFO:teuthology.orchestra.run.smithi045.stdout:4940 read 0xce56853 thru 0xce5c3b0 (0x5b5e bytes) 2024-09-17T07:29:56.626 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:56.625+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:56.631 INFO:teuthology.orchestra.run.smithi045.stdout:4942 trunc from 0xd53ce7d to 0x5273e0d 2024-09-17T07:29:56.645 INFO:teuthology.orchestra.run.smithi045.stdout:4943 write 0x438003f thru 0x4385de0 (0x5da2 bytes) 2024-09-17T07:29:56.661 INFO:teuthology.orchestra.run.smithi045.stdout:4944 writesame 0xcf7f0de thru 0xcf8d501 data_size 0x7c(0xe424 bytes) 2024-09-17T07:29:56.662 INFO:teuthology.orchestra.run.smithi045.stdout:4947 write 0xb930189 thru 0xb9375f4 (0x746c bytes) 2024-09-17T07:29:56.665 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:56.665 INFO:teuthology.orchestra.run.smithi045.stdout:4950 write 0x374a6b3 thru 0x3757b35 (0xd483 bytes) 2024-09-17T07:29:56.667 INFO:teuthology.orchestra.run.smithi045.stdout:4951 read 0xf2cbb6 thru 0xf32cdd (0x6128 bytes) 2024-09-17T07:29:56.668 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:56.668 INFO:teuthology.orchestra.run.smithi045.stdout:4954 write 0x317ad1e thru 0x317c17c (0x145f bytes) 2024-09-17T07:29:56.670 INFO:teuthology.orchestra.run.smithi045.stdout:4955 write 0x50d6c80 thru 0x50dfae6 (0x8e67 bytes) 2024-09-17T07:29:56.673 INFO:teuthology.orchestra.run.smithi045.stdout:4956 read 0xc333b4d thru 0xc33ba5e (0x7f12 bytes) 2024-09-17T07:29:56.674 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:56.674 INFO:teuthology.orchestra.run.smithi045.stdout:4959 write 0xc362677 thru 0xc3691d9 (0x6b63 bytes) 2024-09-17T07:29:56.686 INFO:teuthology.orchestra.run.smithi045.stdout:4961 trunc from 0xcf8d502 to 0x2f1a42f 2024-09-17T07:29:56.709 INFO:teuthology.orchestra.run.smithi045.stdout:4963 write 0x9a7e0ba thru 0x9a8b552 (0xd499 bytes) 2024-09-17T07:29:56.711 INFO:teuthology.orchestra.run.smithi045.stdout:4964 read 0x6f5ab03 thru 0x6f60969 (0x5e67 bytes) 2024-09-17T07:29:56.718 INFO:teuthology.orchestra.run.smithi045.stdout:4966 writesame 0xb4ddf85 thru 0xb4e2549 data_size 0x1(0x45c5 bytes) 2024-09-17T07:29:56.719 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:56.720 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:56.720 INFO:teuthology.orchestra.run.smithi045.stdout:4968 read 0x465c0d6 thru 0x4667998 (0xb8c3 bytes) 2024-09-17T07:29:56.721 INFO:teuthology.orchestra.run.smithi045.stdout:4969 punch from 0xb4046d2 to 0xb412bfe, (0xe52c bytes) 2024-09-17T07:29:56.729 INFO:teuthology.orchestra.run.smithi045.stdout:4970 write 0xca7bc34 thru 0xca7f059 (0x3426 bytes) 2024-09-17T07:29:56.730 INFO:teuthology.orchestra.run.smithi045.stdout:4971 read 0x8e7c912 thru 0x8e7cf60 (0x64f bytes) 2024-09-17T07:29:56.730 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:29:56.731 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:56.729+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:56.736 INFO:teuthology.orchestra.run.smithi045.stdout:4972 writesame 0xde249b8 thru 0xde2dfd7 data_size 0x10(0x9620 bytes) 2024-09-17T07:29:56.738 INFO:teuthology.orchestra.run.smithi045.stdout:4974 trunc from 0xde2dfd8 to 0xd8e9c9d 2024-09-17T07:29:56.747 INFO:teuthology.orchestra.run.smithi045.stdout:4975 read 0x67ada89 thru 0x67ae463 (0x9db bytes) 2024-09-17T07:29:56.747 INFO:teuthology.orchestra.run.smithi045.stdout:4976 write 0xd878d35 thru 0xd88452d (0xb7f9 bytes) 2024-09-17T07:29:56.749 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:56.749 INFO:teuthology.orchestra.run.smithi045.stdout:4980 writesame 0x2dd8a12 thru 0x2de0bd6 data_size 0x1(0x81c5 bytes) 2024-09-17T07:29:56.765 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:56.765 INFO:teuthology.orchestra.run.smithi045.stdout:4982 read 0x1ecf832 thru 0x1ed35e3 (0x3db2 bytes) 2024-09-17T07:29:56.766 INFO:teuthology.orchestra.run.smithi045.stdout:4983 write 0x6f26449 thru 0x6f310aa (0xac62 bytes) 2024-09-17T07:29:56.767 INFO:teuthology.orchestra.run.smithi045.stdout:4984 read 0x37e29f0 thru 0x37ea869 (0x7e7a bytes) 2024-09-17T07:29:56.768 INFO:teuthology.orchestra.run.smithi045.stdout:4986 punch from 0xd43f276 to 0xd44755b, (0x82e5 bytes) 2024-09-17T07:29:56.768 INFO:teuthology.orchestra.run.smithi045.stdout:4987 write 0x28aa598 thru 0x28ae863 (0x42cc bytes) 2024-09-17T07:29:56.769 INFO:teuthology.orchestra.run.smithi045.stdout:4988 write 0xa95ee30 thru 0xa967964 (0x8b35 bytes) 2024-09-17T07:29:56.770 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:56.771 INFO:teuthology.orchestra.run.smithi045.stdout:4991 read 0xd7457b2 thru 0xd75494a (0xf199 bytes) 2024-09-17T07:29:56.771 INFO:teuthology.orchestra.run.smithi045.stdout:4994 write 0x3cabc79 thru 0x3caca49 (0xdd1 bytes) 2024-09-17T07:29:56.772 INFO:teuthology.orchestra.run.smithi045.stdout:4995 writesame 0xc49e7d8 thru 0xc49ed3d data_size 0x1(0x566 bytes) 2024-09-17T07:29:56.774 INFO:teuthology.orchestra.run.smithi045.stdout:4996 writesame 0x12fe79c thru 0x13034d9 data_size 0x1(0x4d3e bytes) 2024-09-17T07:29:56.781 INFO:teuthology.orchestra.run.smithi045.stdout:4997 write 0xbc1c8d5 thru 0xbc2371e (0x6e4a bytes) 2024-09-17T07:29:56.783 INFO:teuthology.orchestra.run.smithi045.stdout:4998 read 0x3a7a09d thru 0x3a87d1b (0xdc7f bytes) 2024-09-17T07:29:56.784 INFO:teuthology.orchestra.run.smithi045.stdout:4999 write 0xeb465c thru 0xec443e (0xfde3 bytes) 2024-09-17T07:29:56.785 INFO:teuthology.orchestra.run.smithi045.stdout:5000 punch from 0x74bb81 to 0x75b727, (0xfba6 bytes) 2024-09-17T07:29:56.798 INFO:teuthology.orchestra.run.smithi045.stdout:5002 trunc from 0xd8e9c9d to 0x382f2ed 2024-09-17T07:29:56.817 INFO:teuthology.orchestra.run.smithi045.stdout:5003 trunc from 0x382f2ed to 0xb1ac078 2024-09-17T07:29:56.830 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:56.830+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:56.831 INFO:teuthology.orchestra.run.smithi045.stdout:5004 write 0xc55c8eb thru 0xc568975 (0xc08b bytes) 2024-09-17T07:29:56.832 INFO:teuthology.orchestra.run.smithi045.stdout:5006 read 0xbe0fa16 thru 0xbe143a4 (0x498f bytes) 2024-09-17T07:29:56.832 INFO:teuthology.orchestra.run.smithi045.stdout:5007 write 0x987929d thru 0x9882b62 (0x98c6 bytes) 2024-09-17T07:29:56.834 INFO:teuthology.orchestra.run.smithi045.stdout:5010 write 0x6c59177 thru 0x6c60a31 (0x78bb bytes) 2024-09-17T07:29:56.836 INFO:teuthology.orchestra.run.smithi045.stdout:5011 writesame 0x69ed4ac thru 0x69ee5da data_size 0x1(0x112f bytes) 2024-09-17T07:29:56.838 INFO:teuthology.orchestra.run.smithi045.stdout:5012 write 0x35186ad thru 0x352789c (0xf1f0 bytes) 2024-09-17T07:29:56.840 INFO:teuthology.orchestra.run.smithi045.stdout:5013 punch from 0x9ec9e6c to 0x9ed6753, (0xc8e7 bytes) 2024-09-17T07:29:56.840 INFO:teuthology.orchestra.run.smithi045.stdout:5014 write 0x2e437b7 thru 0x2e52dfb (0xf645 bytes) 2024-09-17T07:29:56.847 INFO:teuthology.orchestra.run.smithi045.stdout:5015 write 0xc60d38c thru 0xc60fa38 (0x26ad bytes) 2024-09-17T07:29:56.849 INFO:teuthology.orchestra.run.smithi045.stdout:5016 punch from 0x82ba2b2 to 0x82bb620, (0x136e bytes) 2024-09-17T07:29:56.849 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:56.849 INFO:teuthology.orchestra.run.smithi045.stdout:5018 write 0x99f3eeb thru 0x99fc592 (0x86a8 bytes) 2024-09-17T07:29:56.850 INFO:teuthology.orchestra.run.smithi045.stdout:5019 writesame 0x52e66b2 thru 0x52f48c8 data_size 0x6d(0xe217 bytes) 2024-09-17T07:29:56.852 INFO:teuthology.orchestra.run.smithi045.stdout:5020 writesame 0xee6ac9 thru 0xef5673 data_size 0x1(0xebab bytes) 2024-09-17T07:29:56.863 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:56.863 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:56.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:29:56.881 INFO:teuthology.orchestra.run.smithi045.stdout:5021 punch from 0x31fe29a to 0x3209060, (0xadc6 bytes) 2024-09-17T07:29:56.881 INFO:teuthology.orchestra.run.smithi045.stdout:5022 read 0xb8d369c thru 0xb8d9226 (0x5b8b bytes) 2024-09-17T07:29:56.889 INFO:teuthology.orchestra.run.smithi045.stdout:5023 write 0xd59550c thru 0xd59d93c (0x8431 bytes) 2024-09-17T07:29:56.891 INFO:teuthology.orchestra.run.smithi045.stdout:5024 writesame 0x5e6ebbc thru 0x5e7820e data_size 0x1(0x9653 bytes) 2024-09-17T07:29:56.909 INFO:teuthology.orchestra.run.smithi045.stdout:5025 write 0xc91a16f thru 0xc91a57d (0x40f bytes) 2024-09-17T07:29:56.910 INFO:teuthology.orchestra.run.smithi045.stdout:5026 write 0x352bc0a thru 0x35332ad (0x76a4 bytes) 2024-09-17T07:29:56.912 INFO:teuthology.orchestra.run.smithi045.stdout:5028 read 0x624a1eb thru 0x624e2c1 (0x40d7 bytes) 2024-09-17T07:29:56.913 INFO:teuthology.orchestra.run.smithi045.stdout:5030 write 0x5774a82 thru 0x577ac7b (0x61fa bytes) 2024-09-17T07:29:56.917 INFO:teuthology.orchestra.run.smithi045.stdout:5031 trunc from 0xd59d93d to 0xb1e45ad 2024-09-17T07:29:56.925 INFO:teuthology.orchestra.run.smithi045.stdout:5032 read 0x54a8d62 thru 0x54af382 (0x6621 bytes) 2024-09-17T07:29:56.926 INFO:teuthology.orchestra.run.smithi045.stdout:5033 writesame 0x4b13908 thru 0x4b198a8 data_size 0x1(0x5fa1 bytes) 2024-09-17T07:29:56.931 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:56.931+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:56.938 INFO:teuthology.orchestra.run.smithi045.stdout:5037 read 0x9b92c07 thru 0x9ba0114 (0xd50e bytes) 2024-09-17T07:29:56.938 INFO:teuthology.orchestra.run.smithi045.stdout:5038 punch from 0x6f29c72 to 0x6f321f1, (0x857f bytes) 2024-09-17T07:29:56.944 INFO:teuthology.orchestra.run.smithi045.stdout:5039 trunc from 0xb1e45ad to 0x64c5c19 2024-09-17T07:29:56.962 INFO:teuthology.orchestra.run.smithi045.stdout:5040 trunc from 0x64c5c19 to 0xb445610 2024-09-17T07:29:56.969 INFO:teuthology.orchestra.run.smithi045.stdout:5041 writesame 0x6206637 thru 0x6206a51 data_size 0x1(0x41b bytes) 2024-09-17T07:29:56.971 INFO:teuthology.orchestra.run.smithi045.stdout:5043 read 0x2efdffa thru 0x2f0dc61 (0xfc68 bytes) 2024-09-17T07:29:56.980 INFO:teuthology.orchestra.run.smithi045.stdout:5044 trunc from 0xb445610 to 0x37c77dc 2024-09-17T07:29:56.991 INFO:teuthology.orchestra.run.smithi045.stdout:5046 write 0x1deb190 thru 0x1df8353 (0xd1c4 bytes) 2024-09-17T07:29:56.993 INFO:teuthology.orchestra.run.smithi045.stdout:5048 punch from 0x11118fc to 0x1117757, (0x5e5b bytes) 2024-09-17T07:29:56.999 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:56.999 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:57.009 INFO:teuthology.orchestra.run.smithi045.stdout:5052 write 0xc47a5c5 thru 0xc47e5ee (0x402a bytes) 2024-09-17T07:29:57.011 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:29:57.011 INFO:teuthology.orchestra.run.smithi045.stdout:5053 read 0x1118c3d thru 0x112554f (0xc913 bytes) 2024-09-17T07:29:57.011 INFO:teuthology.orchestra.run.smithi045.stdout:5054 writesame 0xc2fa7c0 thru 0xc2fdfe9 data_size 0x4f(0x382a bytes) 2024-09-17T07:29:57.013 INFO:teuthology.orchestra.run.smithi045.stdout:5055 writesame 0x486364f thru 0x4865c16 data_size 0x3e(0x25c8 bytes) 2024-09-17T07:29:57.014 INFO:teuthology.orchestra.run.smithi045.stdout:5057 write 0xc09df86 thru 0xc0a6576 (0x85f1 bytes) 2024-09-17T07:29:57.015 INFO:teuthology.orchestra.run.smithi045.stdout:5058 read 0x24189aa thru 0x241c262 (0x38b9 bytes) 2024-09-17T07:29:57.016 INFO:teuthology.orchestra.run.smithi045.stdout:5059 write 0x31b1bae thru 0x31b91ed (0x7640 bytes) 2024-09-17T07:29:57.029 INFO:teuthology.orchestra.run.smithi045.stdout:5060 trunc from 0xc47e5ef to 0x28b81b3 2024-09-17T07:29:57.032 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:57.032+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:57.040 INFO:teuthology.orchestra.run.smithi045.stdout:5061 read 0x1229a9b thru 0x12381c3 (0xe729 bytes) 2024-09-17T07:29:57.041 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.051 INFO:teuthology.orchestra.run.smithi045.stdout:5064 writesame 0x66b0f36 thru 0x66b68e5 data_size 0x70(0x59b0 bytes) 2024-09-17T07:29:57.053 INFO:teuthology.orchestra.run.smithi045.stdout:5065 write 0xb4434e thru 0xb53544 (0xf1f7 bytes) 2024-09-17T07:29:57.056 INFO:teuthology.orchestra.run.smithi045.stdout:5066 read 0x2d7118d thru 0x2d7b467 (0xa2db bytes) 2024-09-17T07:29:57.061 INFO:teuthology.orchestra.run.smithi045.stdout:5067 trunc from 0x66b68e6 to 0xa998c4d 2024-09-17T07:29:57.067 INFO:teuthology.orchestra.run.smithi045.stdout:5068 read 0x9d50514 thru 0x9d58460 (0x7f4d bytes) 2024-09-17T07:29:57.067 INFO:teuthology.orchestra.run.smithi045.stdout:5069 write 0x2191b0e thru 0x21a147f (0xf972 bytes) 2024-09-17T07:29:57.069 INFO:teuthology.orchestra.run.smithi045.stdout:5073 writesame 0x6e51fce thru 0x6e616d0 data_size 0x1(0xf703 bytes) 2024-09-17T07:29:57.100 INFO:teuthology.orchestra.run.smithi045.stdout:5074 write 0xa1d7a65 thru 0xa1e1d7f (0xa31b bytes) 2024-09-17T07:29:57.101 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.102 INFO:teuthology.orchestra.run.smithi045.stdout:5076 writesame 0x5cd1831 thru 0x5cd4056 data_size 0x9(0x2826 bytes) 2024-09-17T07:29:57.103 INFO:teuthology.orchestra.run.smithi045.stdout:5077 read 0x578db66 thru 0x5797a65 (0x9f00 bytes) 2024-09-17T07:29:57.104 INFO:teuthology.orchestra.run.smithi045.stdout:5078 punch from 0xa6f65a2 to 0xa6fd666, (0x70c4 bytes) 2024-09-17T07:29:57.104 INFO:teuthology.orchestra.run.smithi045.stdout:5079 write 0x2da4703 thru 0x2dae9de (0xa2dc bytes) 2024-09-17T07:29:57.105 INFO:teuthology.orchestra.run.smithi045.stdout:5082 read 0x5c9349b thru 0x5ca011f (0xcc85 bytes) 2024-09-17T07:29:57.105 INFO:teuthology.orchestra.run.smithi045.stdout:5083 read 0x9ca87a0 thru 0x9cb21fe (0x9a5f bytes) 2024-09-17T07:29:57.106 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.106 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.109 INFO:teuthology.orchestra.run.smithi045.stdout:5086 trunc from 0xa998c4d to 0xd558731 2024-09-17T07:29:57.115 INFO:teuthology.orchestra.run.smithi045.stdout:5087 write 0xc9bb572 thru 0xc9c7c85 (0xc714 bytes) 2024-09-17T07:29:57.116 INFO:teuthology.orchestra.run.smithi045.stdout:5088 read 0x360f296 thru 0x36191b1 (0x9f1c bytes) 2024-09-17T07:29:57.117 INFO:teuthology.orchestra.run.smithi045.stdout:5089 write 0x9fcb0d5 thru 0x9fce292 (0x31be bytes) 2024-09-17T07:29:57.118 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.118 INFO:teuthology.orchestra.run.smithi045.stdout:5091 write 0x14ac17b thru 0x14b983a (0xd6c0 bytes) 2024-09-17T07:29:57.120 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.121 INFO:teuthology.orchestra.run.smithi045.stdout:5094 writesame 0x1d6e3cc thru 0x1d7c226 data_size 0x1(0xde5b bytes) 2024-09-17T07:29:57.130 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T07:29:57.133 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:57.132+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:57.137 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:57.137 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:57.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:29:57.155 INFO:teuthology.orchestra.run.smithi045.stdout:5095 writesame 0x6e56536 thru 0x6e5b228 data_size 0x1(0x4cf3 bytes) 2024-09-17T07:29:57.164 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:29:57.162+0000 7fef82d54640 -1 osd.3 89 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T07:29:57.172 INFO:teuthology.orchestra.run.smithi045.stdout:5096 trunc from 0xd558731 to 0x9548851 2024-09-17T07:29:57.186 INFO:teuthology.orchestra.run.smithi045.stdout:5098 read 0x3e442eb thru 0x3e4be97 (0x7bad bytes) 2024-09-17T07:29:57.187 INFO:teuthology.orchestra.run.smithi045.stdout:5100 read 0x58ea4d7 thru 0x58f38fd (0x9427 bytes) 2024-09-17T07:29:57.189 INFO:teuthology.orchestra.run.smithi045.stdout:5101 trunc from 0x9548851 to 0xb28dfed 2024-09-17T07:29:57.195 INFO:teuthology.orchestra.run.smithi045.stdout:5103 write 0x7189f4f thru 0x7192a15 (0x8ac7 bytes) 2024-09-17T07:29:57.197 INFO:teuthology.orchestra.run.smithi045.stdout:5104 punch from 0x7f6026b to 0x7f6af79, (0xad0e bytes) 2024-09-17T07:29:57.197 INFO:teuthology.orchestra.run.smithi045.stdout:5105 read 0x69f7671 thru 0x6a01cfc (0xa68c bytes) 2024-09-17T07:29:57.201 INFO:teuthology.orchestra.run.smithi045.stdout:5106 trunc from 0xb28dfed to 0x7eb91d1 2024-09-17T07:29:57.214 INFO:teuthology.orchestra.run.smithi045.stdout:5108 read 0x727ddb7 thru 0x7288628 (0xa872 bytes) 2024-09-17T07:29:57.224 INFO:teuthology.orchestra.run.smithi045.stdout:5112 trunc from 0x7eb91d1 to 0x1016d6 2024-09-17T07:29:57.234 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:57.233+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:57.246 INFO:teuthology.orchestra.run.smithi045.stdout:5113 read 0x1ba71 thru 0x28a85 (0xd015 bytes) 2024-09-17T07:29:57.247 INFO:teuthology.orchestra.run.smithi045.stdout:5114 punch from 0x37793 to 0x3be8f, (0x46fc bytes) 2024-09-17T07:29:57.265 INFO:teuthology.orchestra.run.smithi045.stdout:5115 write 0xa781129 thru 0xa783b3b (0x2a13 bytes) 2024-09-17T07:29:57.269 INFO:teuthology.orchestra.run.smithi045.stdout:5116 writesame 0x30130f5 thru 0x301e895 data_size 0x1(0xb7a1 bytes) 2024-09-17T07:29:57.282 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:57.283 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:57.290 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:29:57.294 INFO:teuthology.orchestra.run.smithi045.stdout:5118 trunc from 0xa783b3c to 0xe4b11b2 2024-09-17T07:29:57.305 INFO:teuthology.orchestra.run.smithi045.stdout:5120 trunc from 0xe4b11b2 to 0xa2aacf0 2024-09-17T07:29:57.316 INFO:teuthology.orchestra.run.smithi045.stdout:5121 read 0x5827dbc thru 0x583001c (0x8261 bytes) 2024-09-17T07:29:57.316 INFO:teuthology.orchestra.run.smithi045.stdout:5122 write 0x7beb10c thru 0x7bfaaf9 (0xf9ee bytes) 2024-09-17T07:29:57.320 INFO:teuthology.orchestra.run.smithi045.stdout:5123 read 0x8d1b2e5 thru 0x8d25218 (0x9f34 bytes) 2024-09-17T07:29:57.320 INFO:teuthology.orchestra.run.smithi045.stdout:5125 punch from 0x1869e18 to 0x18710e0, (0x72c8 bytes) 2024-09-17T07:29:57.320 INFO:teuthology.orchestra.run.smithi045.stdout:5127 write 0x3172ff thru 0x3195a6 (0x22a8 bytes) 2024-09-17T07:29:57.330 INFO:teuthology.orchestra.run.smithi045.stdout:5128 writesame 0xbd8a646 thru 0xbd917c1 data_size 0x36(0x717c bytes) 2024-09-17T07:29:57.331 INFO:teuthology.orchestra.run.smithi045.stdout:5130 writesame 0xb0b16d1 thru 0xb0bd7bf data_size 0x1(0xc0ef bytes) 2024-09-17T07:29:57.334 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:57.333+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:57.363 INFO:teuthology.orchestra.run.smithi045.stdout:5131 writesame 0xee5d75a thru 0xee6a5dd data_size 0x2(0xce84 bytes) 2024-09-17T07:29:57.364 INFO:teuthology.orchestra.run.smithi045.stdout:5132 write 0xb0078b0 thru 0xb009339 (0x1a8a bytes) 2024-09-17T07:29:57.380 INFO:teuthology.orchestra.run.smithi045.stdout:5133 trunc from 0xee6a5de to 0xec7a0e 2024-09-17T07:29:57.391 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.391 INFO:teuthology.orchestra.run.smithi045.stdout:5136 read 0x701121 thru 0x706eee (0x5dce bytes) 2024-09-17T07:29:57.392 INFO:teuthology.orchestra.run.smithi045.stdout:5137 trunc from 0xec7a0e to 0x106c7dc 2024-09-17T07:29:57.397 INFO:teuthology.orchestra.run.smithi045.stdout:5139 punch from 0x868da7 to 0x8700ca, (0x7323 bytes) 2024-09-17T07:29:57.397 INFO:teuthology.orchestra.run.smithi045.stdout:5140 read 0x55ba4f thru 0x55e157 (0x2709 bytes) 2024-09-17T07:29:57.417 INFO:teuthology.orchestra.run.smithi045.stdout:5141 write 0xdbdda6b thru 0xdbe29d8 (0x4f6e bytes) 2024-09-17T07:29:57.419 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:57.419 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:57.421 INFO:teuthology.orchestra.run.smithi045.stdout:5142 writesame 0x2cbba2a thru 0x2cbf0e0 data_size 0x45(0x36b7 bytes) 2024-09-17T07:29:57.422 INFO:teuthology.orchestra.run.smithi045.stdout:5143 punch from 0x9492c2c to 0x94942b2, (0x1686 bytes) 2024-09-17T07:29:57.429 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:29:57.429 INFO:teuthology.orchestra.run.smithi045.stdout:5145 write 0xde6de81 thru 0xde7a86a (0xc9ea bytes) 2024-09-17T07:29:57.432 INFO:teuthology.orchestra.run.smithi045.stdout:5146 read 0xb6a6a39 thru 0xb6ac028 (0x55f0 bytes) 2024-09-17T07:29:57.432 INFO:teuthology.orchestra.run.smithi045.stdout:5148 punch from 0x85213db to 0x8524532, (0x3157 bytes) 2024-09-17T07:29:57.432 INFO:teuthology.orchestra.run.smithi045.stdout:5151 trunc from 0xde7a86b to 0xe179398 2024-09-17T07:29:57.435 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:57.435+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:57.438 INFO:teuthology.orchestra.run.smithi045.stdout:5154 write 0x1af8fd6 thru 0x1aff926 (0x6951 bytes) 2024-09-17T07:29:57.439 INFO:teuthology.orchestra.run.smithi045.stdout:5155 write 0xc8490ca thru 0xc84fd48 (0x6c7f bytes) 2024-09-17T07:29:57.441 INFO:teuthology.orchestra.run.smithi045.stdout:5156 read 0x25287ce thru 0x2534f84 (0xc7b7 bytes) 2024-09-17T07:29:57.441 INFO:teuthology.orchestra.run.smithi045.stdout:5158 read 0x1742cff thru 0x17494f0 (0x67f2 bytes) 2024-09-17T07:29:57.449 INFO:teuthology.orchestra.run.smithi045.stdout:5159 trunc from 0xe179398 to 0x7c21245 2024-09-17T07:29:57.470 INFO:teuthology.orchestra.run.smithi045.stdout:5161 write 0xd532a16 thru 0xd53d4e1 (0xaacc bytes) 2024-09-17T07:29:57.471 INFO:teuthology.orchestra.run.smithi045.stdout:5162 write 0xa53a054 thru 0xa5473b1 (0xd35e bytes) 2024-09-17T07:29:57.473 INFO:teuthology.orchestra.run.smithi045.stdout:5164 write 0xc29246e thru 0xc2a1e0d (0xf9a0 bytes) 2024-09-17T07:29:57.476 INFO:teuthology.orchestra.run.smithi045.stdout:5166 writesame 0x261551b thru 0x2620ab5 data_size 0x1(0xb59b bytes) 2024-09-17T07:29:57.492 INFO:teuthology.orchestra.run.smithi045.stdout:5167 write 0x1a07096 thru 0x1a11851 (0xa7bc bytes) 2024-09-17T07:29:57.493 INFO:teuthology.orchestra.run.smithi045.stdout:5168 punch from 0x3daaa17 to 0x3daaef2, (0x4db bytes) 2024-09-17T07:29:57.493 INFO:teuthology.orchestra.run.smithi045.stdout:5170 read 0x9947ac5 thru 0x994d744 (0x5c80 bytes) 2024-09-17T07:29:57.494 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.494 INFO:teuthology.orchestra.run.smithi045.stdout:5173 read 0x637e4e9 thru 0x638bc09 (0xd721 bytes) 2024-09-17T07:29:57.501 INFO:teuthology.orchestra.run.smithi045.stdout:5174 write 0xe58cc67 thru 0xe598529 (0xb8c3 bytes) 2024-09-17T07:29:57.504 INFO:teuthology.orchestra.run.smithi045.stdout:5175 read 0x713efa1 thru 0x714394c (0x49ac bytes) 2024-09-17T07:29:57.505 INFO:teuthology.orchestra.run.smithi045.stdout:5176 trunc from 0xe59852a to 0xd972a2b 2024-09-17T07:29:57.518 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.518 INFO:teuthology.orchestra.run.smithi045.stdout:5178 writesame 0x8ed4b78 thru 0x8edb9cf data_size 0x42(0x6e58 bytes) 2024-09-17T07:29:57.522 INFO:teuthology.orchestra.run.smithi045.stdout:5181 trunc from 0xd972a2b to 0xb06d9fc 2024-09-17T07:29:57.531 INFO:teuthology.orchestra.run.smithi045.stdout:5182 write 0x1f77f8a thru 0x1f877e0 (0xf857 bytes) 2024-09-17T07:29:57.532 INFO:teuthology.orchestra.run.smithi045.stdout:5183 read 0x953f174 thru 0x9544495 (0x5322 bytes) 2024-09-17T07:29:57.536 INFO:teuthology.orchestra.run.smithi045.stdout:5185 trunc from 0xb06d9fc to 0x7bc3a36 2024-09-17T07:29:57.537 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:57.535+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:57.548 INFO:teuthology.orchestra.run.smithi045.stdout:5186 trunc from 0x7bc3a36 to 0x580cc56 2024-09-17T07:29:57.554 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:57.554 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:57.565 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:29:57.572 INFO:teuthology.orchestra.run.smithi045.stdout:5188 writesame 0xd26defb thru 0xd26ff09 data_size 0x1(0x200f bytes) 2024-09-17T07:29:57.574 INFO:teuthology.orchestra.run.smithi045.stdout:5190 writesame 0xb01bf69 thru 0xb020f66 data_size 0x3(0x4ffe bytes) 2024-09-17T07:29:57.577 INFO:teuthology.orchestra.run.smithi045.stdout:5191 writesame 0xce2af59 thru 0xce3339e data_size 0x1(0x8446 bytes) 2024-09-17T07:29:57.594 INFO:teuthology.orchestra.run.smithi045.stdout:5192 write 0x7a5897 thru 0x7ae533 (0x8c9d bytes) 2024-09-17T07:29:57.596 INFO:teuthology.orchestra.run.smithi045.stdout:5193 punch from 0x3c61fe3 to 0x3c694e3, (0x7500 bytes) 2024-09-17T07:29:57.596 INFO:teuthology.orchestra.run.smithi045.stdout:5194 trunc from 0xd26ff0a to 0xd2a11d7 2024-09-17T07:29:57.601 INFO:teuthology.orchestra.run.smithi045.stdout:5195 write 0x5b09655 thru 0x5b17db5 (0xe761 bytes) 2024-09-17T07:29:57.602 INFO:teuthology.orchestra.run.smithi045.stdout:5197 write 0x7aa52c7 thru 0x7aaaf43 (0x5c7d bytes) 2024-09-17T07:29:57.604 INFO:teuthology.orchestra.run.smithi045.stdout:5198 write 0x6829db9 thru 0x682e5f5 (0x483d bytes) 2024-09-17T07:29:57.616 INFO:teuthology.orchestra.run.smithi045.stdout:5200 trunc from 0xd2a11d7 to 0x3e89ca4 2024-09-17T07:29:57.633 INFO:teuthology.orchestra.run.smithi045.stdout:5201 write 0x5b8c5d9 thru 0x5b9abfb (0xe623 bytes) 2024-09-17T07:29:57.635 INFO:teuthology.orchestra.run.smithi045.stdout:5202 punch from 0xfe3487 to 0xfe98ef, (0x6468 bytes) 2024-09-17T07:29:57.637 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:57.636+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:57.639 INFO:teuthology.orchestra.run.smithi045.stdout:5206 trunc from 0x5b9abfc to 0x26be550 2024-09-17T07:29:57.651 INFO:teuthology.orchestra.run.smithi045.stdout:5207 writesame 0x36a75d thru 0x377530 data_size 0x6(0xcdd4 bytes) 2024-09-17T07:29:57.666 INFO:teuthology.orchestra.run.smithi045.stdout:5208 trunc from 0x26be550 to 0xbd60d5e 2024-09-17T07:29:57.677 INFO:teuthology.orchestra.run.smithi045.stdout:5209 write 0xbe655fa thru 0xbe69be7 (0x45ee bytes) 2024-09-17T07:29:57.679 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.679 INFO:teuthology.orchestra.run.smithi045.stdout:5213 punch from 0x17f345b to 0x17f915c, (0x5d01 bytes) 2024-09-17T07:29:57.681 INFO:teuthology.orchestra.run.smithi045.stdout:5214 trunc from 0xbe69be8 to 0xdc5421e 2024-09-17T07:29:57.687 INFO:teuthology.orchestra.run.smithi045.stdout:5215 writesame 0x69541df thru 0x695ec95 data_size 0x1d(0xaab7 bytes) 2024-09-17T07:29:57.689 INFO:teuthology.orchestra.run.smithi045.stdout:5216 punch from 0x6b1e93a to 0x6b211c5, (0x288b bytes) 2024-09-17T07:29:57.689 INFO:teuthology.orchestra.run.smithi045.stdout:5217 read 0xb341c54 thru 0xb34345d (0x180a bytes) 2024-09-17T07:29:57.703 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:57.703 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:57.705 INFO:teuthology.orchestra.run.smithi045.stdout:5218 trunc from 0xdc5421e to 0x85d9f0 2024-09-17T07:29:57.715 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:29:57.715 INFO:teuthology.orchestra.run.smithi045.stdout:5220 punch from 0x21fa2b to 0x22c3ec, (0xc9c1 bytes) 2024-09-17T07:29:57.716 INFO:teuthology.orchestra.run.smithi045.stdout:5221 punch from 0x44c9ee to 0x44fa47, (0x3059 bytes) 2024-09-17T07:29:57.727 INFO:teuthology.orchestra.run.smithi045.stdout:5222 writesame 0x60c1dd6 thru 0x60ce1d5 data_size 0x80(0xc400 bytes) 2024-09-17T07:29:57.738 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:57.736+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:57.741 INFO:teuthology.orchestra.run.smithi045.stdout:5223 write 0xc50d370 thru 0xc51b4c3 (0xe154 bytes) 2024-09-17T07:29:57.742 INFO:teuthology.orchestra.run.smithi045.stdout:5224 write 0x689a42d thru 0x689ee8b (0x4a5f bytes) 2024-09-17T07:29:57.744 INFO:teuthology.orchestra.run.smithi045.stdout:5225 write 0x91042a0 thru 0x910a9be (0x671f bytes) 2024-09-17T07:29:57.746 INFO:teuthology.orchestra.run.smithi045.stdout:5226 read 0xb5607f6 thru 0xb56864f (0x7e5a bytes) 2024-09-17T07:29:57.752 INFO:teuthology.orchestra.run.smithi045.stdout:5227 write 0xcb09344 thru 0xcb0f03e (0x5cfb bytes) 2024-09-17T07:29:57.757 INFO:teuthology.orchestra.run.smithi045.stdout:5228 punch from 0xa742c54 to 0xa750297, (0xd643 bytes) 2024-09-17T07:29:57.757 INFO:teuthology.orchestra.run.smithi045.stdout:5229 read 0x725d4cc thru 0x7263cc0 (0x67f5 bytes) 2024-09-17T07:29:57.757 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.757 INFO:teuthology.orchestra.run.smithi045.stdout:5234 writesame 0x1721308 thru 0x172a308 data_size 0x49(0x9001 bytes) 2024-09-17T07:29:57.757 INFO:teuthology.orchestra.run.smithi045.stdout:5235 read 0x4cea63d thru 0x4cf36f5 (0x90b9 bytes) 2024-09-17T07:29:57.763 INFO:teuthology.orchestra.run.smithi045.stdout:5237 writesame 0xdc8c567 thru 0xdc9aee7 data_size 0x17(0xe981 bytes) 2024-09-17T07:29:57.765 INFO:teuthology.orchestra.run.smithi045.stdout:5238 read 0x2f7ecf4 thru 0x2f8c1d0 (0xd4dd bytes) 2024-09-17T07:29:57.765 INFO:teuthology.orchestra.run.smithi045.stdout:5240 read 0x6556a6e thru 0x65651ec (0xe77f bytes) 2024-09-17T07:29:57.765 INFO:teuthology.orchestra.run.smithi045.stdout:5241 read 0x474c9f8 thru 0x4752529 (0x5b32 bytes) 2024-09-17T07:29:57.766 INFO:teuthology.orchestra.run.smithi045.stdout:5242 write 0x6c830df thru 0x6c861dc (0x30fe bytes) 2024-09-17T07:29:57.767 INFO:teuthology.orchestra.run.smithi045.stdout:5243 write 0xc0537ec thru 0xc05c875 (0x908a bytes) 2024-09-17T07:29:57.770 INFO:teuthology.orchestra.run.smithi045.stdout:5244 writesame 0x9a9c355 thru 0x9aab005 data_size 0x3b(0xecb1 bytes) 2024-09-17T07:29:57.773 INFO:teuthology.orchestra.run.smithi045.stdout:5248 trunc from 0xdc9aee8 to 0xdc526ef 2024-09-17T07:29:57.779 INFO:teuthology.orchestra.run.smithi045.stdout:5249 writesame 0x36a8df8 thru 0x36ae6fd data_size 0x56(0x5906 bytes) 2024-09-17T07:29:57.781 INFO:teuthology.orchestra.run.smithi045.stdout:5250 writesame 0x958d682 thru 0x958e7d1 data_size 0x4(0x1150 bytes) 2024-09-17T07:29:57.798 INFO:teuthology.orchestra.run.smithi045.stdout:5252 trunc from 0xdc526ef to 0xdeb9ab 2024-09-17T07:29:57.805 INFO:teuthology.orchestra.run.smithi045.stdout:5253 read 0xda5865 thru 0xdb2af8 (0xd294 bytes) 2024-09-17T07:29:57.806 INFO:teuthology.orchestra.run.smithi045.stdout:5254 read 0x6b47d3 thru 0x6bf198 (0xa9c6 bytes) 2024-09-17T07:29:57.806 INFO:teuthology.orchestra.run.smithi045.stdout:5255 punch from 0x9303cf to 0x93aa28, (0xa659 bytes) 2024-09-17T07:29:57.806 INFO:teuthology.orchestra.run.smithi045.stdout:5256 read 0x2b444 thru 0x34cdb (0x9898 bytes) 2024-09-17T07:29:57.807 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.821 INFO:teuthology.orchestra.run.smithi045.stdout:5258 write 0x869b180 thru 0x869d398 (0x2219 bytes) 2024-09-17T07:29:57.823 INFO:teuthology.orchestra.run.smithi045.stdout:5260 write 0x6a82a2b thru 0x6a83b3c (0x1112 bytes) 2024-09-17T07:29:57.825 INFO:teuthology.orchestra.run.smithi045.stdout:5261 read 0x211ac5e thru 0x2120894 (0x5c37 bytes) 2024-09-17T07:29:57.825 INFO:teuthology.orchestra.run.smithi045.stdout:5263 punch from 0x5ead30b to 0x5eaf9d1, (0x26c6 bytes) 2024-09-17T07:29:57.827 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:57.827 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:57.836 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:29:57.839 INFO:teuthology.orchestra.run.smithi045.stdout:5264 writesame 0xe385118 thru 0xe393fc9 data_size 0x1(0xeeb2 bytes) 2024-09-17T07:29:57.839 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.839 INFO:teuthology.orchestra.run.smithi045.stdout:5266 write 0x660d823 thru 0x6617c0d (0xa3eb bytes) 2024-09-17T07:29:57.839 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:57.837+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:57.840 INFO:teuthology.orchestra.run.smithi045.stdout:5267 read 0x46f4a3c thru 0x4701977 (0xcf3c bytes) 2024-09-17T07:29:57.840 INFO:teuthology.orchestra.run.smithi045.stdout:5268 write 0xba0ec70 thru 0xba121e1 (0x3572 bytes) 2024-09-17T07:29:57.842 INFO:teuthology.orchestra.run.smithi045.stdout:5269 read 0x43243b0 thru 0x432d037 (0x8c88 bytes) 2024-09-17T07:29:57.843 INFO:teuthology.orchestra.run.smithi045.stdout:5270 write 0xa1b90cd thru 0xa1bac9c (0x1bd0 bytes) 2024-09-17T07:29:57.844 INFO:teuthology.orchestra.run.smithi045.stdout:5271 read 0x399a827 thru 0x39a18ca (0x70a4 bytes) 2024-09-17T07:29:57.845 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.845 INFO:teuthology.orchestra.run.smithi045.stdout:5274 read 0xcaba5c thru 0xcba69b (0xec40 bytes) 2024-09-17T07:29:57.845 INFO:teuthology.orchestra.run.smithi045.stdout:5275 read 0x8af020a thru 0x8afae54 (0xac4b bytes) 2024-09-17T07:29:57.846 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.861 INFO:teuthology.orchestra.run.smithi045.stdout:5278 trunc from 0xe393fca to 0x9f9e7a 2024-09-17T07:29:57.871 INFO:teuthology.orchestra.run.smithi045.stdout:5279 punch from 0x7d85f3 to 0x7de38d, (0x5d9a bytes) 2024-09-17T07:29:57.890 INFO:teuthology.orchestra.run.smithi045.stdout:5281 write 0xbbe69db thru 0xbbed1c7 (0x67ed bytes) 2024-09-17T07:29:57.892 INFO:teuthology.orchestra.run.smithi045.stdout:5282 read 0x599a19b thru 0x599e2bc (0x4122 bytes) 2024-09-17T07:29:57.893 INFO:teuthology.orchestra.run.smithi045.stdout:5283 writesame 0x41a7190 thru 0x41afb9a data_size 0x1(0x8a0b bytes) 2024-09-17T07:29:57.909 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.909 INFO:teuthology.orchestra.run.smithi045.stdout:5287 write 0x2b0a457 thru 0x2b11c16 (0x77c0 bytes) 2024-09-17T07:29:57.913 INFO:teuthology.orchestra.run.smithi045.stdout:5288 trunc from 0xbbed1c8 to 0xd8fc8e7 2024-09-17T07:29:57.918 INFO:teuthology.orchestra.run.smithi045.stdout:5289 punch from 0x40e98ca to 0x40f3743, (0x9e79 bytes) 2024-09-17T07:29:57.919 INFO:teuthology.orchestra.run.smithi045.stdout:5290 trunc from 0xd8fc8e7 to 0xe76b636 2024-09-17T07:29:57.925 INFO:teuthology.orchestra.run.smithi045.stdout:5292 punch from 0xda5687c to 0xda6106c, (0xa7f0 bytes) 2024-09-17T07:29:57.925 INFO:teuthology.orchestra.run.smithi045.stdout:5293 write 0x67f7a98 thru 0x67f7d06 (0x26f bytes) 2024-09-17T07:29:57.927 INFO:teuthology.orchestra.run.smithi045.stdout:5295 read 0x8816c50 thru 0x8818842 (0x1bf3 bytes) 2024-09-17T07:29:57.927 INFO:teuthology.orchestra.run.smithi045.stdout:5296 read 0x6a8cc30 thru 0x6a99f19 (0xd2ea bytes) 2024-09-17T07:29:57.928 INFO:teuthology.orchestra.run.smithi045.stdout:5298 read 0xe634bf2 thru 0xe644752 (0xfb61 bytes) 2024-09-17T07:29:57.933 INFO:teuthology.orchestra.run.smithi045.stdout:5299 write 0xea8a155 thru 0xea91d32 (0x7bde bytes) 2024-09-17T07:29:57.935 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:29:57.938 INFO:teuthology.orchestra.run.smithi045.stdout:5300 punch from 0x8cf6735 to 0x8cfdd63, (0x762e bytes) 2024-09-17T07:29:57.938 INFO:teuthology.orchestra.run.smithi045.stdout:5301 punch from 0x9346a66 to 0x934cd4f, (0x62e9 bytes) 2024-09-17T07:29:57.938 INFO:teuthology.orchestra.run.smithi045.stdout:5302 write 0x828c80c thru 0x8293ec8 (0x76bd bytes) 2024-09-17T07:29:57.939 INFO:teuthology.orchestra.run.smithi045.stdout:5303 write 0x996d5c6 thru 0x99709f4 (0x342f bytes) 2024-09-17T07:29:57.939 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:57.939+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:57.940 INFO:teuthology.orchestra.run.smithi045.stdout:5304 writesame 0x431210f thru 0x431552b data_size 0x1(0x341d bytes) 2024-09-17T07:29:57.943 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:57.943 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:57.946 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.946 INFO:teuthology.orchestra.run.smithi045.stdout:5306 read 0x561a885 thru 0x561c5b8 (0x1d34 bytes) 2024-09-17T07:29:57.947 INFO:teuthology.orchestra.run.smithi045.stdout:5307 punch from 0x2cac383 to 0x2cb26d7, (0x6354 bytes) 2024-09-17T07:29:57.947 INFO:teuthology.orchestra.run.smithi045.stdout:5308 read 0x1c3e75f thru 0x1c4e6f6 (0xff98 bytes) 2024-09-17T07:29:57.947 INFO:teuthology.orchestra.run.smithi045.stdout:5309 write 0x5714ae0 thru 0x5722ba9 (0xe0ca bytes) 2024-09-17T07:29:57.950 INFO:teuthology.orchestra.run.smithi045.stdout:5310 write 0xb6e1b57 thru 0xb6ea3ce (0x8878 bytes) 2024-09-17T07:29:57.951 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:57.951 INFO:teuthology.orchestra.run.smithi045.stdout:5312 write 0x96777ab thru 0x9684b50 (0xd3a6 bytes) 2024-09-17T07:29:57.953 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:29:57.954 INFO:teuthology.orchestra.run.smithi045.stdout:5313 read 0x7ea4d77 thru 0x7ea5833 (0xabd bytes) 2024-09-17T07:29:57.954 INFO:teuthology.orchestra.run.smithi045.stdout:5314 write 0xa351c23 thru 0xa359fad (0x838b bytes) 2024-09-17T07:29:57.955 INFO:teuthology.orchestra.run.smithi045.stdout:5315 trunc from 0xea91d33 to 0xe865ad4 2024-09-17T07:29:57.963 INFO:teuthology.orchestra.run.smithi045.stdout:5317 punch from 0xc771a72 to 0xc77981e, (0x7dac bytes) 2024-09-17T07:29:57.963 INFO:teuthology.orchestra.run.smithi045.stdout:5319 write 0x96eff79 thru 0x96f49a1 (0x4a29 bytes) 2024-09-17T07:29:57.964 INFO:teuthology.orchestra.run.smithi045.stdout:5320 write 0xe6290fe thru 0xe62ac7e (0x1b81 bytes) 2024-09-17T07:29:57.967 INFO:teuthology.orchestra.run.smithi045.stdout:5321 clone 41 order 24 su 1048576 sc 5 2024-09-17T07:29:58.040 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:58.040+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:58.060 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:29:58.060 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:29:58.077 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:29:58.077 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:29:58.093 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:29:58.094 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:29:58.094 INFO:teuthology.orchestra.run.smithi045.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T07:29:58.098 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:29:58.098 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:29:58.106 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:29:58.110 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:29:58.111 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:29:58.128 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-17T07:29:58.133 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:29:58.133 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:29:58.141 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:58.140+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:58.157 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:29:58.158 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:29:58.181 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:29:58.181 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:29:58.184 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T07:29:58.185 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T07:29:58.205 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone40 from 0xe865ad4 (overlap 0x1016d6) to 0xb432f 2024-09-17T07:29:58.241 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:58.240+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:58.261 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:58.261 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:58.261 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:58.261 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:58.270 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:29:58.305 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-17T07:29:58.342 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:58.342+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:58.351 INFO:teuthology.orchestra.run.smithi045.stdout:5322 write 0xb5f0919 thru 0xb5fcb1f (0xc207 bytes) 2024-09-17T07:29:58.360 INFO:teuthology.orchestra.run.smithi045.stdout:5325 flatten 2024-09-17T07:29:58.392 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:58.392 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:58.392 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:58.392 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:58.392 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:58.393 INFO:teuthology.orchestra.run.smithi045.stdout:5327 trunc from 0xe865ad4 to 0xcdec8be 2024-09-17T07:29:58.402 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:29:58.404 INFO:teuthology.orchestra.run.smithi045.stdout:5328 punch from 0x674ad74 to 0x674df22, (0x31ae bytes) 2024-09-17T07:29:58.408 INFO:teuthology.orchestra.run.smithi045.stdout:5330 trunc from 0xcdec8be to 0x9540b94 2024-09-17T07:29:58.421 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:58.421 INFO:teuthology.orchestra.run.smithi045.stdout:5332 write 0x757c2a2 thru 0x75800db (0x3e3a bytes) 2024-09-17T07:29:58.425 INFO:teuthology.orchestra.run.smithi045.stdout:5333 trunc from 0x9540b94 to 0xb84d4b5 2024-09-17T07:29:58.437 INFO:teuthology.orchestra.run.smithi045.stdout:5334 writesame 0xc6fd999 thru 0xc709e83 data_size 0x1(0xc4eb bytes) 2024-09-17T07:29:58.438 INFO:teuthology.orchestra.run.smithi045.stdout:5336 read 0x6abac3d thru 0x6ac7791 (0xcb55 bytes) 2024-09-17T07:29:58.438 INFO:teuthology.orchestra.run.smithi045.stdout:5337 write 0x63709a2 thru 0x6376102 (0x5761 bytes) 2024-09-17T07:29:58.440 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:58.441 INFO:teuthology.orchestra.run.smithi045.stdout:5343 punch from 0x1d6f772 to 0x1d7c83b, (0xd0c9 bytes) 2024-09-17T07:29:58.441 INFO:teuthology.orchestra.run.smithi045.stdout:5344 write 0x61e9342 thru 0x61f6949 (0xd608 bytes) 2024-09-17T07:29:58.442 INFO:teuthology.orchestra.run.smithi045.stdout:5345 write 0x4172148 thru 0x41750e4 (0x2f9d bytes) 2024-09-17T07:29:58.443 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:58.442+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:58.443 INFO:teuthology.orchestra.run.smithi045.stdout:5346 read 0xe4a380 thru 0xe55653 (0xb2d4 bytes) 2024-09-17T07:29:58.443 INFO:teuthology.orchestra.run.smithi045.stdout:5348 punch from 0x1f90469 to 0x1f92f70, (0x2b07 bytes) 2024-09-17T07:29:58.451 INFO:teuthology.orchestra.run.smithi045.stdout:5349 write 0xe797d42 thru 0xe7a627f (0xe53e bytes) 2024-09-17T07:29:58.455 INFO:teuthology.orchestra.run.smithi045.stdout:5350 writesame 0x98fc3b5 thru 0x990977d data_size 0x1(0xd3c9 bytes) 2024-09-17T07:29:58.479 INFO:teuthology.orchestra.run.smithi045.stdout:5352 write 0x4fb6a96 thru 0x4fbcafe (0x6069 bytes) 2024-09-17T07:29:58.483 INFO:teuthology.orchestra.run.smithi045.stdout:5353 writesame 0xc711c83 thru 0xc712f54 data_size 0x3(0x12d2 bytes) 2024-09-17T07:29:58.485 INFO:teuthology.orchestra.run.smithi045.stdout:5354 writesame 0x5140961 thru 0x5142fce data_size 0x1(0x266e bytes) 2024-09-17T07:29:58.505 INFO:teuthology.orchestra.run.smithi045.stdout:5355 trunc from 0xe7a6280 to 0x2487fd2 2024-09-17T07:29:58.525 INFO:teuthology.orchestra.run.smithi045.stdout:5356 punch from 0x1766c34 to 0x17740df, (0xd4ab bytes) 2024-09-17T07:29:58.540 INFO:teuthology.orchestra.run.smithi045.stdout:5359 write 0x9959b75 thru 0x995d777 (0x3c03 bytes) 2024-09-17T07:29:58.543 INFO:teuthology.orchestra.run.smithi045.stdout:5360 punch from 0x6306a5c to 0x6311612, (0xabb6 bytes) 2024-09-17T07:29:58.543 INFO:teuthology.orchestra.run.smithi045.stdout:5361 read 0x23a0937 thru 0x23a4150 (0x381a bytes) 2024-09-17T07:29:58.543 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:58.543+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:58.553 INFO:teuthology.orchestra.run.smithi045.stdout:5362 trunc from 0x995d778 to 0x1b28ad1 2024-09-17T07:29:58.553 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:58.553 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:58.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:58.553 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:58.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.553 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19985 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5275648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.554 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:58.554 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.299152468999999, 2024-09-17T07:29:58.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071819998999999995, 2024-09-17T07:29:58.554 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.554 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.554 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.554 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19985 2024-09-17T07:29:58.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.554 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:58.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:58.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:29:58.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:58.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252493+0000", 2024-09-17T07:29:58.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6300000000000004e-07 2024-09-17T07:29:58.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252496+0000", 2024-09-17T07:29:58.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-17T07:29:58.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252499+0000", 2024-09-17T07:29:58.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:29:58.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324036+0000", 2024-09-17T07:29:58.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071536587999999998 2024-09-17T07:29:58.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324118+0000", 2024-09-17T07:29:58.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2336999999999995e-05 2024-09-17T07:29:58.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324312+0000", 2024-09-17T07:29:58.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000193709 2024-09-17T07:29:58.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.558 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.558 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19986 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5505024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.558 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:58.558 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.299141317, 2024-09-17T07:29:58.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072083761999999996, 2024-09-17T07:29:58.558 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.558 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.558 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.559 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19986 2024-09-17T07:29:58.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.559 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:58.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:58.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:29:58.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:29:58.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252508+0000", 2024-09-17T07:29:58.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:29:58.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252511+0000", 2024-09-17T07:29:58.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.145e-06 2024-09-17T07:29:58.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252514+0000", 2024-09-17T07:29:58.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5569999999999998e-06 2024-09-17T07:29:58.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324326+0000", 2024-09-17T07:29:58.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071812681000000003 2024-09-17T07:29:58.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324372+0000", 2024-09-17T07:29:58.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5893000000000003e-05 2024-09-17T07:29:58.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324587+0000", 2024-09-17T07:29:58.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214608 2024-09-17T07:29:58.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.563 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.563 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19987 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5734400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.563 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:58.563 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.299127699000003, 2024-09-17T07:29:58.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072276676999999998, 2024-09-17T07:29:58.563 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.563 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.563 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.564 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.564 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19987 2024-09-17T07:29:58.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.564 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:58.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:58.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:29:58.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:58.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252518+0000", 2024-09-17T07:29:58.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-17T07:29:58.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252522+0000", 2024-09-17T07:29:58.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2440000000000004e-06 2024-09-17T07:29:58.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252525+0000", 2024-09-17T07:29:58.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5919999999999999e-06 2024-09-17T07:29:58.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324621+0000", 2024-09-17T07:29:58.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072096442999999996 2024-09-17T07:29:58.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324699+0000", 2024-09-17T07:29:58.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8061999999999999e-05 2024-09-17T07:29:58.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324793+0000", 2024-09-17T07:29:58.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4207000000000001e-05 2024-09-17T07:29:58.567 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.567 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.567 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.567 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19988 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.568 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:58.568 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.299116728000001, 2024-09-17T07:29:58.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072585798000000007, 2024-09-17T07:29:58.568 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.568 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.568 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.568 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19988 2024-09-17T07:29:58.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.568 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:58.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:58.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:29:58.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:58.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252529+0000", 2024-09-17T07:29:58.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:29:58.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252532+0000", 2024-09-17T07:29:58.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9579999999999999e-06 2024-09-17T07:29:58.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252534+0000", 2024-09-17T07:29:58.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5730000000000002e-06 2024-09-17T07:29:58.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324825+0000", 2024-09-17T07:29:58.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072290757999999997 2024-09-17T07:29:58.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324874+0000", 2024-09-17T07:29:58.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9360000000000002e-05 2024-09-17T07:29:58.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325113+0000", 2024-09-17T07:29:58.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000239025 2024-09-17T07:29:58.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.572 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.572 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19989 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.572 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:58.572 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.299107106999998, 2024-09-17T07:29:58.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072963494000000004, 2024-09-17T07:29:58.573 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.573 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.573 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.573 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19989 2024-09-17T07:29:58.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.573 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:58.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:29:58.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:29:58.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:29:58.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:29:58.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2900000000000002e-07 2024-09-17T07:29:58.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252542+0000", 2024-09-17T07:29:58.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8600000000000003e-06 2024-09-17T07:29:58.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252545+0000", 2024-09-17T07:29:58.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.605e-06 2024-09-17T07:29:58.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325144+0000", 2024-09-17T07:29:58.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072599460000000005 2024-09-17T07:29:58.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325223+0000", 2024-09-17T07:29:58.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8473999999999994e-05 2024-09-17T07:29:58.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325501+0000", 2024-09-17T07:29:58.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000278063 2024-09-17T07:29:58.577 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.577 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.577 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.577 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19990 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6422528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.577 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:58.577 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.299096636000002, 2024-09-17T07:29:58.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073241127000000003, 2024-09-17T07:29:58.577 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.577 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.577 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.578 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.578 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.578 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.578 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19990 2024-09-17T07:29:58.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.578 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:58.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:58.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:29:58.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:58.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252552+0000", 2024-09-17T07:29:58.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4719999999999997e-06 2024-09-17T07:29:58.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252557+0000", 2024-09-17T07:29:58.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3039999999999998e-06 2024-09-17T07:29:58.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252560+0000", 2024-09-17T07:29:58.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8830000000000002e-06 2024-09-17T07:29:58.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325529+0000", 2024-09-17T07:29:58.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072968880999999999 2024-09-17T07:29:58.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325574+0000", 2024-09-17T07:29:58.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466000000000001e-05 2024-09-17T07:29:58.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325789+0000", 2024-09-17T07:29:58.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021490999999999999 2024-09-17T07:29:58.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.581 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.582 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19991 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6651904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.582 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:58.582 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.299077334000003, 2024-09-17T07:29:58.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073720552999999994, 2024-09-17T07:29:58.582 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.582 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.582 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.582 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19991 2024-09-17T07:29:58.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.583 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:58.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:58.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:29:58.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4699999999999998e-07 2024-09-17T07:29:58.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252568+0000", 2024-09-17T07:29:58.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5300000000000002e-07 2024-09-17T07:29:58.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252572+0000", 2024-09-17T07:29:58.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9160000000000003e-06 2024-09-17T07:29:58.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252575+0000", 2024-09-17T07:29:58.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:29:58.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325820+0000", 2024-09-17T07:29:58.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073245597999999995 2024-09-17T07:29:58.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325901+0000", 2024-09-17T07:29:58.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0913000000000003e-05 2024-09-17T07:29:58.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326288+0000", 2024-09-17T07:29:58.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038646900000000002 2024-09-17T07:29:58.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.586 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.586 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19992 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6881280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.586 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:58.586 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.299066981000003, 2024-09-17T07:29:58.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074054683999999996, 2024-09-17T07:29:58.587 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.587 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.587 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.587 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19992 2024-09-17T07:29:58.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.587 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:58.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:29:58.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252578+0000", 2024-09-17T07:29:58.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:29:58.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252579+0000", 2024-09-17T07:29:58.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2500000000000004e-07 2024-09-17T07:29:58.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252582+0000", 2024-09-17T07:29:58.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-17T07:29:58.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252585+0000", 2024-09-17T07:29:58.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.542e-06 2024-09-17T07:29:58.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326315+0000", 2024-09-17T07:29:58.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073729921000000004 2024-09-17T07:29:58.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326359+0000", 2024-09-17T07:29:58.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4095999999999999e-05 2024-09-17T07:29:58.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326632+0000", 2024-09-17T07:29:58.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027314499999999997 2024-09-17T07:29:58.590 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.591 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.591 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.591 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19993 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7110656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.591 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:58.591 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.299056677000003, 2024-09-17T07:29:58.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074425142999999999, 2024-09-17T07:29:58.591 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.591 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.591 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.592 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.592 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19993 2024-09-17T07:29:58.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.592 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:58.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:58.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:29:58.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:29:58.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252589+0000", 2024-09-17T07:29:58.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.33e-07 2024-09-17T07:29:58.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252593+0000", 2024-09-17T07:29:58.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:29:58.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252595+0000", 2024-09-17T07:29:58.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.621e-06 2024-09-17T07:29:58.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326666+0000", 2024-09-17T07:29:58.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074070787999999999 2024-09-17T07:29:58.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326752+0000", 2024-09-17T07:29:58.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5748000000000006e-05 2024-09-17T07:29:58.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327013+0000", 2024-09-17T07:29:58.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026098400000000002 2024-09-17T07:29:58.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.595 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.595 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19994 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7340032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.596 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:58.596 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.299046218000001, 2024-09-17T07:29:58.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074765920999999999, 2024-09-17T07:29:58.596 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.596 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.596 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.596 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.596 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19994 2024-09-17T07:29:58.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.596 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:58.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:58.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:29:58.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:29:58.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252599+0000", 2024-09-17T07:29:58.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2099999999999995e-07 2024-09-17T07:29:58.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252603+0000", 2024-09-17T07:29:58.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:29:58.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252606+0000", 2024-09-17T07:29:58.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-09-17T07:29:58.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327047+0000", 2024-09-17T07:29:58.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074441050999999994 2024-09-17T07:29:58.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327121+0000", 2024-09-17T07:29:58.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4283999999999996e-05 2024-09-17T07:29:58.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327364+0000", 2024-09-17T07:29:58.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000243155 2024-09-17T07:29:58.601 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.601 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.601 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.601 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19995 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7569408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.601 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:58.601 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.299035521, 2024-09-17T07:29:58.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075172694999999998, 2024-09-17T07:29:58.601 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.601 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.601 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.601 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.602 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.602 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19995 2024-09-17T07:29:58.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.602 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:58.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:58.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:29:58.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:58.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252610+0000", 2024-09-17T07:29:58.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6799999999999999e-07 2024-09-17T07:29:58.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252614+0000", 2024-09-17T07:29:58.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000002e-06 2024-09-17T07:29:58.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252616+0000", 2024-09-17T07:29:58.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.419e-06 2024-09-17T07:29:58.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327407+0000", 2024-09-17T07:29:58.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074790617000000004 2024-09-17T07:29:58.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327498+0000", 2024-09-17T07:29:58.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0758000000000001e-05 2024-09-17T07:29:58.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327782+0000", 2024-09-17T07:29:58.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028395599999999998 2024-09-17T07:29:58.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.605 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.605 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.606 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7798784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.606 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:58.606 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.299025205, 2024-09-17T07:29:58.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075605832999999997, 2024-09-17T07:29:58.606 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.606 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.606 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.606 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:29:58.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.606 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:58.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:58.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:29:58.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:29:58.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:29:58.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:29:58.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:29:58.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:29:58.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:29:58.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:29:58.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327844+0000", 2024-09-17T07:29:58.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075217709999999993 2024-09-17T07:29:58.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327927+0000", 2024-09-17T07:29:58.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2628999999999998e-05 2024-09-17T07:29:58.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328225+0000", 2024-09-17T07:29:58.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029800600000000002 2024-09-17T07:29:58.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.610 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.610 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8028160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.610 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:58.610 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.299014851999999, 2024-09-17T07:29:58.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076061237000000004, 2024-09-17T07:29:58.610 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.611 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.611 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.611 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:29:58.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.611 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:58.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:58.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:29:58.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:58.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:29:58.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:29:58.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:29:58.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:29:58.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:29:58.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:29:58.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328268+0000", 2024-09-17T07:29:58.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075626323999999995 2024-09-17T07:29:58.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328352+0000", 2024-09-17T07:29:58.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3578999999999994e-05 2024-09-17T07:29:58.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328691+0000", 2024-09-17T07:29:58.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033915099999999999 2024-09-17T07:29:58.614 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.615 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.615 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.615 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8257536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.615 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:58.615 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.298999401000003, 2024-09-17T07:29:58.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076441903000000005, 2024-09-17T07:29:58.615 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.615 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.615 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.615 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.615 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.616 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.616 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:29:58.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.616 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:58.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:58.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:29:58.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:29:58.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:29:58.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:29:58.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:29:58.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:29:58.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:29:58.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:29:58.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328747+0000", 2024-09-17T07:29:58.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076094192000000005 2024-09-17T07:29:58.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328824+0000", 2024-09-17T07:29:58.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7096999999999997e-05 2024-09-17T07:29:58.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.329087+0000", 2024-09-17T07:29:58.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026249999999999998 2024-09-17T07:29:58.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.619 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.620 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20029 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.620 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:58.620 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.288914413000001, 2024-09-17T07:29:58.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071783696999999994, 2024-09-17T07:29:58.620 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.620 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.620 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.620 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20029 2024-09-17T07:29:58.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.620 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:58.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:29:58.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262735+0000", 2024-09-17T07:29:58.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.955e-06 2024-09-17T07:29:58.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262737+0000", 2024-09-17T07:29:58.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0090000000000002e-06 2024-09-17T07:29:58.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262743+0000", 2024-09-17T07:29:58.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2890000000000003e-06 2024-09-17T07:29:58.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262748+0000", 2024-09-17T07:29:58.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:29:58.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334163+0000", 2024-09-17T07:29:58.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071415336999999995 2024-09-17T07:29:58.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334243+0000", 2024-09-17T07:29:58.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9548000000000005e-05 2024-09-17T07:29:58.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334514+0000", 2024-09-17T07:29:58.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027086800000000001 2024-09-17T07:29:58.624 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.624 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.624 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.624 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7356416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.624 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:58.624 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.288883241999997, 2024-09-17T07:29:58.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072172616999999994, 2024-09-17T07:29:58.624 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.624 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.624 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.625 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:29:58.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.625 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:58.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:29:58.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:29:58.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:29:58.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:29:58.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:29:58.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:29:58.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:29:58.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:29:58.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:29:58.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334558+0000", 2024-09-17T07:29:58.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071780790999999997 2024-09-17T07:29:58.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334647+0000", 2024-09-17T07:29:58.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9448000000000001e-05 2024-09-17T07:29:58.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334934+0000", 2024-09-17T07:29:58.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028654599999999999 2024-09-17T07:29:58.628 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.628 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.628 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.629 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7585792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.629 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:58.629 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.288861529999998, 2024-09-17T07:29:58.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072606135000000002, 2024-09-17T07:29:58.629 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.629 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.629 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.629 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.629 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:29:58.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:29:58.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:29:58.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:29:58.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:29:58.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:29:58.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:29:58.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:29:58.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:29:58.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335116+0000", 2024-09-17T07:29:58.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072323897999999998 2024-09-17T07:29:58.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335166+0000", 2024-09-17T07:29:58.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9882000000000001e-05 2024-09-17T07:29:58.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335389+0000", 2024-09-17T07:29:58.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022334300000000001 2024-09-17T07:29:58.633 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.633 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.633 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.633 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7815168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.633 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:58.633 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.288849331000002, 2024-09-17T07:29:58.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073051230999999994, 2024-09-17T07:29:58.633 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.634 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.634 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.634 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.634 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.634 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.634 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:29:58.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.634 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:58.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:29:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:29:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:29:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:29:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:29:58.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:29:58.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:29:58.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:29:58.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335431+0000", 2024-09-17T07:29:58.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072627852000000007 2024-09-17T07:29:58.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335525+0000", 2024-09-17T07:29:58.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3708000000000005e-05 2024-09-17T07:29:58.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335846+0000", 2024-09-17T07:29:58.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032182000000000001 2024-09-17T07:29:58.637 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.637 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.637 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.638 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.638 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:58.638 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.288838472999998, 2024-09-17T07:29:58.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073284199999999994, 2024-09-17T07:29:58.638 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.638 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.638 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.638 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.638 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.638 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.638 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:29:58.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.639 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:58.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:58.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:29:58.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:29:58.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:29:58.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:29:58.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:29:58.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:29:58.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:29:58.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:29:58.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335863+0000", 2024-09-17T07:29:58.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073048921000000003 2024-09-17T07:29:58.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335904+0000", 2024-09-17T07:29:58.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1545000000000003e-05 2024-09-17T07:29:58.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336090+0000", 2024-09-17T07:29:58.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185787 2024-09-17T07:29:58.642 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.642 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.642 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.642 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8273920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:29:58.642 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:58.642 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.288824212999998, 2024-09-17T07:29:58.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073670478999999997, 2024-09-17T07:29:58.643 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:58.643 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:58.643 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:58.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:58.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:58.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:58.643 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:29:58.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.643 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:58.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:58.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:58.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:58.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:58.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:58.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:58.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:29:58.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:29:58.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:58.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:29:58.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:29:58.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:58.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:29:58.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:29:58.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:58.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:29:58.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:29:58.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:58.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336126+0000", 2024-09-17T07:29:58.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073297254000000006 2024-09-17T07:29:58.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:58.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336211+0000", 2024-09-17T07:29:58.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4728000000000006e-05 2024-09-17T07:29:58.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:58.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:58.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:58.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336491+0000", 2024-09-17T07:29:58.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027979399999999998 2024-09-17T07:29:58.646 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.646 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:58.647 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:58.647 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:58.649 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:29:58.650 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:58.650 INFO:teuthology.orchestra.run.smithi045.stdout:5364 read 0xb2a1dd thru 0xb33265 (0x9089 bytes) 2024-09-17T07:29:58.650 INFO:teuthology.orchestra.run.smithi045.stdout:5365 write 0x2ee8d51 thru 0x2ee923e (0x4ee bytes) 2024-09-17T07:29:58.650 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:58.651 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:58.651 INFO:teuthology.orchestra.run.smithi045.stdout:5370 punch from 0x2d7b2fc to 0x2d88788, (0xd48c bytes) 2024-09-17T07:29:58.651 INFO:teuthology.orchestra.run.smithi045.stdout:5371 write 0x4e3ef37 thru 0x4e4d15d (0xe227 bytes) 2024-09-17T07:29:58.651 INFO:teuthology.orchestra.run.smithi045.stdout:5372 write 0x80d5ada thru 0x80e3573 (0xda9a bytes) 2024-09-17T07:29:58.651 INFO:teuthology.orchestra.run.smithi045.stdout:5373 read 0x4d408a8 thru 0x4d4dda1 (0xd4fa bytes) 2024-09-17T07:29:58.651 INFO:teuthology.orchestra.run.smithi045.stdout:5374 write 0x45d94a2 thru 0x45dec6c (0x57cb bytes) 2024-09-17T07:29:58.651 INFO:teuthology.orchestra.run.smithi045.stdout:5375 writesame 0x43dba31 thru 0x43e0f7d data_size 0x1d(0x554d bytes) 2024-09-17T07:29:58.651 INFO:teuthology.orchestra.run.smithi045.stdout:5376 write 0xdebf996 thru 0xdec0568 (0xbd3 bytes) 2024-09-17T07:29:58.651 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:58.651 INFO:teuthology.orchestra.run.smithi045.stdout:5380 read 0xba8b1bb thru 0xba921f9 (0x703f bytes) 2024-09-17T07:29:58.651 INFO:teuthology.orchestra.run.smithi045.stdout:5381 write 0x8ba796f thru 0x8baeedb (0x756d bytes) 2024-09-17T07:29:58.651 INFO:teuthology.orchestra.run.smithi045.stdout:5382 read 0x1e5d4e6 thru 0x1e63f86 (0x6aa1 bytes) 2024-09-17T07:29:58.651 INFO:teuthology.orchestra.run.smithi045.stdout:5385 write 0x80d2e28 thru 0x80d3374 (0x54d bytes) 2024-09-17T07:29:58.651 INFO:teuthology.orchestra.run.smithi045.stdout:5387 read 0x6c3fcfd thru 0x6c473e7 (0x76eb bytes) 2024-09-17T07:29:58.652 INFO:teuthology.orchestra.run.smithi045.stdout:5389 write 0x83884ee thru 0x838bc27 (0x373a bytes) 2024-09-17T07:29:58.652 INFO:teuthology.orchestra.run.smithi045.stdout:5390 writesame 0x2738897 thru 0x274064f data_size 0x29(0x7db9 bytes) 2024-09-17T07:29:58.652 INFO:teuthology.orchestra.run.smithi045.stdout:5391 clone 42 order 25 su 1048576 sc 12 2024-09-17T07:29:58.652 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:58.647+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:58.749 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:58.749+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:58.793 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:58.793 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:58.793 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:58.793 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:58.801 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:29:58.885 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:58.885+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:58.946 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:58.946 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:58.946 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:58.951 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:29:58.951+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:29:58.954 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:29:59.052 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:59.052+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:59.100 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:59.100 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:59.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:59.100 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:59.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.101 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:59.101 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:59.101 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.193297516999998, 2024-09-17T07:29:59.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:29:59.101 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.101 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.101 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.102 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:29:59.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.102 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:59.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:59.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:29:59.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:29:59.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:29:59.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:29:59.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:29:59.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:29:59.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:29:59.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:29:59.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:29:59.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:29:59.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:29:59.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:29:59.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:29:59.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:29:59.106 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.106 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.106 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.106 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:59.106 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:59.106 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.193278839999998, 2024-09-17T07:29:59.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:29:59.106 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.106 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.106 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.107 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:29:59.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.107 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:59.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:29:59.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:29:59.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:29:59.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:29:59.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:29:59.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:29:59.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:29:59.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:29:59.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:29:59.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:29:59.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:29:59.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:29:59.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:29:59.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:29:59.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:29:59.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.110 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.111 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:59.111 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:59.111 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.193260487000003, 2024-09-17T07:29:59.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:29:59.111 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.111 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.111 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.111 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:29:59.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.112 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:59.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:59.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:29:59.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:29:59.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:29:59.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:29:59.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:29:59.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:29:59.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:29:59.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:29:59.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:29:59.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:29:59.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:29:59.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:29:59.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:29:59.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:29:59.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.115 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.115 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:59.115 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:59.115 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.193240889000002, 2024-09-17T07:29:59.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:29:59.116 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.116 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.116 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.116 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:29:59.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.116 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:59.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:29:59.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:29:59.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:29:59.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:29:59.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:29:59.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:29:59.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:29:59.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:29:59.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:29:59.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:29:59.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:29:59.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:29:59.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:29:59.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:29:59.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:29:59.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.120 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.121 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:59.121 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:59.121 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.193220934000003, 2024-09-17T07:29:59.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:29:59.121 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.121 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.121 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.121 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:29:59.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.121 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:59.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:59.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:29:59.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:29:59.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:29:59.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:29:59.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:29:59.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:29:59.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:29:59.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:29:59.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:29:59.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:29:59.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:29:59.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:29:59.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:29:59.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:29:59.125 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.125 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.125 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.125 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:59.125 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:59.125 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.193202278999998, 2024-09-17T07:29:59.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:29:59.126 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.126 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.126 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.126 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.126 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.126 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.126 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:29:59.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.126 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:59.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:59.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:29:59.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:29:59.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:29:59.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:29:59.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:29:59.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:29:59.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:29:59.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:29:59.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:29:59.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:29:59.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:29:59.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:29:59.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:29:59.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:29:59.129 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.129 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.130 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.130 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:59.130 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:59.130 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.193183802999997, 2024-09-17T07:29:59.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:29:59.130 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.130 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.130 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.130 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.130 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.131 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.131 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:29:59.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.131 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:59.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:29:59.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:29:59.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:29:59.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:29:59.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:29:59.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:29:59.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:29:59.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:29:59.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:29:59.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:29:59.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:29:59.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:29:59.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:29:59.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:29:59.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:29:59.134 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.134 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.134 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.135 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:59.135 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:59.135 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.193164203999999, 2024-09-17T07:29:59.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:29:59.135 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.135 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.135 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.135 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.135 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:29:59.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.135 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:59.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:59.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:29:59.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:29:59.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:29:59.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:29:59.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:29:59.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:29:59.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:29:59.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:29:59.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:29:59.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:29:59.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:29:59.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:29:59.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:29:59.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:29:59.139 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.139 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.139 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.139 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:59.139 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:59.139 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.193145629, 2024-09-17T07:29:59.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:29:59.139 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.140 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.140 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.140 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:29:59.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.140 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:59.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:29:59.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:29:59.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:59.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:29:59.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:29:59.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:29:59.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:29:59.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:29:59.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:29:59.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:29:59.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:29:59.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:29:59.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:29:59.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:29:59.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:29:59.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.144 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.144 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:59.144 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:59.144 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.193123202000002, 2024-09-17T07:29:59.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:29:59.144 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.144 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.144 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.145 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:29:59.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.145 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:59.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:59.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:29:59.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:29:59.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:29:59.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:29:59.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:29:59.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:29:59.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:29:59.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:29:59.148 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.148 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.148 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.148 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:59.148 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:59.148 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.193095384999999, 2024-09-17T07:29:59.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:29:59.148 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.148 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.149 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.149 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.149 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.149 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.149 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:29:59.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.149 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:59.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:59.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:29:59.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:29:59.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:29:59.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:29:59.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:29:59.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:29:59.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:29:59.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:29:59.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:29:59.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:29:59.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:29:59.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:29:59.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:29:59.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:29:59.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.153 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.153 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:59.153 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:59.153 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.193073441000003, 2024-09-17T07:29:59.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:29:59.153 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.153 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.154 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.154 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:29:59.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.154 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:59.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:59.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:29:59.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:29:59.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:29:59.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:29:59.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:29:59.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:29:59.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:29:59.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:29:59.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:29:59.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:29:59.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:29:59.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:29:59.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:29:59.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:29:59.157 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.157 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.157 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.158 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:59.158 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:59.158 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.193052336999997, 2024-09-17T07:29:59.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:29:59.158 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.158 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.158 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.158 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:29:59.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.158 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:59.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:59.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:29:59.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:29:59.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:29:59.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:29:59.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:29:59.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:29:59.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:29:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:29:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:29:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:29:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:29:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:29:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:29:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:29:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:29:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.193032733999999, 2024-09-17T07:29:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:29:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:29:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:29:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:29:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:29:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:29:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:29:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:29:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:29:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:29:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:29:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:29:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:29:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.240091620999998, 2024-09-17T07:29:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:29:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:29:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:29:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:29:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:29:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:29:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:29:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:29:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:29:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:29:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:29:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:29:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:29:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:29:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:29:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:29:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:29:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.240077960999997, 2024-09-17T07:29:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:29:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:29:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:29:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:29:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:29:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:29:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:29:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:29:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:29:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:29:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:29:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:29:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:29:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:29:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:29:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:29:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.240063179000003, 2024-09-17T07:29:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:29:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:29:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:29:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:29:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:29:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:29:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:29:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:29:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:29:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:29:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:29:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:29:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:29:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:29:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:29:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:29:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.240052288000001, 2024-09-17T07:29:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:29:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:29:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:29:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:29:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:29:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:29:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:29:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:29:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:29:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:29:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:29:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:29:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:29:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:29:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:29:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.240042727999999, 2024-09-17T07:29:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:29:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:29:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:29:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:29:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:29:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:29:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:29:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:29:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:29:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:29:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:29:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:29:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:29:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:29:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:29:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:29:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:29:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:29:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.240032167999999, 2024-09-17T07:29:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:29:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:29:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:29:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:29:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:29:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:29:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:29:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:29:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:29:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:29:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:29:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:29:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:29:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:29:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:29:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:29:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:29:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.193 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:59.195 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:29:59.195 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:59.194+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:29:59.295 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:29:59.294+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:29:59.343 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:59.343 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:59.344 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:29:59.344 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:59.352 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:29:59.355 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-17T07:29:59.360 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:59.360 INFO:teuthology.orchestra.run.smithi045.stdout:5393 punch from 0x1af08db to 0x1afa973, (0xa098 bytes) 2024-09-17T07:29:59.364 INFO:teuthology.orchestra.run.smithi045.stdout:5395 writesame 0xc121f16 thru 0xc123208 data_size 0x15(0x12f3 bytes) 2024-09-17T07:29:59.369 INFO:teuthology.orchestra.run.smithi045.stdout:5396 read 0x601e1c5 thru 0x602c560 (0xe39c bytes) 2024-09-17T07:29:59.369 INFO:teuthology.orchestra.run.smithi045.stdout:5397 punch from 0x7111d42 to 0x71183dd, (0x669b bytes) 2024-09-17T07:29:59.380 INFO:teuthology.orchestra.run.smithi045.stdout:5398 trunc from 0xdec0569 to 0x44b33ae 2024-09-17T07:29:59.396 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:59.396+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:59.410 INFO:teuthology.orchestra.run.smithi045.stdout:5399 punch from 0x17244b5 to 0x172ad30, (0x687b bytes) 2024-09-17T07:29:59.410 INFO:teuthology.orchestra.run.smithi045.stdout:5400 read 0x2a34f7e thru 0x2a3c322 (0x73a5 bytes) 2024-09-17T07:29:59.411 INFO:teuthology.orchestra.run.smithi045.stdout:5401 read 0x394f7c8 thru 0x394f7ef (0x28 bytes) 2024-09-17T07:29:59.411 INFO:teuthology.orchestra.run.smithi045.stdout:5402 punch from 0x3a8b09f to 0x3a996df, (0xe640 bytes) 2024-09-17T07:29:59.411 INFO:teuthology.orchestra.run.smithi045.stdout:5403 read 0x37bcc53 thru 0x37bfc3c (0x2fea bytes) 2024-09-17T07:29:59.411 INFO:teuthology.orchestra.run.smithi045.stdout:5404 read 0x3c3cc0f thru 0x3c3eb4f (0x1f41 bytes) 2024-09-17T07:29:59.411 INFO:teuthology.orchestra.run.smithi045.stdout:5405 write 0x1cf3e3d thru 0x1cf6484 (0x2648 bytes) 2024-09-17T07:29:59.413 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:59.424 INFO:teuthology.orchestra.run.smithi045.stdout:5411 write 0x94e4742 thru 0x94e7e00 (0x36bf bytes) 2024-09-17T07:29:59.426 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:59.427 INFO:teuthology.orchestra.run.smithi045.stdout:5413 trunc from 0x94e7e01 to 0xadc6eb8 2024-09-17T07:29:59.438 INFO:teuthology.orchestra.run.smithi045.stdout:5415 trunc from 0xadc6eb8 to 0xe9bda77 2024-09-17T07:29:59.444 INFO:teuthology.orchestra.run.smithi045.stdout:5416 punch from 0xccc7a86 to 0xccd4d22, (0xd29c bytes) 2024-09-17T07:29:59.444 INFO:teuthology.orchestra.run.smithi045.stdout:5417 punch from 0xa0a60a6 to 0xa0b185c, (0xb7b6 bytes) 2024-09-17T07:29:59.444 INFO:teuthology.orchestra.run.smithi045.stdout:5418 write 0x6f72b0f thru 0x6f77c20 (0x5112 bytes) 2024-09-17T07:29:59.446 INFO:teuthology.orchestra.run.smithi045.stdout:5419 write 0x90fa16e thru 0x9105751 (0xb5e4 bytes) 2024-09-17T07:29:59.447 INFO:teuthology.orchestra.run.smithi045.stdout:5420 read 0xccd5581 thru 0xccd6caa (0x172a bytes) 2024-09-17T07:29:59.447 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:59.448 INFO:teuthology.orchestra.run.smithi045.stdout:5423 read 0xb8b8881 thru 0xb8c6985 (0xe105 bytes) 2024-09-17T07:29:59.448 INFO:teuthology.orchestra.run.smithi045.stdout:5425 write 0xb576407 thru 0xb5848d0 (0xe4ca bytes) 2024-09-17T07:29:59.449 INFO:teuthology.orchestra.run.smithi045.stdout:5428 read 0x46a67a2 thru 0x46acef9 (0x6758 bytes) 2024-09-17T07:29:59.450 INFO:teuthology.orchestra.run.smithi045.stdout:5429 trunc from 0xe9bda77 to 0xea1d9b5 2024-09-17T07:29:59.455 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:59.455 INFO:teuthology.orchestra.run.smithi045.stdout:5431 write 0x43449c7 thru 0x434fb1f (0xb159 bytes) 2024-09-17T07:29:59.457 INFO:teuthology.orchestra.run.smithi045.stdout:5432 punch from 0x101e759 to 0x102e44c, (0xfcf3 bytes) 2024-09-17T07:29:59.457 INFO:teuthology.orchestra.run.smithi045.stdout:5433 read 0x8b5727a thru 0x8b57e0f (0xb96 bytes) 2024-09-17T07:29:59.457 INFO:teuthology.orchestra.run.smithi045.stdout:5434 write 0x6d08d71 thru 0x6d17f49 (0xf1d9 bytes) 2024-09-17T07:29:59.459 INFO:teuthology.orchestra.run.smithi045.stdout:5435 read 0x1bb939 thru 0x1c3e2c (0x84f4 bytes) 2024-09-17T07:29:59.459 INFO:teuthology.orchestra.run.smithi045.stdout:5438 writesame 0xe0efb42 thru 0xe0f15ec data_size 0x1(0x1aab bytes) 2024-09-17T07:29:59.464 INFO:teuthology.orchestra.run.smithi045.stdout:5439 read 0x2e2d996 thru 0x2e2fd5b (0x23c6 bytes) 2024-09-17T07:29:59.464 INFO:teuthology.orchestra.run.smithi045.stdout:5440 read 0xb342fdc thru 0xb3430d7 (0xfc bytes) 2024-09-17T07:29:59.465 INFO:teuthology.orchestra.run.smithi045.stdout:5441 punch from 0x12ee05c to 0x12fc94e, (0xe8f2 bytes) 2024-09-17T07:29:59.465 INFO:teuthology.orchestra.run.smithi045.stdout:5442 write 0x9ed58f thru 0x9f29f4 (0x5466 bytes) 2024-09-17T07:29:59.467 INFO:teuthology.orchestra.run.smithi045.stdout:5443 punch from 0x9bf749d to 0x9c040a3, (0xcc06 bytes) 2024-09-17T07:29:59.467 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:59.467 INFO:teuthology.orchestra.run.smithi045.stdout:5447 read 0x5bf4728 thru 0x5bf8cea (0x45c3 bytes) 2024-09-17T07:29:59.467 INFO:teuthology.orchestra.run.smithi045.stdout:5449 punch from 0x4e20dc7 to 0x4e27093, (0x62cc bytes) 2024-09-17T07:29:59.467 INFO:teuthology.orchestra.run.smithi045.stdout:5450 write 0xaac2ef7 thru 0xaac9404 (0x650e bytes) 2024-09-17T07:29:59.468 INFO:teuthology.orchestra.run.smithi045.stdout:5451 read 0x6f83ffa thru 0x6f9018e (0xc195 bytes) 2024-09-17T07:29:59.469 INFO:teuthology.orchestra.run.smithi045.stdout:5452 writesame 0xa5a0f53 thru 0xa5ac0d0 data_size 0x3(0xb17e bytes) 2024-09-17T07:29:59.470 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:59.471 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:29:59.471 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:29:59.471 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:29:59.471 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:59.480 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:29:59.483 INFO:teuthology.orchestra.run.smithi045.stdout:5454 trunc from 0xea1d9b5 to 0x8a6145d 2024-09-17T07:29:59.493 INFO:teuthology.orchestra.run.smithi045.stdout:5455 writesame 0x67edb53 thru 0x67f9b50 data_size 0x19(0xbffe bytes) 2024-09-17T07:29:59.501 INFO:teuthology.orchestra.run.smithi045.stdout:5456 write 0x95bf643 thru 0x95c3513 (0x3ed1 bytes) 2024-09-17T07:29:59.508 INFO:teuthology.orchestra.run.smithi045.stdout:5457 trunc from 0x95c3514 to 0x4cf706f 2024-09-17T07:29:59.518 INFO:teuthology.orchestra.run.smithi045.stdout:5458 read 0xcdba0f thru 0xce4a10 (0x9002 bytes) 2024-09-17T07:29:59.522 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:29:59.522+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:29:59.535 INFO:teuthology.orchestra.run.smithi045.stdout:5459 write 0xd479220 thru 0xd4815a3 (0x8384 bytes) 2024-09-17T07:29:59.537 INFO:teuthology.orchestra.run.smithi045.stdout:5461 read 0xc788c8 thru 0xc864b0 (0xdbe9 bytes) 2024-09-17T07:29:59.537 INFO:teuthology.orchestra.run.smithi045.stdout:5464 write 0x593c951 thru 0x5947667 (0xad17 bytes) 2024-09-17T07:29:59.547 INFO:teuthology.orchestra.run.smithi045.stdout:5469 trunc from 0xd4815a4 to 0x66401b6 2024-09-17T07:29:59.557 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:59.557 INFO:teuthology.orchestra.run.smithi045.stdout:5471 read 0x1457f56 thru 0x1462aba (0xab65 bytes) 2024-09-17T07:29:59.557 INFO:teuthology.orchestra.run.smithi045.stdout:5472 write 0x55e06b3 thru 0x55efdf6 (0xf744 bytes) 2024-09-17T07:29:59.559 INFO:teuthology.orchestra.run.smithi045.stdout:5473 read 0x52f268e thru 0x52f58cc (0x323f bytes) 2024-09-17T07:29:59.567 INFO:teuthology.orchestra.run.smithi045.stdout:5475 writesame 0x7fbdec6 thru 0x7fc397b data_size 0x2(0x5ab6 bytes) 2024-09-17T07:29:59.569 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:59.583 INFO:teuthology.orchestra.run.smithi045.stdout:5478 write 0xe9d3bec thru 0xe9db898 (0x7cad bytes) 2024-09-17T07:29:59.585 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:59.585 INFO:teuthology.orchestra.run.smithi045.stdout:5482 write 0x199ee8d thru 0x19ac6aa (0xd81e bytes) 2024-09-17T07:29:59.587 INFO:teuthology.orchestra.run.smithi045.stdout:5483 read 0xeaa7dc thru 0xeaebfe (0x4423 bytes) 2024-09-17T07:29:59.587 INFO:teuthology.orchestra.run.smithi045.stdout:5484 read 0x996ffc1 thru 0x99789c3 (0x8a03 bytes) 2024-09-17T07:29:59.592 INFO:teuthology.orchestra.run.smithi045.stdout:5485 trunc from 0xe9db899 to 0xa0d08cc 2024-09-17T07:29:59.597 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:29:59.597+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:29:59.602 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:59.602 INFO:teuthology.orchestra.run.smithi045.stdout:5487 write 0x6562f50 thru 0x6569499 (0x654a bytes) 2024-09-17T07:29:59.613 INFO:teuthology.orchestra.run.smithi045.stdout:5488 trunc from 0xa0d08cc to 0x1255c6b 2024-09-17T07:29:59.632 INFO:teuthology.orchestra.run.smithi045.stdout:5489 write 0x3ba8736 thru 0x3bb0348 (0x7c13 bytes) 2024-09-17T07:29:59.633 INFO:teuthology.orchestra.run.smithi045.stdout:5490 write 0x2117817 thru 0x211f5d1 (0x7dbb bytes) 2024-09-17T07:29:59.635 INFO:teuthology.orchestra.run.smithi045.stdout:5491 read 0x204d671 thru 0x204f5d0 (0x1f60 bytes) 2024-09-17T07:29:59.635 INFO:teuthology.orchestra.run.smithi045.stdout:5493 read 0x32aa912 thru 0x32acec8 (0x25b7 bytes) 2024-09-17T07:29:59.636 INFO:teuthology.orchestra.run.smithi045.stdout:5494 writesame 0xa9c972 thru 0xaa4362 data_size 0x1f(0x79f1 bytes) 2024-09-17T07:29:59.639 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:29:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:29:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:29:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:29:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:29:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 93.302750266999993, 2024-09-17T07:29:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:29:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:29:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:29:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:29:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:29:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:29:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:29:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:29:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:29:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:29:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:29:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:29:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:29:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:29:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:29:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:29:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:29:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:29:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:29:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:29:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:29:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:29:59.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:29:59.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:29:59.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:29:59.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:29:59.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:29:59.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:29:59.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:29:59.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:29:59.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:29:59.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:29:59.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:29:59.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:29:59.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:29:59.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:29:59.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:29:59.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.647 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.647 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7378 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7176192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.647 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:59.647 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.713708233000006, 2024-09-17T07:29:59.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073003980999999996, 2024-09-17T07:29:59.647 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.648 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.648 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.648 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7378 2024-09-17T07:29:59.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.648 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:59.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:59.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:29:59.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1799999999999999e-07 2024-09-17T07:29:59.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923713+0000", 2024-09-17T07:29:59.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.057e-06 2024-09-17T07:29:59.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923716+0000", 2024-09-17T07:29:59.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0599999999999999e-06 2024-09-17T07:29:59.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923719+0000", 2024-09-17T07:29:59.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8219999999999998e-06 2024-09-17T07:29:59.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996401+0000", 2024-09-17T07:29:59.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072681467999999999 2024-09-17T07:29:59.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996452+0000", 2024-09-17T07:29:59.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.134e-05 2024-09-17T07:29:59.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996716+0000", 2024-09-17T07:29:59.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000264016 2024-09-17T07:29:59.652 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.652 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.652 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.652 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.653 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:59.653 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.713698117000007, 2024-09-17T07:29:59.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:29:59.653 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.653 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.653 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.653 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.653 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:29:59.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.654 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:59.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:29:59.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:29:59.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:29:59.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:29:59.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:29:59.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:29:59.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:29:59.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:29:59.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:29:59.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:29:59.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:29:59.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:29:59.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:29:59.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:29:59.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:29:59.657 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.657 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.658 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.658 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7398 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6389760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.658 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:59.658 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.708429582999997, 2024-09-17T07:29:59.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072891580999999997, 2024-09-17T07:29:59.658 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.658 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.658 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.659 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.659 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7398 2024-09-17T07:29:59.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.659 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:59.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:59.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:29:59.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:59.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928995+0000", 2024-09-17T07:29:59.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0759999999999996e-06 2024-09-17T07:29:59.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928999+0000", 2024-09-17T07:29:59.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1609999999999996e-06 2024-09-17T07:29:59.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929002+0000", 2024-09-17T07:29:59.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8449999999999999e-06 2024-09-17T07:29:59.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001598+0000", 2024-09-17T07:29:59.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072596318000000007 2024-09-17T07:29:59.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001663+0000", 2024-09-17T07:29:59.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4995999999999999e-05 2024-09-17T07:29:59.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001882+0000", 2024-09-17T07:29:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021897299999999999 2024-09-17T07:29:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.708415001000006, 2024-09-17T07:29:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:29:59.663 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:29:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:59.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:29:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:29:59.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:29:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:29:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:29:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:29:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:29:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:29:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:29:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:29:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:29:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:29:59.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:29:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:29:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.668 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.708384021000001, 2024-09-17T07:29:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:29:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:29:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:29:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:29:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:29:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:29:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:29:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:29:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:29:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:29:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:29:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:29:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:29:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:29:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:29:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:29:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:29:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.708353971999998, 2024-09-17T07:29:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:29:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:29:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:29:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:29:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:29:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:29:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:29:59.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:29:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:29:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:29:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:29:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:29:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:29:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:29:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:29:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:29:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:29:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.708325564999996, 2024-09-17T07:29:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:29:59.679 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:29:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:29:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:29:59.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:29:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:29:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:29:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:29:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:29:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:29:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:29:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:29:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:29:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:29:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:29:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:29:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.684 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.708305744, 2024-09-17T07:29:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:29:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:29:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.685 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:29:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:29:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:29:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:29:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:29:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:29:59.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:29:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:29:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:29:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:29:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:29:59.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:29:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:29:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:29:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:29:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.689 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.708281491999998, 2024-09-17T07:29:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:29:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.690 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:29:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:29:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:29:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:29:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:29:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:29:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:29:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:29:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:29:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:29:59.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:29:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:29:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:29:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:29:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:29:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:29:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7418 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7700480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.702875469000006, 2024-09-17T07:29:59.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072752965000000003, 2024-09-17T07:29:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7418 2024-09-17T07:29:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:29:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3599999999999999e-07 2024-09-17T07:29:59.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934547+0000", 2024-09-17T07:29:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3350000000000001e-06 2024-09-17T07:29:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934554+0000", 2024-09-17T07:29:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9859999999999997e-06 2024-09-17T07:29:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934557+0000", 2024-09-17T07:29:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.818e-06 2024-09-17T07:29:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007104+0000", 2024-09-17T07:29:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072546779000000006 2024-09-17T07:29:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007147+0000", 2024-09-17T07:29:59.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2904000000000001e-05 2024-09-17T07:29:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007298+0000", 2024-09-17T07:29:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015080700000000001 2024-09-17T07:29:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.700 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.702858903999996, 2024-09-17T07:29:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:29:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:29:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.701 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:29:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:29:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:29:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:29:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:29:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:29:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:29:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:29:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:29:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:29:59.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:29:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:29:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:29:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:29:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:29:59.705 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.702812029, 2024-09-17T07:29:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:29:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.706 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:29:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:29:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:29:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:29:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:29:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:29:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:29:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:29:59.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:29:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:29:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:29:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:29:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:29:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:29:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:29:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:59.711 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.702797325000006, 2024-09-17T07:29:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:29:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:29:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:29:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:29:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:29:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:29:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:29:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:29:59.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:29:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:29:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:29:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:29:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:29:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:29:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:29:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:29:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.717 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.717 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:59.717 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.702774230000003, 2024-09-17T07:29:59.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:29:59.717 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.717 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.717 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.717 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:29:59.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.718 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:59.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:59.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:29:59.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:29:59.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:29:59.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:29:59.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:29:59.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:29:59.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:29:59.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:29:59.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:29:59.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:29:59.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:29:59.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:29:59.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:29:59.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:29:59.721 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.722 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.722 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.722 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.722 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:59.722 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.702742854999997, 2024-09-17T07:29:59.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:29:59.722 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.722 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.722 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.722 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.723 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.723 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.723 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:29:59.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.723 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:59.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:29:59.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:29:59.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:29:59.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:29:59.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:29:59.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:29:59.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:29:59.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:29:59.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:29:59.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:29:59.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:29:59.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:29:59.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:29:59.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:29:59.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:29:59.727 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.727 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.727 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.727 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.727 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:59.727 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.702727594999999, 2024-09-17T07:29:59.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:29:59.727 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.727 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.727 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.728 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.728 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.728 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.728 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:29:59.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.728 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:59.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:59.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:29:59.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:29:59.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:29:59.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:29:59.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:29:59.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:29:59.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:29:59.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:29:59.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:29:59.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:29:59.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:29:59.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:29:59.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:29:59.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:29:59.732 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.732 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.732 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.732 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.732 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:59.732 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.702707524999994, 2024-09-17T07:29:59.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:29:59.733 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.733 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.733 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.733 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:29:59.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.733 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:59.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:59.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:29:59.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:29:59.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:29:59.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:29:59.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:29:59.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:29:59.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:29:59.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:29:59.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:29:59.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:29:59.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:29:59.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:29:59.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:29:59.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:29:59.737 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.737 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.737 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.737 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.738 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:59.738 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.702695585000001, 2024-09-17T07:29:59.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:29:59.738 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.738 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.738 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.738 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:29:59.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.739 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:59.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:59.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:29:59.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:29:59.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:29:59.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:29:59.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:29:59.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:29:59.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:29:59.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:29:59.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:29:59.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:29:59.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:29:59.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:29:59.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:29:59.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:29:59.742 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.742 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.742 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.743 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:29:59.743 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:59.743 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.702672426999996, 2024-09-17T07:29:59.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:29:59.743 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:29:59.743 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:29:59.743 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:29:59.743 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:29:59.743 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:29:59.743 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:29:59.744 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:29:59.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.744 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:29:59.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:29:59.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:59.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:29:59.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:29:59.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:29:59.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:29:59.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:29:59.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:29:59.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:29:59.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:29:59.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:29:59.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:29:59.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:29:59.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:29:59.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:29:59.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:29:59.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:29:59.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:29:59.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:29:59.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:29:59.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:29:59.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:29:59.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:29:59.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:29:59.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:29:59.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:29:59.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:29:59.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:29:59.747 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.748 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.748 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.748 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:29:59.748 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:29:59.748 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:29:59.750 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:29:59.751 INFO:teuthology.orchestra.run.smithi045.stdout:5495 write 0x8d1e12b thru 0x8d23b57 (0x5a2d bytes) 2024-09-17T07:29:59.751 INFO:teuthology.orchestra.run.smithi045.stdout:5496 write 0x64759ed thru 0x6482584 (0xcb98 bytes) 2024-09-17T07:29:59.751 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:59.751 INFO:teuthology.orchestra.run.smithi045.stdout:5498 trunc from 0x8d23b58 to 0xa9a733a 2024-09-17T07:29:59.751 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:59.751 INFO:teuthology.orchestra.run.smithi045.stdout:5500 write 0x5d1e020 thru 0x5d27244 (0x9225 bytes) 2024-09-17T07:29:59.752 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:59.752 INFO:teuthology.orchestra.run.smithi045.stdout:5502 read 0x11bd7ef thru 0x11c7a3a (0xa24c bytes) 2024-09-17T07:29:59.752 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:29:59.752 INFO:teuthology.orchestra.run.smithi045.stdout:5504 punch from 0x969875e to 0x96996d3, (0xf75 bytes) 2024-09-17T07:29:59.752 INFO:teuthology.orchestra.run.smithi045.stdout:5506 writesame 0xc5c331c thru 0xc5d2491 data_size 0x1f(0xf176 bytes) 2024-09-17T07:29:59.752 INFO:teuthology.orchestra.run.smithi045.stdout:5508 punch from 0xa1fe4a0 to 0xa202f89, (0x4ae9 bytes) 2024-09-17T07:29:59.752 INFO:teuthology.orchestra.run.smithi045.stdout:5509 clone 43 order 25 su 524288 sc 13 2024-09-17T07:29:59.752 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:29:59.748+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:29:59.850 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:29:59.848+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:29:59.859 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:29:59.859 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:29:59.859 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:29:59.860 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:29:59.868 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:29:59.951 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:29:59.950+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:00.015 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:00.015 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:00.015 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:00.023 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:00.051 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:00.051+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:00.135 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:00.135 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:00.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:00.135 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:00.135 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.135 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:00.135 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:00.135 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 62.000824518000002, 2024-09-17T07:30:00.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:30:00.136 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.136 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.136 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:00.136 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.136 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.136 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.136 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:30:00.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.136 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.136 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.136 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.136 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:00.136 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.137 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:00.137 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.137 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:30:00.137 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:30:00.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.137 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:30:00.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:30:00.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.138 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.138 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:30:00.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:30:00.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.138 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.138 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:30:00.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:30:00.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.138 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.138 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:30:00.139 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:30:00.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.139 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:30:00.139 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:30:00.139 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:30:00.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.139 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.139 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:30:00.139 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:30:00.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.140 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:00.140 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:30:00.140 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:30:00.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.140 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.140 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:30:00.140 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:30:00.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.140 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:00.140 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:30:00.140 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:30:00.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.141 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.141 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:30:00.141 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:30:00.141 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.141 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.141 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.141 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.141 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:00.141 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.694857814000002, 2024-09-17T07:30:00.141 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:30:00.142 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.142 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.142 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:00.142 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.142 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.142 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.142 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:30:00.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.142 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.142 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.142 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.142 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:00.142 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.142 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.143 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.143 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:00.143 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.143 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.143 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:30:00.143 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:30:00.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.143 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.143 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:30:00.143 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:30:00.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.144 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.144 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.144 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:30:00.144 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:30:00.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.144 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.144 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.144 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:30:00.144 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:30:00.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.145 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.145 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.145 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:30:00.145 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:30:00.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.145 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.145 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:00.145 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:30:00.145 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:30:00.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.145 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.145 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.145 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:30:00.145 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:30:00.146 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.146 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.146 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.146 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.146 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:00.146 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.694835699000002, 2024-09-17T07:30:00.146 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:30:00.146 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.146 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.146 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.146 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.146 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.147 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.147 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:30:00.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.147 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.147 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.147 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.147 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:00.147 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.147 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.147 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.147 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:00.147 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.148 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.148 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.148 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:30:00.148 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:00.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.148 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.148 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.148 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:30:00.148 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:30:00.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.148 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.148 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.148 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:30:00.149 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:30:00.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.149 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.149 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:30:00.149 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:30:00.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.149 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.149 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:30:00.149 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:30:00.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.149 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.150 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:30:00.150 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:30:00.150 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.150 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.150 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.150 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.150 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:00.150 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.694811594999997, 2024-09-17T07:30:00.150 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:30:00.150 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.150 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.150 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.151 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.151 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.151 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.151 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:30:00.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.151 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.151 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.151 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:00.151 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.151 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.151 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:00.151 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.152 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.152 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:30:00.152 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:00.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.152 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.152 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:30:00.152 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:30:00.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.152 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.152 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:30:00.153 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:30:00.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.153 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.153 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:30:00.153 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:30:00.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.153 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.153 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:30:00.153 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:30:00.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.153 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.154 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:30:00.154 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:30:00.154 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.154 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.154 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.154 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.154 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:00.154 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.694787505999997, 2024-09-17T07:30:00.154 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:30:00.154 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.154 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.154 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.155 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.155 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.155 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.155 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:30:00.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.155 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.155 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.155 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:00.155 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.155 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.155 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:00.156 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.156 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.156 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:00.156 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:30:00.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.156 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.156 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:30:00.156 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:30:00.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.157 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.157 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:30:00.157 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:30:00.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.157 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.157 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:30:00.157 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:30:00.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.157 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.157 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:30:00.157 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:30:00.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.158 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.158 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:30:00.158 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:30:00.158 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.158 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.158 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.158 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.158 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:00.158 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.694763086999998, 2024-09-17T07:30:00.158 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:30:00.158 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.159 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.159 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.159 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.159 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.159 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:30:00.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.159 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:00.159 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:00.160 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.160 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:00.160 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:30:00.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.160 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:30:00.160 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:30:00.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.161 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:30:00.161 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:30:00.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.161 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.161 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:30:00.161 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:00.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.161 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.161 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:30:00.161 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:30:00.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.162 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.162 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:30:00.162 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:30:00.162 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.162 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.162 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.162 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.162 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:00.162 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.694741387000001, 2024-09-17T07:30:00.162 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:30:00.162 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.163 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.163 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.163 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.163 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.163 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:30:00.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.163 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.163 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.163 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:00.163 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.163 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:00.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:00.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:30:00.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:30:00.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:30:00.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.165 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.165 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:30:00.165 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:30:00.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.165 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.165 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:30:00.165 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:30:00.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.165 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.165 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:30:00.165 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:30:00.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.166 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.166 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:30:00.166 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:30:00.166 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.166 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.166 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.166 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.166 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:00.166 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.694696655000001, 2024-09-17T07:30:00.166 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:30:00.167 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.167 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.167 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.167 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.167 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.167 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.167 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:30:00.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.167 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.167 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.167 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:00.167 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.168 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.168 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:00.168 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.168 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.168 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:30:00.168 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:30:00.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.168 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.168 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:30:00.169 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:30:00.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.169 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.169 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:30:00.169 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:30:00.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.169 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.169 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:30:00.169 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:30:00.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.170 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.170 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:30:00.170 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:30:00.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.170 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.170 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:30:00.170 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:30:00.170 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.170 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.170 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.170 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.171 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:00.171 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.694668188999998, 2024-09-17T07:30:00.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:30:00.171 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.171 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.171 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.171 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.171 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.171 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.171 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:30:00.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.171 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.171 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:00.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.172 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:00.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.172 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:00.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:30:00.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.173 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.173 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:30:00.173 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:30:00.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.173 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.173 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:30:00.173 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:30:00.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.173 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.173 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:30:00.173 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:30:00.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.174 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.174 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:30:00.174 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:30:00.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.174 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.174 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:30:00.174 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:30:00.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.174 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.174 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.175 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:00.175 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.694550937000002, 2024-09-17T07:30:00.175 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:30:00.175 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.175 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.175 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.175 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.175 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.175 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.175 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:30:00.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.175 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.175 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.176 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:00.176 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.176 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.176 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:00.176 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.176 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.176 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:30:00.176 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:30:00.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.177 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.177 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:30:00.177 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:30:00.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.177 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.177 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:30:00.177 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:30:00.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.177 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.177 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:30:00.177 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:30:00.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.178 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.178 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:30:00.178 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:30:00.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.178 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.178 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:30:00.178 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:30:00.178 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.178 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.178 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.178 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.179 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:00.179 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.694124834999997, 2024-09-17T07:30:00.179 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:30:00.179 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.179 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.179 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.179 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.179 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.179 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.179 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:30:00.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.179 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.179 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.180 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:00.180 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.180 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.180 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:00.180 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.180 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.180 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:30:00.180 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:00.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.181 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.181 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:30:00.181 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:30:00.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.181 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.181 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:30:00.181 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:30:00.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.181 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.181 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:30:00.181 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:30:00.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.182 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.182 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:30:00.182 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:30:00.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.182 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.182 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:30:00.182 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:30:00.182 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.182 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.182 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.694105164, 2024-09-17T07:30:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:30:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:30:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.184 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:00.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.184 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.184 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:00.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.184 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.184 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:00.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:30:00.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.185 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.185 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:30:00.185 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:30:00.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.185 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.185 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:30:00.185 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:30:00.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.185 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.185 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:30:00.185 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:30:00.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.186 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.186 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:30:00.186 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:30:00.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.186 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.186 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:30:00.186 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:30:00.186 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.186 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.186 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.186 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.187 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:00.187 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.694082299999998, 2024-09-17T07:30:00.187 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:30:00.187 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.187 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.187 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.187 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.187 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.187 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:30:00.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.187 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.187 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.187 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:00.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:00.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:00.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:30:00.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.189 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:30:00.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:30:00.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.189 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.189 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:30:00.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:30:00.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.189 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.189 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:30:00.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:30:00.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.190 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.190 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:30:00.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:30:00.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.190 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.190 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:30:00.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:30:00.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.190 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.190 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.190 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:00.191 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.694060999000001, 2024-09-17T07:30:00.191 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:30:00.191 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.191 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.191 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.191 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.191 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.191 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.191 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:30:00.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.191 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.191 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.191 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:00.192 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.192 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.192 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:00.192 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.192 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.192 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:30:00.192 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:30:00.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.192 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.193 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:30:00.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:30:00.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.193 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.193 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:30:00.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:30:00.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.193 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.193 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:30:00.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:30:00.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.194 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.194 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:30:00.194 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:30:00.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.194 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.194 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:30:00.194 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:30:00.194 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.194 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.194 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.194 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.195 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:00.195 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.694013210999998, 2024-09-17T07:30:00.195 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:30:00.195 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.195 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.195 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.195 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.195 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.195 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.195 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:30:00.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.195 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.195 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.195 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:00.196 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.196 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:00.196 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.196 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:30:00.196 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:30:00.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.196 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:30:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:30:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:30:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:30:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:30:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:30:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.197 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:30:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:30:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:30:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:30:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.198 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.693985988999998, 2024-09-17T07:30:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:30:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:30:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:30:00.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:30:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:30:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:30:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:30:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:30:00.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:30:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:30:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:30:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:30:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:30:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.693965601999999, 2024-09-17T07:30:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:30:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:30:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.203 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.204 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:30:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:30:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:30:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:30:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.205 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:30:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:30:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:30:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:30:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:30:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:30:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:30:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:30:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.208 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.693945896999999, 2024-09-17T07:30:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:30:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:30:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.209 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:30:00.210 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:30:00.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.211 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.211 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:30:00.211 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:30:00.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.211 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.211 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:30:00.211 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:30:00.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.211 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.211 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:30:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:30:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:30:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:30:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:30:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:30:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.212 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.693922819000001, 2024-09-17T07:30:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:30:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.213 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:30:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.214 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:30:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:30:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:30:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:30:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:30:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:30:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:30:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:30:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:30:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:30:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:30:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:30:00.216 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.693902256999998, 2024-09-17T07:30:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:30:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:00.217 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:30:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.219 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:00.219 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:00.219 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:30:00.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.219 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:00.219 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:30:00.219 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:30:00.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.219 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:00.219 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:30:00.219 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:30:00.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.220 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:00.220 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:30:00.220 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:30:00.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.220 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:00.220 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:30:00.220 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:30:00.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:00.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:00.220 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:00.220 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:30:00.221 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:30:00.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.221 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:00.221 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:00.221 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:00.221 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T07:30:00.222 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:30:00.223 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:00.222+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:00.224 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:00.224 INFO:teuthology.orchestra.run.smithi045.stdout:flattening image image_client.0-clone42 2024-09-17T07:30:00.323 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:00.323+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:00.339 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-17T07:30:00.384 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:00.384 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:00.384 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:00.384 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:00.400 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:00.410 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:00.410 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:00.430 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:00.431 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:00.431 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:00.429+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:00.448 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:00.449 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:00.465 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:00.466 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-17T07:30:00.483 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:00.483 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:00.501 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:00.501 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:00.518 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:00.518 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:00.526 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:00.524+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:00.526 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:00.526 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:00.535 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:00.536 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:00.536 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:00.598 INFO:teuthology.orchestra.run.smithi045.stdout:5510 write 0xe5c1bd7 thru 0xe5cdf8c (0xc3b6 bytes) 2024-09-17T07:30:00.605 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:00.605 INFO:teuthology.orchestra.run.smithi045.stdout:5515 write 0x7a0a03f thru 0x7a109a2 (0x6964 bytes) 2024-09-17T07:30:00.610 INFO:teuthology.orchestra.run.smithi045.stdout:5516 read 0x16a369f thru 0x16b1ffc (0xe95e bytes) 2024-09-17T07:30:00.611 INFO:teuthology.orchestra.run.smithi045.stdout:5518 punch from 0x7550323 to 0x7551a95, (0x1772 bytes) 2024-09-17T07:30:00.611 INFO:teuthology.orchestra.run.smithi045.stdout:5519 write 0x61b2514 thru 0x61ba1e4 (0x7cd1 bytes) 2024-09-17T07:30:00.627 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:00.627+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:00.632 INFO:teuthology.orchestra.run.smithi045.stdout:5520 trunc from 0xe5cdf8d to 0x22c5fdd 2024-09-17T07:30:00.663 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:00.664 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:00.664 INFO:teuthology.orchestra.run.smithi045.stdout:5526 read 0x224fb30 thru 0x225844a (0x891b bytes) 2024-09-17T07:30:00.664 INFO:teuthology.orchestra.run.smithi045.stdout:5527 punch from 0x69211b to 0x698247, (0x612c bytes) 2024-09-17T07:30:00.664 INFO:teuthology.orchestra.run.smithi045.stdout:5528 writesame 0xa6b458 thru 0xa76586 data_size 0x1(0xb12f bytes) 2024-09-17T07:30:00.687 INFO:teuthology.orchestra.run.smithi045.stdout:5529 trunc from 0x22c5fdd to 0x4140378 2024-09-17T07:30:00.692 INFO:teuthology.orchestra.run.smithi045.stdout:5531 punch from 0x26e7975 to 0x26f5f71, (0xe5fc bytes) 2024-09-17T07:30:00.692 INFO:teuthology.orchestra.run.smithi045.stdout:5532 punch from 0x3862c61 to 0x386d74e, (0xaaed bytes) 2024-09-17T07:30:00.696 INFO:teuthology.orchestra.run.smithi045.stdout:5533 trunc from 0x4140378 to 0x7a35485 2024-09-17T07:30:00.702 INFO:teuthology.orchestra.run.smithi045.stdout:5534 read 0xfcbf7 thru 0x104e7c (0x8286 bytes) 2024-09-17T07:30:00.703 INFO:teuthology.orchestra.run.smithi045.stdout:5536 write 0x17eeff9 thru 0x17fcc93 (0xdc9b bytes) 2024-09-17T07:30:00.705 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:00.705 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:00.705 INFO:teuthology.orchestra.run.smithi045.stdout:5538 trunc from 0x7a35485 to 0x8201133 2024-09-17T07:30:00.710 INFO:teuthology.orchestra.run.smithi045.stdout:5539 read 0x48d8e2 thru 0x499c4c (0xc36b bytes) 2024-09-17T07:30:00.715 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:00.719 INFO:teuthology.orchestra.run.smithi045.stdout:5540 write 0xade9bd5 thru 0xadf19d5 (0x7e01 bytes) 2024-09-17T07:30:00.721 INFO:teuthology.orchestra.run.smithi045.stdout:5541 write 0x6b64566 thru 0x6b7063c (0xc0d7 bytes) 2024-09-17T07:30:00.722 INFO:teuthology.orchestra.run.smithi045.stdout:5543 read 0x223a855 thru 0x223b329 (0xad5 bytes) 2024-09-17T07:30:00.722 INFO:teuthology.orchestra.run.smithi045.stdout:5544 writesame 0x59de67a thru 0x59e0358 data_size 0x1(0x1cdf bytes) 2024-09-17T07:30:00.727 INFO:teuthology.orchestra.run.smithi045.stdout:5545 read 0x77c7bfe thru 0x77cbaa8 (0x3eab bytes) 2024-09-17T07:30:00.727 INFO:teuthology.orchestra.run.smithi045.stdout:5548 read 0x14c2c18 thru 0x14d05fe (0xd9e7 bytes) 2024-09-17T07:30:00.728 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:00.727+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:00.728 INFO:teuthology.orchestra.run.smithi045.stdout:5550 write 0xd8a597 thru 0xd93d88 (0x97f2 bytes) 2024-09-17T07:30:00.729 INFO:teuthology.orchestra.run.smithi045.stdout:5551 write 0x690b020 thru 0x69191b6 (0xe197 bytes) 2024-09-17T07:30:00.731 INFO:teuthology.orchestra.run.smithi045.stdout:5552 write 0x6b6b891 thru 0x6b7338f (0x7aff bytes) 2024-09-17T07:30:00.732 INFO:teuthology.orchestra.run.smithi045.stdout:5553 punch from 0x71d91e to 0x729ea6, (0xc588 bytes) 2024-09-17T07:30:00.733 INFO:teuthology.orchestra.run.smithi045.stdout:5555 read 0x99b81a thru 0x99d9e1 (0x21c8 bytes) 2024-09-17T07:30:00.733 INFO:teuthology.orchestra.run.smithi045.stdout:5556 write 0x6aac164 thru 0x6ab0623 (0x44c0 bytes) 2024-09-17T07:30:00.734 INFO:teuthology.orchestra.run.smithi045.stdout:5557 write 0x8ee26d9 thru 0x8eef17e (0xcaa6 bytes) 2024-09-17T07:30:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:5559 punch from 0x8a5cfd to 0x8a8f95, (0x3298 bytes) 2024-09-17T07:30:00.736 INFO:teuthology.orchestra.run.smithi045.stdout:5560 trunc from 0xadf19d6 to 0xacc645a 2024-09-17T07:30:00.743 INFO:teuthology.orchestra.run.smithi045.stdout:5561 writesame 0x50c4beb thru 0x50d0404 data_size 0xf(0xb81a bytes) 2024-09-17T07:30:00.748 INFO:teuthology.orchestra.run.smithi045.stdout:5563 trunc from 0xacc645a to 0xcb3591e 2024-09-17T07:30:00.753 INFO:teuthology.orchestra.run.smithi045.stdout:5564 write 0xfeef76 thru 0xff3040 (0x40cb bytes) 2024-09-17T07:30:00.757 INFO:teuthology.orchestra.run.smithi045.stdout:5565 trunc from 0xcb3591e to 0xeaee5d4 2024-09-17T07:30:00.763 INFO:teuthology.orchestra.run.smithi045.stdout:5566 read 0x7f4efdb thru 0x7f520da (0x3100 bytes) 2024-09-17T07:30:00.763 INFO:teuthology.orchestra.run.smithi045.stdout:5568 read 0x47cf857 thru 0x47d87f2 (0x8f9c bytes) 2024-09-17T07:30:00.763 INFO:teuthology.orchestra.run.smithi045.stdout:5569 writesame 0x223ad3c thru 0x2240cd5 data_size 0x3(0x5f9a bytes) 2024-09-17T07:30:00.768 INFO:teuthology.orchestra.run.smithi045.stdout:5570 punch from 0xadd325f to 0xaddee5c, (0xbbfd bytes) 2024-09-17T07:30:00.768 INFO:teuthology.orchestra.run.smithi045.stdout:5571 writesame 0xdf0dd1 thru 0xdfc664 data_size 0x2(0xb894 bytes) 2024-09-17T07:30:00.780 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:00.780 INFO:teuthology.orchestra.run.smithi045.stdout:5575 write 0xb90e308 thru 0xb9132a6 (0x4f9f bytes) 2024-09-17T07:30:00.782 INFO:teuthology.orchestra.run.smithi045.stdout:5576 write 0x63f38b1 thru 0x63fc675 (0x8dc5 bytes) 2024-09-17T07:30:00.783 INFO:teuthology.orchestra.run.smithi045.stdout:5577 read 0xb966614 thru 0xb967358 (0xd45 bytes) 2024-09-17T07:30:00.793 INFO:teuthology.orchestra.run.smithi045.stdout:5578 trunc from 0xeaee5d4 to 0x62ec7d1 2024-09-17T07:30:00.802 INFO:teuthology.orchestra.run.smithi045.stdout:5579 read 0x5ceb290 thru 0x5cee49d (0x320e bytes) 2024-09-17T07:30:00.812 INFO:teuthology.orchestra.run.smithi045.stdout:5580 write 0x9f40d22 thru 0x9f41b17 (0xdf6 bytes) 2024-09-17T07:30:00.813 INFO:teuthology.orchestra.run.smithi045.stdout:5581 read 0x82b99f2 thru 0x82c30ba (0x96c9 bytes) 2024-09-17T07:30:00.814 INFO:teuthology.orchestra.run.smithi045.stdout:5582 read 0x18451f4 thru 0x18452cf (0xdc bytes) 2024-09-17T07:30:00.814 INFO:teuthology.orchestra.run.smithi045.stdout:5583 read 0x25f75fe thru 0x25fcf8f (0x5992 bytes) 2024-09-17T07:30:00.814 INFO:teuthology.orchestra.run.smithi045.stdout:5584 read 0x791ece8 thru 0x7929fa4 (0xb2bd bytes) 2024-09-17T07:30:00.815 INFO:teuthology.orchestra.run.smithi045.stdout:5585 read 0x186cc77 thru 0x186f381 (0x270b bytes) 2024-09-17T07:30:00.822 INFO:teuthology.orchestra.run.smithi045.stdout:5586 write 0xb45c40a thru 0xb45c79e (0x395 bytes) 2024-09-17T07:30:00.828 INFO:teuthology.orchestra.run.smithi045.stdout:5587 trunc from 0xb45c79f to 0xea96c21 2024-09-17T07:30:00.828 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:00.827+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:00.832 INFO:teuthology.orchestra.run.smithi045.stdout:5588 read 0x7f75dca thru 0x7f791e4 (0x341b bytes) 2024-09-17T07:30:00.833 INFO:teuthology.orchestra.run.smithi045.stdout:5589 writesame 0xe034b1f thru 0xe034b35 data_size 0x1(0x17 bytes) 2024-09-17T07:30:00.834 INFO:teuthology.orchestra.run.smithi045.stdout:5591 read 0xbb89a1d thru 0xbb9483a (0xae1e bytes) 2024-09-17T07:30:00.834 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:00.837 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:00.837 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:00.842 INFO:teuthology.orchestra.run.smithi045.stdout:5593 trunc from 0xea96c21 to 0x829bfcb 2024-09-17T07:30:00.850 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:00.851 INFO:teuthology.orchestra.run.smithi045.stdout:5594 clone 44 order 25 su 1048576 sc 3 2024-09-17T07:30:00.865 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:00.929 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:00.929+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:00.972 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:00.972 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:00.981 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:01.012 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:01.012 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:01.062 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:01.061+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:01.089 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:01.089 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:01.097 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:01.138 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:01.137+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:01.142 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-17T07:30:01.205 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone43 from 0x829bfcb (overlap 0x22c5fdd) to 0x14dd31e 2024-09-17T07:30:01.212 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:01.213 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:01.225 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:01.268 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:01.268+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:01.273 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-17T07:30:01.331 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:01.331 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:01.334 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:01.334+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:01.340 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:01.436 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:01.434+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:01.483 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:01.483 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:01.486 INFO:teuthology.orchestra.run.smithi045.stdout:5595 read 0x39f6f0b thru 0x3a01389 (0xa47f bytes) 2024-09-17T07:30:01.489 INFO:teuthology.orchestra.run.smithi045.stdout:5596 read 0x3023f61 thru 0x3031106 (0xd1a6 bytes) 2024-09-17T07:30:01.489 INFO:teuthology.orchestra.run.smithi045.stdout:5597 read 0x71064fd thru 0x710fcd6 (0x97da bytes) 2024-09-17T07:30:01.489 INFO:teuthology.orchestra.run.smithi045.stdout:5599 read 0x4eff9c9 thru 0x4f02d4c (0x3384 bytes) 2024-09-17T07:30:01.489 INFO:teuthology.orchestra.run.smithi045.stdout:5600 write 0x35908b3 thru 0x359ea96 (0xe1e4 bytes) 2024-09-17T07:30:01.492 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:01.512 INFO:teuthology.orchestra.run.smithi045.stdout:5601 writesame 0xdb894d0 thru 0xdb98758 data_size 0x1(0xf289 bytes) 2024-09-17T07:30:01.512 INFO:teuthology.orchestra.run.smithi045.stdout:5604 writesame 0x7e78480 thru 0x7e7e568 data_size 0x1(0x60e9 bytes) 2024-09-17T07:30:01.528 INFO:teuthology.orchestra.run.smithi045.stdout:5605 read 0xa403fb1 thru 0xa413c76 (0xfcc6 bytes) 2024-09-17T07:30:01.528 INFO:teuthology.orchestra.run.smithi045.stdout:5606 write 0x6a0bb3b thru 0x6a19305 (0xd7cb bytes) 2024-09-17T07:30:01.532 INFO:teuthology.orchestra.run.smithi045.stdout:5607 write 0xc5074ad thru 0xc5148ce (0xd422 bytes) 2024-09-17T07:30:01.536 INFO:teuthology.orchestra.run.smithi045.stdout:5608 read 0xd2e45b8 thru 0xd2f2890 (0xe2d9 bytes) 2024-09-17T07:30:01.536 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:01.536+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:01.547 INFO:teuthology.orchestra.run.smithi045.stdout:5609 trunc from 0xdb98759 to 0x32f1988 2024-09-17T07:30:01.582 INFO:teuthology.orchestra.run.smithi045.stdout:5610 write 0x540b41b thru 0x540ee0a (0x39f0 bytes) 2024-09-17T07:30:01.584 INFO:teuthology.orchestra.run.smithi045.stdout:5611 read 0xe5e28f thru 0xe69d4e (0xbac0 bytes) 2024-09-17T07:30:01.587 INFO:teuthology.orchestra.run.smithi045.stdout:5612 trunc from 0x540ee0b to 0x739b0bf 2024-09-17T07:30:01.592 INFO:teuthology.orchestra.run.smithi045.stdout:5613 read 0x136e58c thru 0x13719de (0x3453 bytes) 2024-09-17T07:30:01.593 INFO:teuthology.orchestra.run.smithi045.stdout:5614 read 0x6b50fa9 thru 0x6b59965 (0x89bd bytes) 2024-09-17T07:30:01.593 INFO:teuthology.orchestra.run.smithi045.stdout:5616 read 0x57b151b thru 0x57be269 (0xcd4f bytes) 2024-09-17T07:30:01.593 INFO:teuthology.orchestra.run.smithi045.stdout:5618 read 0x4f2528 thru 0x4f5eb7 (0x3990 bytes) 2024-09-17T07:30:01.594 INFO:teuthology.orchestra.run.smithi045.stdout:5619 read 0x3ee01a4 thru 0x3ee1e60 (0x1cbd bytes) 2024-09-17T07:30:01.594 INFO:teuthology.orchestra.run.smithi045.stdout:5620 read 0x23821f1 thru 0x2390f96 (0xeda6 bytes) 2024-09-17T07:30:01.594 INFO:teuthology.orchestra.run.smithi045.stdout:5622 punch from 0x339413d to 0x33a2ed8, (0xed9b bytes) 2024-09-17T07:30:01.595 INFO:teuthology.orchestra.run.smithi045.stdout:5623 read 0x1dd2a72 thru 0x1ddc446 (0x99d5 bytes) 2024-09-17T07:30:01.601 INFO:teuthology.orchestra.run.smithi045.stdout:5624 write 0x844fc4e thru 0x845a97e (0xad31 bytes) 2024-09-17T07:30:01.603 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:01.603 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:01.604 INFO:teuthology.orchestra.run.smithi045.stdout:5625 punch from 0x2c0bcc5 to 0x2c0f4a8, (0x37e3 bytes) 2024-09-17T07:30:01.607 INFO:teuthology.orchestra.run.smithi045.stdout:5626 trunc from 0x845a97f to 0xaf70e00 2024-09-17T07:30:01.612 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:01.614 INFO:teuthology.orchestra.run.smithi045.stdout:5628 writesame 0x5c2343a thru 0x5c2fed1 data_size 0xc(0xca98 bytes) 2024-09-17T07:30:01.616 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.617 INFO:teuthology.orchestra.run.smithi045.stdout:5630 read 0xa63977b thru 0xa64259d (0x8e23 bytes) 2024-09-17T07:30:01.617 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.624 INFO:teuthology.orchestra.run.smithi045.stdout:5632 write 0xb4ad782 thru 0xb4baa9d (0xd31c bytes) 2024-09-17T07:30:01.626 INFO:teuthology.orchestra.run.smithi045.stdout:5633 writesame 0x28153fe thru 0x2821a35 data_size 0x4(0xc638 bytes) 2024-09-17T07:30:01.633 INFO:teuthology.orchestra.run.smithi045.stdout:5634 punch from 0xb2c7c7e to 0xb2d3e4e, (0xc1d0 bytes) 2024-09-17T07:30:01.633 INFO:teuthology.orchestra.run.smithi045.stdout:5635 read 0x605ecf3 thru 0x6063678 (0x4986 bytes) 2024-09-17T07:30:01.633 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.633 INFO:teuthology.orchestra.run.smithi045.stdout:5637 writesame 0x97c83ce thru 0x97d4671 data_size 0x2(0xc2a4 bytes) 2024-09-17T07:30:01.647 INFO:teuthology.orchestra.run.smithi045.stdout:5638 trunc from 0xb4baa9e to 0xa0fe039 2024-09-17T07:30:01.653 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:01.653+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:01.654 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.654 INFO:teuthology.orchestra.run.smithi045.stdout:5640 read 0x1dce3b thru 0x1e9c59 (0xce1f bytes) 2024-09-17T07:30:01.654 INFO:teuthology.orchestra.run.smithi045.stdout:5641 read 0x94b2d07 thru 0x94bc442 (0x973c bytes) 2024-09-17T07:30:01.655 INFO:teuthology.orchestra.run.smithi045.stdout:5642 read 0x1772d1e thru 0x177dfb6 (0xb299 bytes) 2024-09-17T07:30:01.655 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.655 INFO:teuthology.orchestra.run.smithi045.stdout:5644 read 0xdd61f1 thru 0xdd8c79 (0x2a89 bytes) 2024-09-17T07:30:01.656 INFO:teuthology.orchestra.run.smithi045.stdout:5645 read 0x68144dc thru 0x681a327 (0x5e4c bytes) 2024-09-17T07:30:01.656 INFO:teuthology.orchestra.run.smithi045.stdout:5646 read 0x4954b18 thru 0x4955856 (0xd3f bytes) 2024-09-17T07:30:01.656 INFO:teuthology.orchestra.run.smithi045.stdout:5648 read 0x4bb0f00 thru 0x4bb4a8e (0x3b8f bytes) 2024-09-17T07:30:01.656 INFO:teuthology.orchestra.run.smithi045.stdout:5649 writesame 0x1929fa7 thru 0x1934d10 data_size 0x15(0xad6a bytes) 2024-09-17T07:30:01.658 INFO:teuthology.orchestra.run.smithi045.stdout:5650 write 0xe4f5f7 thru 0xe50c66 (0x1670 bytes) 2024-09-17T07:30:01.659 INFO:teuthology.orchestra.run.smithi045.stdout:5651 write 0x6750c61 thru 0x675ebc1 (0xdf61 bytes) 2024-09-17T07:30:01.671 INFO:teuthology.orchestra.run.smithi045.stdout:5652 write 0xd45210c thru 0xd4523cc (0x2c1 bytes) 2024-09-17T07:30:01.674 INFO:teuthology.orchestra.run.smithi045.stdout:5654 write 0x122fe96 thru 0x1234ca4 (0x4e0f bytes) 2024-09-17T07:30:01.675 INFO:teuthology.orchestra.run.smithi045.stdout:5655 write 0xbdc3819 thru 0xbdc395f (0x147 bytes) 2024-09-17T07:30:01.677 INFO:teuthology.orchestra.run.smithi045.stdout:5656 read 0xa4adeb3 thru 0xa4b4ae8 (0x6c36 bytes) 2024-09-17T07:30:01.677 INFO:teuthology.orchestra.run.smithi045.stdout:5660 punch from 0x77e63f0 to 0x77ee471, (0x8081 bytes) 2024-09-17T07:30:01.677 INFO:teuthology.orchestra.run.smithi045.stdout:5661 punch from 0x1381e65 to 0x138a695, (0x8830 bytes) 2024-09-17T07:30:01.677 INFO:teuthology.orchestra.run.smithi045.stdout:5662 punch from 0x1a3c3b1 to 0x1a4609f, (0x9cee bytes) 2024-09-17T07:30:01.677 INFO:teuthology.orchestra.run.smithi045.stdout:5664 read 0x9d576b6 thru 0x9d5b978 (0x42c3 bytes) 2024-09-17T07:30:01.677 INFO:teuthology.orchestra.run.smithi045.stdout:5665 write 0x24cc162 thru 0x24d7552 (0xb3f1 bytes) 2024-09-17T07:30:01.678 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.678 INFO:teuthology.orchestra.run.smithi045.stdout:5667 read 0xccd13b6 thru 0xccde438 (0xd083 bytes) 2024-09-17T07:30:01.685 INFO:teuthology.orchestra.run.smithi045.stdout:5668 writesame 0xd550ded thru 0xd556f47 data_size 0x1(0x615b bytes) 2024-09-17T07:30:01.685 INFO:teuthology.orchestra.run.smithi045.stdout:5669 read 0x703bfe4 thru 0x704b688 (0xf6a5 bytes) 2024-09-17T07:30:01.685 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.685 INFO:teuthology.orchestra.run.smithi045.stdout:5672 read 0x2621cf0 thru 0x262248f (0x7a0 bytes) 2024-09-17T07:30:01.685 INFO:teuthology.orchestra.run.smithi045.stdout:5673 writesame 0xa2b8ebf thru 0xa2be412 data_size 0x4(0x5554 bytes) 2024-09-17T07:30:01.688 INFO:teuthology.orchestra.run.smithi045.stdout:5674 writesame 0xa9e2a thru 0xb4fad data_size 0x2a(0xb184 bytes) 2024-09-17T07:30:01.690 INFO:teuthology.orchestra.run.smithi045.stdout:5676 punch from 0xaec30cc to 0xaecc50b, (0x943f bytes) 2024-09-17T07:30:01.690 INFO:teuthology.orchestra.run.smithi045.stdout:5677 read 0x5f269c5 thru 0x5f28fee (0x262a bytes) 2024-09-17T07:30:01.691 INFO:teuthology.orchestra.run.smithi045.stdout:5678 writesame 0x4d1a556 thru 0x4d1dd20 data_size 0x1b(0x37cb bytes) 2024-09-17T07:30:01.692 INFO:teuthology.orchestra.run.smithi045.stdout:5680 write 0x5c42520 thru 0x5c4c1f7 (0x9cd8 bytes) 2024-09-17T07:30:01.694 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.694 INFO:teuthology.orchestra.run.smithi045.stdout:5683 punch from 0x4d2c934 to 0x4d37b3a, (0xb206 bytes) 2024-09-17T07:30:01.694 INFO:teuthology.orchestra.run.smithi045.stdout:5684 read 0xc03ec42 thru 0xc04e5ad (0xf96c bytes) 2024-09-17T07:30:01.702 INFO:teuthology.orchestra.run.smithi045.stdout:5686 trunc from 0xd556f48 to 0x6128c2f 2024-09-17T07:30:01.713 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.722 INFO:teuthology.orchestra.run.smithi045.stdout:5688 writesame 0x838656d thru 0x8386700 data_size 0x2(0x194 bytes) 2024-09-17T07:30:01.734 INFO:teuthology.orchestra.run.smithi045.stdout:5689 write 0xd05b1e9 thru 0xd064aea (0x9902 bytes) 2024-09-17T07:30:01.737 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.737 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:01.737+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:01.739 INFO:teuthology.orchestra.run.smithi045.stdout:5691 trunc from 0xd064aeb to 0xb77e675 2024-09-17T07:30:01.757 INFO:teuthology.orchestra.run.smithi045.stdout:5692 write 0xd29205c thru 0xd29effa (0xcf9f bytes) 2024-09-17T07:30:01.761 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.761 INFO:teuthology.orchestra.run.smithi045.stdout:5695 punch from 0x3bb62c to 0x3bddf7, (0x27cb bytes) 2024-09-17T07:30:01.762 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:01.762 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:01.764 INFO:teuthology.orchestra.run.smithi045.stdout:5696 trunc from 0xd29effb to 0x9fb3a3b 2024-09-17T07:30:01.771 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:01.778 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.778 INFO:teuthology.orchestra.run.smithi045.stdout:5700 writesame 0x3d1d300 thru 0x3d22fd8 data_size 0x3(0x5cd9 bytes) 2024-09-17T07:30:01.791 INFO:teuthology.orchestra.run.smithi045.stdout:5702 trunc from 0x9fb3a3b to 0x21cafe9 2024-09-17T07:30:01.816 INFO:teuthology.orchestra.run.smithi045.stdout:5704 write 0x6d1a023 thru 0x6d27948 (0xd926 bytes) 2024-09-17T07:30:01.828 INFO:teuthology.orchestra.run.smithi045.stdout:5705 write 0x9bd42fb thru 0x9be0073 (0xbd79 bytes) 2024-09-17T07:30:01.832 INFO:teuthology.orchestra.run.smithi045.stdout:5706 write 0x920a581 thru 0x920bfa7 (0x1a27 bytes) 2024-09-17T07:30:01.833 INFO:teuthology.orchestra.run.smithi045.stdout:5707 writesame 0x72b1b88 thru 0x72bcd21 data_size 0x2(0xb19a bytes) 2024-09-17T07:30:01.847 INFO:teuthology.orchestra.run.smithi045.stdout:5708 punch from 0x391ee78 to 0x392896e, (0x9af6 bytes) 2024-09-17T07:30:01.847 INFO:teuthology.orchestra.run.smithi045.stdout:5709 read 0x691b036 thru 0x691fd2d (0x4cf8 bytes) 2024-09-17T07:30:01.847 INFO:teuthology.orchestra.run.smithi045.stdout:5710 punch from 0x10d08d2 to 0x10d2707, (0x1e35 bytes) 2024-09-17T07:30:01.856 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:01.856+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:01.858 INFO:teuthology.orchestra.run.smithi045.stdout:5711 trunc from 0x9be0074 to 0x13777b 2024-09-17T07:30:01.873 INFO:teuthology.orchestra.run.smithi045.stdout:5712 punch from 0xc794c to 0xc8ec6, (0x157a bytes) 2024-09-17T07:30:01.883 INFO:teuthology.orchestra.run.smithi045.stdout:5714 write 0x2389d0d thru 0x2390c83 (0x6f77 bytes) 2024-09-17T07:30:01.886 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.886 INFO:teuthology.orchestra.run.smithi045.stdout:5717 punch from 0x59c8f5 to 0x5a8b13, (0xc21e bytes) 2024-09-17T07:30:01.887 INFO:teuthology.orchestra.run.smithi045.stdout:5718 trunc from 0x2390c84 to 0x22819ae 2024-09-17T07:30:01.895 INFO:teuthology.orchestra.run.smithi045.stdout:5719 read 0x1b67b77 thru 0x1b6cfbb (0x5445 bytes) 2024-09-17T07:30:01.917 INFO:teuthology.orchestra.run.smithi045.stdout:5720 writesame 0xdaa2ae5 thru 0xdab0052 data_size 0x11(0xd56e bytes) 2024-09-17T07:30:01.917 INFO:teuthology.orchestra.run.smithi045.stdout:5721 write 0x678c5f2 thru 0x6798c24 (0xc633 bytes) 2024-09-17T07:30:01.922 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:01.922 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:01.922 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.923 INFO:teuthology.orchestra.run.smithi045.stdout:5725 read 0x542ae57 thru 0x5431626 (0x67d0 bytes) 2024-09-17T07:30:01.926 INFO:teuthology.orchestra.run.smithi045.stdout:5726 trunc from 0xdab0053 to 0xb4360d0 2024-09-17T07:30:01.931 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:01.945 INFO:teuthology.orchestra.run.smithi045.stdout:5727 write 0xe6c5e48 thru 0xe6d22dd (0xc496 bytes) 2024-09-17T07:30:01.949 INFO:teuthology.orchestra.run.smithi045.stdout:5728 read 0x4c599ed thru 0x4c62cc5 (0x92d9 bytes) 2024-09-17T07:30:01.949 INFO:teuthology.orchestra.run.smithi045.stdout:5729 read 0x824142f thru 0x82437d9 (0x23ab bytes) 2024-09-17T07:30:01.950 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.950 INFO:teuthology.orchestra.run.smithi045.stdout:5732 read 0xb05b961 thru 0xb05c65c (0xcfc bytes) 2024-09-17T07:30:01.950 INFO:teuthology.orchestra.run.smithi045.stdout:5733 read 0x28f94a5 thru 0x290151a (0x8076 bytes) 2024-09-17T07:30:01.950 INFO:teuthology.orchestra.run.smithi045.stdout:5734 read 0x9f1ff33 thru 0x9f22e94 (0x2f62 bytes) 2024-09-17T07:30:01.950 INFO:teuthology.orchestra.run.smithi045.stdout:5735 read 0x677852f thru 0x677b897 (0x3369 bytes) 2024-09-17T07:30:01.950 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.950 INFO:teuthology.orchestra.run.smithi045.stdout:5737 read 0x4f26cd1 thru 0x4f2d998 (0x6cc8 bytes) 2024-09-17T07:30:01.951 INFO:teuthology.orchestra.run.smithi045.stdout:5738 write 0x27c349b thru 0x27caaf4 (0x765a bytes) 2024-09-17T07:30:01.952 INFO:teuthology.orchestra.run.smithi045.stdout:5740 write 0x2afd98b thru 0x2b01d77 (0x43ed bytes) 2024-09-17T07:30:01.954 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.954 INFO:teuthology.orchestra.run.smithi045.stdout:5744 read 0xdb464b3 thru 0xdb496d1 (0x321f bytes) 2024-09-17T07:30:01.964 INFO:teuthology.orchestra.run.smithi045.stdout:5745 trunc from 0xe6d22de to 0x4f87a9f 2024-09-17T07:30:01.972 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:01.972+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:01.981 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:01.986 INFO:teuthology.orchestra.run.smithi045.stdout:5750 trunc from 0x4f87a9f to 0xad22dd 2024-09-17T07:30:01.994 INFO:teuthology.orchestra.run.smithi045.stdout:5752 read 0x48ec29 thru 0x498e6f (0xa247 bytes) 2024-09-17T07:30:02.001 INFO:teuthology.orchestra.run.smithi045.stdout:5754 trunc from 0xad22dd to 0x57a73b3 2024-09-17T07:30:02.007 INFO:teuthology.orchestra.run.smithi045.stdout:5755 read 0x571808d thru 0x571f418 (0x738c bytes) 2024-09-17T07:30:02.013 INFO:teuthology.orchestra.run.smithi045.stdout:5756 trunc from 0x57a73b3 to 0xae5d783 2024-09-17T07:30:02.032 INFO:teuthology.orchestra.run.smithi045.stdout:5757 trunc from 0xae5d783 to 0x1acd9 2024-09-17T07:30:02.040 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:02.040+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:02.044 INFO:teuthology.orchestra.run.smithi045.stdout:5758 punch from 0x5871 to 0xd208, (0x7997 bytes) 2024-09-17T07:30:02.061 INFO:teuthology.orchestra.run.smithi045.stdout:5759 write 0x7fc6fe7 thru 0x7fcb067 (0x4081 bytes) 2024-09-17T07:30:02.064 INFO:teuthology.orchestra.run.smithi045.stdout:5762 read 0x7eefdd4 thru 0x7ef040f (0x63c bytes) 2024-09-17T07:30:02.064 INFO:teuthology.orchestra.run.smithi045.stdout:5763 read 0x4196df2 thru 0x41982ac (0x14bb bytes) 2024-09-17T07:30:02.064 INFO:teuthology.orchestra.run.smithi045.stdout:5766 write 0x90f8ab thru 0x91ad9e (0xb4f4 bytes) 2024-09-17T07:30:02.066 INFO:teuthology.orchestra.run.smithi045.stdout:5767 punch from 0x32adc17 to 0x32b1333, (0x371c bytes) 2024-09-17T07:30:02.066 INFO:teuthology.orchestra.run.smithi045.stdout:5768 write 0x508aece thru 0x5095cef (0xae22 bytes) 2024-09-17T07:30:02.081 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:02.081 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:02.082 INFO:teuthology.orchestra.run.smithi045.stdout:5769 write 0xe583859 thru 0xe5862f2 (0x2a9a bytes) 2024-09-17T07:30:02.085 INFO:teuthology.orchestra.run.smithi045.stdout:5770 punch from 0x5552926 to 0x5555740, (0x2e1a bytes) 2024-09-17T07:30:02.086 INFO:teuthology.orchestra.run.smithi045.stdout:5771 read 0xac052e5 thru 0xac131ab (0xdec7 bytes) 2024-09-17T07:30:02.086 INFO:teuthology.orchestra.run.smithi045.stdout:5773 write 0x8a7f8f5 thru 0x8a88960 (0x906c bytes) 2024-09-17T07:30:02.089 INFO:teuthology.orchestra.run.smithi045.stdout:5774 write 0x51a022 thru 0x528837 (0xe816 bytes) 2024-09-17T07:30:02.091 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:02.091 INFO:teuthology.orchestra.run.smithi045.stdout:5776 writesame 0x1913c04 thru 0x191de3a data_size 0x1(0xa237 bytes) 2024-09-17T07:30:02.113 INFO:teuthology.orchestra.run.smithi045.stdout:5777 writesame 0x4cdac4d thru 0x4ce3047 data_size 0x17(0x83fb bytes) 2024-09-17T07:30:02.115 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:02.115 INFO:teuthology.orchestra.run.smithi045.stdout:5779 writesame 0xa1d9021 thru 0xa1e74dd data_size 0x3(0xe4bd bytes) 2024-09-17T07:30:02.127 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:02.127 INFO:teuthology.orchestra.run.smithi045.stdout:5781 punch from 0x98d2e6b to 0x98d3510, (0x6a5 bytes) 2024-09-17T07:30:02.127 INFO:teuthology.orchestra.run.smithi045.stdout:5782 read 0x944d083 thru 0x945b86c (0xe7ea bytes) 2024-09-17T07:30:02.127 INFO:teuthology.orchestra.run.smithi045.stdout:5783 punch from 0x3908e2f to 0x390cc08, (0x3dd9 bytes) 2024-09-17T07:30:02.127 INFO:teuthology.orchestra.run.smithi045.stdout:5784 write 0xfbf22 thru 0x10570f (0x97ee bytes) 2024-09-17T07:30:02.129 INFO:teuthology.orchestra.run.smithi045.stdout:5785 write 0x6c8afc3 thru 0x6c8d313 (0x2351 bytes) 2024-09-17T07:30:02.130 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:02.130 INFO:teuthology.orchestra.run.smithi045.stdout:5788 write 0x219513e thru 0x2196c3b (0x1afe bytes) 2024-09-17T07:30:02.131 INFO:teuthology.orchestra.run.smithi045.stdout:5789 punch from 0x7020c94 to 0x7029b63, (0x8ecf bytes) 2024-09-17T07:30:02.132 INFO:teuthology.orchestra.run.smithi045.stdout:5791 write 0x87b15d5 thru 0x87b339f (0x1dcb bytes) 2024-09-17T07:30:02.141 INFO:teuthology.orchestra.run.smithi045.stdout:5794 write 0xee08d65 thru 0xee0c676 (0x3912 bytes) 2024-09-17T07:30:02.142 INFO:teuthology.orchestra.run.smithi045.stdout:5795 write 0xfddfb4 thru 0xfde571 (0x5be bytes) 2024-09-17T07:30:02.143 INFO:teuthology.orchestra.run.smithi045.stdout:5796 write 0xaa2d273 thru 0xaa3d212 (0xffa0 bytes) 2024-09-17T07:30:02.145 INFO:teuthology.orchestra.run.smithi045.stdout:5797 punch from 0x26af951 to 0x26bd986, (0xe035 bytes) 2024-09-17T07:30:02.145 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:02.145 INFO:teuthology.orchestra.run.smithi045.stdout:5801 read 0xaef979 thru 0xafe41b (0xeaa3 bytes) 2024-09-17T07:30:02.146 INFO:teuthology.orchestra.run.smithi045.stdout:5802 clone 45 order 24 su 262144 sc 6 2024-09-17T07:30:02.174 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:02.174+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:02.235 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:02.235 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:02.242 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:02.242+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:02.243 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:02.343 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:02.342+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:02.387 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:02.387 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:02.395 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:02.444 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:02.444+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:02.499 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:02.499 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:02.508 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:02.548 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:30:02.549 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:02.548+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:02.647 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:02.645+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:02.651 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:02.651 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:02.660 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:02.683 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:02.683 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:02.700 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:02.700 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:02.716 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:02.716 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:02.732 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:02.732 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:02.747 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:02.747+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:02.749 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:02.749 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:02.763 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:02.764 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:02.766 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:02.766 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:02.773 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:02.783 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:02.784 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:02.802 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:02.802 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:02.854 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:02.853+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:02.881 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:02.882 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:02.882 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:02.882 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:02.890 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:02.974 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:02.973+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:03.038 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:03.038 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:03.038 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:03.046 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:03.088 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:03.086+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:03.152 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:03.150+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:03.156 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T07:30:03.201 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [] 2024-09-17T07:30:03.202 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:03.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:03.214 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone44 from 0xee0c677 (overlap 0x1acd9) to 0x55c5 2024-09-17T07:30:03.253 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:03.251+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:03.283 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-17T07:30:03.320 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.320 INFO:teuthology.orchestra.run.smithi045.stdout:5806 write 0xe8569cd thru 0xe860044 (0x9678 bytes) 2024-09-17T07:30:03.329 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:03.329 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:03.329 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:03.329 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:03.330 INFO:teuthology.orchestra.run.smithi045.stdout:5807 punch from 0x27add2d to 0x27b5028, (0x72fb bytes) 2024-09-17T07:30:03.330 INFO:teuthology.orchestra.run.smithi045.stdout:5808 write 0xaba7e8f thru 0xaba8132 (0x2a4 bytes) 2024-09-17T07:30:03.340 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:03.350 INFO:teuthology.orchestra.run.smithi045.stdout:5810 trunc from 0xee0c677 to 0x809df0d 2024-09-17T07:30:03.354 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:03.353+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:03.391 INFO:teuthology.orchestra.run.smithi045.stdout:5811 read 0x1ab6e3 thru 0x1b272d (0x704b bytes) 2024-09-17T07:30:03.403 INFO:teuthology.orchestra.run.smithi045.stdout:5812 writesame 0xc7e6ef6 thru 0xc7ec4f5 data_size 0x2b(0x5600 bytes) 2024-09-17T07:30:03.403 INFO:teuthology.orchestra.run.smithi045.stdout:5813 trunc from 0xc7ec4f6 to 0xbdfcba6 2024-09-17T07:30:03.410 INFO:teuthology.orchestra.run.smithi045.stdout:5814 write 0x309cc18 thru 0x30ac6b6 (0xfa9f bytes) 2024-09-17T07:30:03.429 INFO:teuthology.orchestra.run.smithi045.stdout:5815 write 0xe51b5a3 thru 0xe529cd1 (0xe72f bytes) 2024-09-17T07:30:03.432 INFO:teuthology.orchestra.run.smithi045.stdout:5817 read 0xc0b4c6f thru 0xc0bcff4 (0x8386 bytes) 2024-09-17T07:30:03.432 INFO:teuthology.orchestra.run.smithi045.stdout:5818 punch from 0xcb36fb5 to 0xcb3941d, (0x2468 bytes) 2024-09-17T07:30:03.433 INFO:teuthology.orchestra.run.smithi045.stdout:5819 write 0x443fc3c thru 0x4444378 (0x473d bytes) 2024-09-17T07:30:03.441 INFO:teuthology.orchestra.run.smithi045.stdout:5821 punch from 0x81ddee0 to 0x81e18d7, (0x39f7 bytes) 2024-09-17T07:30:03.441 INFO:teuthology.orchestra.run.smithi045.stdout:5822 read 0x6e4dff4 thru 0x6e5af89 (0xcf96 bytes) 2024-09-17T07:30:03.441 INFO:teuthology.orchestra.run.smithi045.stdout:5823 writesame 0xa33378c thru 0xa34348e data_size 0x13(0xfd03 bytes) 2024-09-17T07:30:03.443 INFO:teuthology.orchestra.run.smithi045.stdout:5824 write 0x16138db thru 0x16191a9 (0x58cf bytes) 2024-09-17T07:30:03.454 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:03.453+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:03.456 INFO:teuthology.orchestra.run.smithi045.stdout:5825 trunc from 0xe529cd2 to 0xd121dc3 2024-09-17T07:30:03.463 INFO:teuthology.orchestra.run.smithi045.stdout:5826 write 0x2a0706f thru 0x2a0dd65 (0x6cf7 bytes) 2024-09-17T07:30:03.475 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:03.475 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:03.475 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:03.476 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.476 INFO:teuthology.orchestra.run.smithi045.stdout:5829 punch from 0x372a3ee to 0x3733b6d, (0x977f bytes) 2024-09-17T07:30:03.476 INFO:teuthology.orchestra.run.smithi045.stdout:5830 read 0xcc975c5 thru 0xcca40f7 (0xcb33 bytes) 2024-09-17T07:30:03.476 INFO:teuthology.orchestra.run.smithi045.stdout:5831 writesame 0x70b2ca3 thru 0x70b9cb6 data_size 0x12(0x7014 bytes) 2024-09-17T07:30:03.477 INFO:teuthology.orchestra.run.smithi045.stdout:5832 read 0x51e4a4d thru 0x51e7cb4 (0x3268 bytes) 2024-09-17T07:30:03.478 INFO:teuthology.orchestra.run.smithi045.stdout:5833 punch from 0x778d87c to 0x77931ca, (0x594e bytes) 2024-09-17T07:30:03.478 INFO:teuthology.orchestra.run.smithi045.stdout:5834 writesame 0xb8d82d2 thru 0xb8e36ac data_size 0x1(0xb3db bytes) 2024-09-17T07:30:03.484 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:03.499 INFO:teuthology.orchestra.run.smithi045.stdout:5835 write 0xba7543d thru 0xba798ad (0x4471 bytes) 2024-09-17T07:30:03.501 INFO:teuthology.orchestra.run.smithi045.stdout:5836 punch from 0x1848736 to 0x18553fb, (0xccc5 bytes) 2024-09-17T07:30:03.501 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.501 INFO:teuthology.orchestra.run.smithi045.stdout:5838 read 0x82da0f5 thru 0x82e5653 (0xb55f bytes) 2024-09-17T07:30:03.501 INFO:teuthology.orchestra.run.smithi045.stdout:5839 writesame 0x4d27bf1 thru 0x4d3485b data_size 0x1(0xcc6b bytes) 2024-09-17T07:30:03.523 INFO:teuthology.orchestra.run.smithi045.stdout:5840 write 0x94682fe thru 0x946d0a2 (0x4da5 bytes) 2024-09-17T07:30:03.525 INFO:teuthology.orchestra.run.smithi045.stdout:5842 read 0x8ef5351 thru 0x8f036a1 (0xe351 bytes) 2024-09-17T07:30:03.525 INFO:teuthology.orchestra.run.smithi045.stdout:5843 write 0x8972347 thru 0x897bcc2 (0x997c bytes) 2024-09-17T07:30:03.527 INFO:teuthology.orchestra.run.smithi045.stdout:5846 punch from 0xb0bab43 to 0xb0c5695, (0xab52 bytes) 2024-09-17T07:30:03.528 INFO:teuthology.orchestra.run.smithi045.stdout:5848 punch from 0x41cbb4f to 0x41cd288, (0x1739 bytes) 2024-09-17T07:30:03.528 INFO:teuthology.orchestra.run.smithi045.stdout:5849 punch from 0x789eb86 to 0x78a1d70, (0x31ea bytes) 2024-09-17T07:30:03.528 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.528 INFO:teuthology.orchestra.run.smithi045.stdout:5852 read 0x1b4da19 thru 0x1b5be0b (0xe3f3 bytes) 2024-09-17T07:30:03.528 INFO:teuthology.orchestra.run.smithi045.stdout:5853 read 0x6f528cb thru 0x6f5cbc1 (0xa2f7 bytes) 2024-09-17T07:30:03.528 INFO:teuthology.orchestra.run.smithi045.stdout:5854 punch from 0x84d29ca to 0x84d444b, (0x1a81 bytes) 2024-09-17T07:30:03.529 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.535 INFO:teuthology.orchestra.run.smithi045.stdout:5856 trunc from 0xd121dc3 to 0x72a9023 2024-09-17T07:30:03.556 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:03.555+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:03.563 INFO:teuthology.orchestra.run.smithi045.stdout:5857 write 0xd433e61 thru 0xd441432 (0xd5d2 bytes) 2024-09-17T07:30:03.575 INFO:teuthology.orchestra.run.smithi045.stdout:5858 trunc from 0xd441433 to 0x824a8d6 2024-09-17T07:30:03.593 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.593 INFO:teuthology.orchestra.run.smithi045.stdout:5862 read 0x4604aef thru 0x46107e3 (0xbcf5 bytes) 2024-09-17T07:30:03.594 INFO:teuthology.orchestra.run.smithi045.stdout:5864 write 0x7def6a4 thru 0x7df0172 (0xacf bytes) 2024-09-17T07:30:03.595 INFO:teuthology.orchestra.run.smithi045.stdout:5865 punch from 0x526a31e to 0x526c901, (0x25e3 bytes) 2024-09-17T07:30:03.595 INFO:teuthology.orchestra.run.smithi045.stdout:5866 read 0xa45a6b thru 0xa4d071 (0x7607 bytes) 2024-09-17T07:30:03.596 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.596 INFO:teuthology.orchestra.run.smithi045.stdout:5869 writesame 0x7d7f609 thru 0x7d874eb data_size 0x1(0x7ee3 bytes) 2024-09-17T07:30:03.615 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.481712485000003, 2024-09-17T07:30:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:30:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:30:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:30:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:30:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:30:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:30:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:30:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:30:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:30:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:30:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:30:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:30:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:30:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:30:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:30:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:30:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:30:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:30:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:30:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:30:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:30:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:30:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:30:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:30:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:30:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.175745781000003, 2024-09-17T07:30:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:30:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:30:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:30:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:30:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:30:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:30:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:30:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:30:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:30:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:30:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:30:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:30:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:30:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:30:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:30:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:30:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.175723666000003, 2024-09-17T07:30:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:30:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:30:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:30:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:30:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:30:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:30:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:30:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:30:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:30:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:30:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:30:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:30:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:30:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.175699561999998, 2024-09-17T07:30:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:30:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:30:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:30:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:30:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:30:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:30:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:30:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:30:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:30:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:30:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:30:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:30:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:30:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.175675472999998, 2024-09-17T07:30:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:30:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:30:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:30:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:30:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:30:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:30:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:30:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:30:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:30:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:30:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:30:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:30:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:30:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.175651053999999, 2024-09-17T07:30:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:30:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:30:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:30:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:30:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:30:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:30:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:30:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:30:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:30:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:30:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:30:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:30:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.175629354000002, 2024-09-17T07:30:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:30:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:30:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:30:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:30:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:30:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:30:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:30:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:30:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:30:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:30:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:30:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:30:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:30:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.175584622000002, 2024-09-17T07:30:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:30:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:30:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:30:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:30:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:30:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:30:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:30:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:30:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:30:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:30:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:30:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:30:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:30:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:30:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.175556155999999, 2024-09-17T07:30:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:30:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:30:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:30:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:30:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:30:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:30:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:30:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:30:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:30:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:30:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:30:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:30:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:30:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.175438904000004, 2024-09-17T07:30:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:30:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:30:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:30:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:30:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:30:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:30:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:30:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:30:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:30:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:30:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:30:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:30:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:30:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:30:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.175012801999998, 2024-09-17T07:30:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:30:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:30:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:30:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:30:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:30:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:30:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:30:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:30:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:30:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:30:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:30:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:30:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:30:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.174993131000001, 2024-09-17T07:30:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:30:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:30:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:30:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:30:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:30:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:30:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:30:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:30:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:30:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:30:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:30:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:30:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:30:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.174970266999999, 2024-09-17T07:30:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:30:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:30:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:30:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:30:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:30:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:30:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:30:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:30:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:30:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:30:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:30:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:30:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:30:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.174948966000002, 2024-09-17T07:30:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:30:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:30:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:30:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:30:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:30:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:30:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:30:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:30:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:30:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:30:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:30:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:30:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:30:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:30:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.174901177999999, 2024-09-17T07:30:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:30:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:30:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:30:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:30:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:30:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:30:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:30:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:30:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:30:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:30:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:30:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:30:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:30:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:30:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.174873955999999, 2024-09-17T07:30:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:30:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:30:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:30:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:30:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:30:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:30:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:30:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:30:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:30:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:30:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:30:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:30:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:30:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.174853569, 2024-09-17T07:30:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:30:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:30:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:30:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:30:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:30:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:30:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:30:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:30:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:30:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:30:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:30:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:30:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:30:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:30:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.174833864, 2024-09-17T07:30:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:30:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:30:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:30:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:30:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:30:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:30:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:30:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:30:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:30:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:30:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:30:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:30:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:30:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:30:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.174810786000002, 2024-09-17T07:30:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:30:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:30:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:30:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:30:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:30:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:30:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:30:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:30:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:30:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:30:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.698 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:30:03.698 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:30:03.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.698 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.698 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.698 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:30:03.698 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:30:03.698 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.698 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.698 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.698 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.698 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:03.698 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:03.699 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.174790223999999, 2024-09-17T07:30:03.699 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:30:03.699 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:03.699 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:03.699 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:03.699 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:03.699 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:03.699 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:03.699 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:30:03.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.700 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:03.700 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.700 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:03.700 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:03.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.701 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:03.701 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:03.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:03.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.701 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:03.701 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:03.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:30:03.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.702 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:03.702 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:30:03.702 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:30:03.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.702 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:03.702 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:30:03.702 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:30:03.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.702 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:03.703 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:30:03.703 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:30:03.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.703 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:03.703 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:30:03.703 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:30:03.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:03.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:03.703 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:03.703 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:30:03.703 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:30:03.704 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.704 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.704 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.704 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:03.704 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:03.704 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:03.705 INFO:teuthology.orchestra.run.smithi045.stdout:5871 writesame 0x8dc4a91 thru 0x8dd494a data_size 0x5(0xfeba bytes) 2024-09-17T07:30:03.705 INFO:teuthology.orchestra.run.smithi045.stdout:5872 read 0x5b78b36 thru 0x5b7f9b2 (0x6e7d bytes) 2024-09-17T07:30:03.706 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:03.708 INFO:teuthology.orchestra.run.smithi045.stdout:5874 write 0x9f9ccf2 thru 0x9f9f0ba (0x23c9 bytes) 2024-09-17T07:30:03.708 INFO:teuthology.orchestra.run.smithi045.stdout:5875 trunc from 0x9f9f0bb to 0xaf9b17e 2024-09-17T07:30:03.708 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.708 INFO:teuthology.orchestra.run.smithi045.stdout:5877 punch from 0x847e204 to 0x8480ba6, (0x29a2 bytes) 2024-09-17T07:30:03.708 INFO:teuthology.orchestra.run.smithi045.stdout:5878 read 0xa265b67 thru 0xa26e89a (0x8d34 bytes) 2024-09-17T07:30:03.708 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.708 INFO:teuthology.orchestra.run.smithi045.stdout:5880 punch from 0x3f695fb to 0x3f74e40, (0xb845 bytes) 2024-09-17T07:30:03.708 INFO:teuthology.orchestra.run.smithi045.stdout:5881 read 0x3dbdf0b thru 0x3dcb96e (0xda64 bytes) 2024-09-17T07:30:03.708 INFO:teuthology.orchestra.run.smithi045.stdout:5883 write 0x2aa01ba thru 0x2aaf3ec (0xf233 bytes) 2024-09-17T07:30:03.708 INFO:teuthology.orchestra.run.smithi045.stdout:5884 write 0x2a4de10 thru 0x2a5ca72 (0xec63 bytes) 2024-09-17T07:30:03.708 INFO:teuthology.orchestra.run.smithi045.stdout:5885 write 0xdc7fa36 thru 0xdc8c1e6 (0xc7b1 bytes) 2024-09-17T07:30:03.708 INFO:teuthology.orchestra.run.smithi045.stdout:5886 trunc from 0xdc8c1e7 to 0x647da29 2024-09-17T07:30:03.709 INFO:teuthology.orchestra.run.smithi045.stdout:5887 writesame 0x4f37ad4 thru 0x4f40b0c data_size 0x3(0x9039 bytes) 2024-09-17T07:30:03.709 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.709 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.709 INFO:teuthology.orchestra.run.smithi045.stdout:5891 write 0x1eae7ce thru 0x1eb6b7a (0x83ad bytes) 2024-09-17T07:30:03.709 INFO:teuthology.orchestra.run.smithi045.stdout:5893 trunc from 0x647da29 to 0x1227caf 2024-09-17T07:30:03.709 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:03.704+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:03.709 INFO:teuthology.orchestra.run.smithi045.stdout:5894 read 0x62785c thru 0x6354a2 (0xdc47 bytes) 2024-09-17T07:30:03.729 INFO:teuthology.orchestra.run.smithi045.stdout:5895 writesame 0xe6c7cc1 thru 0xe6cbfc0 data_size 0x43(0x4300 bytes) 2024-09-17T07:30:03.730 INFO:teuthology.orchestra.run.smithi045.stdout:5897 punch from 0x4f4017 to 0x4fa5ce, (0x65b7 bytes) 2024-09-17T07:30:03.730 INFO:teuthology.orchestra.run.smithi045.stdout:5898 writesame 0x6939e8a thru 0x6949b18 data_size 0x5(0xfc8f bytes) 2024-09-17T07:30:03.742 INFO:teuthology.orchestra.run.smithi045.stdout:5899 writesame 0xe714572 thru 0xe71b455 data_size 0x4(0x6ee4 bytes) 2024-09-17T07:30:03.742 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.742 INFO:teuthology.orchestra.run.smithi045.stdout:5901 read 0x3d64159 thru 0x3d64735 (0x5dd bytes) 2024-09-17T07:30:03.742 INFO:teuthology.orchestra.run.smithi045.stdout:5902 writesame 0x9d9b379 thru 0x9d9bbe5 data_size 0x1(0x86d bytes) 2024-09-17T07:30:03.744 INFO:teuthology.orchestra.run.smithi045.stdout:5903 write 0xb6bc590 thru 0xb6c69dc (0xa44d bytes) 2024-09-17T07:30:03.748 INFO:teuthology.orchestra.run.smithi045.stdout:5904 writesame 0xcb771c8 thru 0xcb7e6fe data_size 0x1(0x7537 bytes) 2024-09-17T07:30:03.765 INFO:teuthology.orchestra.run.smithi045.stdout:5906 write 0x86581b8 thru 0x866658f (0xe3d8 bytes) 2024-09-17T07:30:03.769 INFO:teuthology.orchestra.run.smithi045.stdout:5909 punch from 0x1a291ac to 0x1a33314, (0xa168 bytes) 2024-09-17T07:30:03.769 INFO:teuthology.orchestra.run.smithi045.stdout:5910 read 0xd3f09c8 thru 0xd3f7801 (0x6e3a bytes) 2024-09-17T07:30:03.769 INFO:teuthology.orchestra.run.smithi045.stdout:5911 read 0xbe2527e thru 0xbe29b55 (0x48d8 bytes) 2024-09-17T07:30:03.769 INFO:teuthology.orchestra.run.smithi045.stdout:5912 read 0x89f6390 thru 0x89fd81e (0x748f bytes) 2024-09-17T07:30:03.770 INFO:teuthology.orchestra.run.smithi045.stdout:5913 write 0x4a243b9 thru 0x4a299ee (0x5636 bytes) 2024-09-17T07:30:03.771 INFO:teuthology.orchestra.run.smithi045.stdout:5914 writesame 0x22b2fab thru 0x22b56c0 data_size 0x1(0x2716 bytes) 2024-09-17T07:30:03.776 INFO:teuthology.orchestra.run.smithi045.stdout:5916 read 0x42a8bb6 thru 0x42b7d12 (0xf15d bytes) 2024-09-17T07:30:03.777 INFO:teuthology.orchestra.run.smithi045.stdout:5917 writesame 0xe4b048c thru 0xe4b343c data_size 0x1(0x2fb1 bytes) 2024-09-17T07:30:03.785 INFO:teuthology.orchestra.run.smithi045.stdout:5918 read 0x9dac323 thru 0x9db54f6 (0x91d4 bytes) 2024-09-17T07:30:03.798 INFO:teuthology.orchestra.run.smithi045.stdout:5919 trunc from 0xe71b456 to 0x29d3d50 2024-09-17T07:30:03.806 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:03.804+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:03.828 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:03.828 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:03.828 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:03.828 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:03.829 INFO:teuthology.orchestra.run.smithi045.stdout:5920 write 0x740c543 thru 0x7416ff2 (0xaab0 bytes) 2024-09-17T07:30:03.838 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:03.846 INFO:teuthology.orchestra.run.smithi045.stdout:5926 writesame 0xe802d5f thru 0xe807289 data_size 0x1(0x452b bytes) 2024-09-17T07:30:03.846 INFO:teuthology.orchestra.run.smithi045.stdout:5927 writesame 0xdcc281f thru 0xdccad77 data_size 0x3(0x8559 bytes) 2024-09-17T07:30:03.868 INFO:teuthology.orchestra.run.smithi045.stdout:5928 trunc from 0xe80728a to 0x27cc1cf 2024-09-17T07:30:03.893 INFO:teuthology.orchestra.run.smithi045.stdout:5929 trunc from 0x27cc1cf to 0xde27546 2024-09-17T07:30:03.899 INFO:teuthology.orchestra.run.smithi045.stdout:5930 write 0x91226aa thru 0x912c02a (0x9981 bytes) 2024-09-17T07:30:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:5931 punch from 0x224e11f to 0x2253997, (0x5878 bytes) 2024-09-17T07:30:03.906 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:03.906+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:03.911 INFO:teuthology.orchestra.run.smithi045.stdout:5933 trunc from 0xde27546 to 0x669ae6a 2024-09-17T07:30:03.928 INFO:teuthology.orchestra.run.smithi045.stdout:5935 write 0x984d4ea thru 0x985c750 (0xf267 bytes) 2024-09-17T07:30:03.932 INFO:teuthology.orchestra.run.smithi045.stdout:5936 write 0x6430126 thru 0x6430b9c (0xa77 bytes) 2024-09-17T07:30:03.933 INFO:teuthology.orchestra.run.smithi045.stdout:5937 writesame 0x6852a46 thru 0x6858bb5 data_size 0x8(0x6170 bytes) 2024-09-17T07:30:03.935 INFO:teuthology.orchestra.run.smithi045.stdout:5938 read 0x2f60450 thru 0x2f66c4b (0x67fc bytes) 2024-09-17T07:30:03.936 INFO:teuthology.orchestra.run.smithi045.stdout:5939 writesame 0x58964ae thru 0x58a462a data_size 0x5(0xe17d bytes) 2024-09-17T07:30:03.945 INFO:teuthology.orchestra.run.smithi045.stdout:5941 trunc from 0x985c751 to 0x65f547d 2024-09-17T07:30:03.952 INFO:teuthology.orchestra.run.smithi045.stdout:5942 read 0x41a04da thru 0x41a789e (0x73c5 bytes) 2024-09-17T07:30:03.957 INFO:teuthology.orchestra.run.smithi045.stdout:5943 writesame 0x66e6ad4 thru 0x66ece1f data_size 0x3e(0x634c bytes) 2024-09-17T07:30:03.959 INFO:teuthology.orchestra.run.smithi045.stdout:5944 punch from 0x4986020 to 0x4994975, (0xe955 bytes) 2024-09-17T07:30:03.964 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:03.965 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:03.965 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:03.965 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:03.965 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:03.970 INFO:teuthology.orchestra.run.smithi045.stdout:5945 writesame 0xbde3851 thru 0xbdf1fef data_size 0xb1(0xe79f bytes) 2024-09-17T07:30:03.972 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.972 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.972 INFO:teuthology.orchestra.run.smithi045.stdout:5949 write 0x8cc6234 thru 0x8ccf238 (0x9005 bytes) 2024-09-17T07:30:03.974 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:03.975 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.976 INFO:teuthology.orchestra.run.smithi045.stdout:5951 write 0x7d1c89c thru 0x7d2894e (0xc0b3 bytes) 2024-09-17T07:30:03.978 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:03.985 INFO:teuthology.orchestra.run.smithi045.stdout:5953 writesame 0xddaa840 thru 0xddacf89 data_size 0x2f(0x274a bytes) 2024-09-17T07:30:03.985 INFO:teuthology.orchestra.run.smithi045.stdout:5956 punch from 0x1bbb605 to 0x1bc839e, (0xcd99 bytes) 2024-09-17T07:30:03.985 INFO:teuthology.orchestra.run.smithi045.stdout:5959 write 0x7557f57 thru 0x755e69c (0x6746 bytes) 2024-09-17T07:30:03.993 INFO:teuthology.orchestra.run.smithi045.stdout:5960 trunc from 0xddacf8a to 0x94ca0f8 2024-09-17T07:30:04.000 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:04.000 INFO:teuthology.orchestra.run.smithi045.stdout:5963 write 0x79b2cd1 thru 0x79c2800 (0xfb30 bytes) 2024-09-17T07:30:04.004 INFO:teuthology.orchestra.run.smithi045.stdout:5964 clone 46 order 24 su 8388608 sc 15 2024-09-17T07:30:04.007 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:04.007+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:04.087 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.952036055999997, 2024-09-17T07:30:04.087 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:30:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:30:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:04.088 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:30:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:30:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.089 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:30:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:30:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:30:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:30:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:30:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:30:04.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:30:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:30:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:30:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:30:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:30:04.091 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:30:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:30:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:30:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:30:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:30:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.092 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:30:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:30:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:30:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:30:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.093 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951999158999996, 2024-09-17T07:30:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:30:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:30:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:30:04.095 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:30:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:30:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:30:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:30:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:30:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.096 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:30:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:30:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:30:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:30:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:30:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.097 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:30:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:30:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:30:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:30:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:30:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:30:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:30:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:30:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:30:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:30:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.099 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951973191999997, 2024-09-17T07:30:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:30:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:30:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.100 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:30:04.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:30:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:30:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:30:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:30:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:30:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:30:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:30:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:30:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:30:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:30:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:30:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:30:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:30:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:30:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.104 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:30:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:30:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:30:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:30:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:30:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:30:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.105 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951948596999998, 2024-09-17T07:30:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:30:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.106 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:30:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.107 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:30:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:30:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:30:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:30:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:30:04.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:30:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:30:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:30:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:30:04.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.112 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:30:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:30:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:30:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:30:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:30:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:30:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.113 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:30:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:30:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:30:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:30:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:30:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:30:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.114 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951925110000005, 2024-09-17T07:30:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:30:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.115 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:30:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.116 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:30:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:30:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:30:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:30:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:30:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:30:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:30:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:30:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:30:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:30:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:30:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:30:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:30:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:30:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:30:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:30:04.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:30:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:30:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:30:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:30:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951911371999998, 2024-09-17T07:30:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:30:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:30:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.121 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:04.122 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:30:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:30:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:30:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:30:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:30:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:30:04.123 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:30:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:30:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:30:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:30:04.124 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:30:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:30:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:30:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:30:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:30:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.125 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:30:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:30:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:30:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:30:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951881182999998, 2024-09-17T07:30:04.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:30:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:30:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:30:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:30:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:30:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:30:04.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:30:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:30:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:30:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:30:04.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:30:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:30:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:30:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:30:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:30:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.130 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:30:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:30:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:30:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:30:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:30:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:30:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:30:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951844678, 2024-09-17T07:30:04.132 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:30:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:30:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:30:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:30:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:30:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:30:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:30:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.135 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:30:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:30:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:30:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:30:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:04.136 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:30:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:30:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:30:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:30:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:30:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.137 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:30:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:30:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:30:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:30:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:30:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:30:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.138 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951816918999995, 2024-09-17T07:30:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:30:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:30:04.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.140 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:30:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:30:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:30:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:30:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:30:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:30:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:30:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:30:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:30:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:30:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:30:04.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.143 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.143 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:30:04.143 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:30:04.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.143 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.143 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:30:04.143 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:04.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.143 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.143 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:30:04.143 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:30:04.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.144 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.144 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.144 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:30:04.144 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:30:04.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.144 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.144 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.144 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:30:04.144 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:30:04.144 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.144 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.144 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.144 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.145 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.145 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:04.145 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951794758999995, 2024-09-17T07:30:04.145 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:30:04.145 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.145 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.145 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.145 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.145 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.145 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.145 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:30:04.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.145 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.145 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.146 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.146 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:04.146 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.146 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.146 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:04.146 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.146 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.146 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:04.146 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:30:04.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.147 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.147 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.147 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:30:04.147 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:30:04.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.147 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.147 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.147 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:30:04.147 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:04.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.147 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.147 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.147 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:30:04.147 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:30:04.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.148 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.148 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.148 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:30:04.148 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:30:04.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.148 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.148 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.148 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:30:04.148 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:30:04.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.148 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.148 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.148 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:30:04.148 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:30:04.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.149 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.149 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:30:04.149 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:30:04.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.149 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.149 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:30:04.149 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:30:04.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.149 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.149 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:30:04.149 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:30:04.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.150 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.150 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:30:04.150 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:30:04.150 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.150 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.150 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.150 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.150 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:04.150 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951781597999997, 2024-09-17T07:30:04.150 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:30:04.151 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.151 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.151 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.151 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.151 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.151 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.151 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:30:04.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.151 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.151 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.151 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:04.151 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.152 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.152 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:04.152 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.152 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.152 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:04.152 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:30:04.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.152 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.152 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:30:04.152 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:30:04.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.153 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.153 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:30:04.153 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:30:04.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.153 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.153 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:30:04.153 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:30:04.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.153 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.154 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:30:04.154 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:30:04.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.154 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.154 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:04.154 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:30:04.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.154 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.154 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:30:04.154 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:30:04.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.155 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.155 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:30:04.155 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:30:04.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.155 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.155 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:30:04.155 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:30:04.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.155 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.155 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:30:04.155 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:30:04.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:30:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:30:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951763009999993, 2024-09-17T07:30:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:30:04.157 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.157 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.157 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.157 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.157 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.157 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:30:04.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.157 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.157 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.157 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:04.157 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.158 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.158 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:04.158 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.158 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.158 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:30:04.158 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:30:04.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:30:04.159 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:30:04.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:04.159 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:30:04.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:04.159 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:30:04.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.160 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:04.160 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:30:04.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.160 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:04.160 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:30:04.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.160 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:30:04.161 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:30:04.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.161 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.161 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:30:04.161 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:30:04.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.161 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.161 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:30:04.161 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:30:04.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.161 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.162 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.162 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:30:04.162 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:30:04.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.162 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.162 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:30:04.162 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:30:04.162 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.162 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.162 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.162 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.162 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:04.163 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951730198999996, 2024-09-17T07:30:04.163 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:30:04.163 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.163 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.163 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.163 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.163 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.163 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.163 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:30:04.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.163 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.163 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.163 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:04.163 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:04.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:04.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:30:04.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:30:04.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:30:04.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.165 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.165 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:30:04.165 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:30:04.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.165 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.165 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:30:04.165 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:30:04.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.165 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.165 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:30:04.166 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:30:04.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.166 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.166 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:04.166 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:04.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.166 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.166 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:30:04.166 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:30:04.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.166 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.166 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:30:04.167 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:04.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.167 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.167 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:30:04.167 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:30:04.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.167 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.167 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:30:04.167 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:30:04.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.167 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.168 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:30:04.168 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:30:04.168 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.168 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.168 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.168 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.168 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:04.168 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951711141999994, 2024-09-17T07:30:04.168 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:30:04.168 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.168 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.168 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.169 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.169 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.169 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.169 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:30:04.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.169 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.169 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.169 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:04.169 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.169 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.169 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:04.169 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.170 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.170 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:30:04.170 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:30:04.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.170 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.170 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:30:04.170 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:30:04.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.170 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.170 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:30:04.170 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:30:04.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.171 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.171 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:30:04.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:30:04.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.171 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.171 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:30:04.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:30:04.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.171 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.171 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:30:04.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:30:04.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.172 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:30:04.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:30:04.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.172 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:30:04.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:30:04.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.173 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.173 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:30:04.173 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:30:04.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.173 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.173 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:30:04.173 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:30:04.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.173 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.173 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:30:04.173 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:30:04.173 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.174 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.174 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.174 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:04.174 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951689032999994, 2024-09-17T07:30:04.174 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:30:04.174 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.174 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.174 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.174 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.174 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.175 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.175 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:30:04.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.175 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.175 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.175 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:04.175 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.175 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.175 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:04.175 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.176 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.176 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:30:04.176 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:30:04.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.176 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.176 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:04.176 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:30:04.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.177 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.177 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:30:04.177 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:30:04.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.177 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.177 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:30:04.177 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:04.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.177 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.177 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:30:04.177 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:30:04.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.178 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.178 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:30:04.178 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:30:04.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.178 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.178 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:30:04.178 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:30:04.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.178 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.178 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:30:04.179 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:30:04.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.179 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.179 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:30:04.179 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:30:04.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.179 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.179 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:30:04.179 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:30:04.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.180 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.180 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:30:04.180 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:30:04.180 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.180 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.180 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.180 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.180 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:04.180 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951662925999997, 2024-09-17T07:30:04.180 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:30:04.181 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.181 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.181 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.181 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.181 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.181 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.181 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:30:04.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.181 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.181 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.181 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:04.181 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.182 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.182 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:04.182 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.182 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.182 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:30:04.182 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:30:04.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.182 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.182 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:04.182 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:30:04.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.183 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:30:04.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:04.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.183 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:30:04.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:30:04.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.183 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:30:04.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:30:04.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.184 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.184 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:30:04.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:30:04.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.184 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.184 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:30:04.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:30:04.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.184 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.185 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:30:04.185 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:30:04.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.185 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.185 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:30:04.185 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:30:04.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.185 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.185 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:30:04.185 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:30:04.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.186 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.186 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:30:04.186 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:30:04.186 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.186 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.186 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.186 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.186 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:04.186 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951633780999998, 2024-09-17T07:30:04.186 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:30:04.186 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.186 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.187 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.187 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.187 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.187 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:30:04.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.187 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.187 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.187 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:04.187 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.187 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:04.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:04.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:30:04.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:30:04.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:04.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.189 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.189 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:30:04.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:30:04.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.189 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.189 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:30:04.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:30:04.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.189 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.189 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:30:04.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:30:04.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.190 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.190 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:30:04.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:30:04.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.190 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.190 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:30:04.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:30:04.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.190 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.191 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:30:04.191 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:30:04.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.191 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.191 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:30:04.191 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:30:04.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.191 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.191 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:30:04.191 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:30:04.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.191 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.192 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:30:04.192 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:30:04.192 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.192 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.192 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.192 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.192 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:04.192 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951601977999999, 2024-09-17T07:30:04.192 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:30:04.192 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.193 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.193 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.193 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.193 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.193 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:30:04.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.193 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.193 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.193 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:04.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.194 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.194 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:04.194 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.194 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.194 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:30:04.194 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:30:04.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.194 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.194 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:04.194 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:30:04.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.195 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.195 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:30:04.195 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:30:04.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.195 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.195 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:30:04.195 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:30:04.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.195 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.195 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:30:04.195 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:30:04.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.196 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:30:04.196 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:30:04.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.196 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:30:04.196 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:30:04.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.196 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:30:04.196 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:30:04.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.197 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.197 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:30:04.197 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:30:04.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.197 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.197 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:30:04.197 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:30:04.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.197 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.197 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:30:04.197 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:30:04.197 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.198 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.198 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.198 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.198 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:04.198 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951575031000004, 2024-09-17T07:30:04.198 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:30:04.198 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.198 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.198 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.198 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.198 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.199 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.199 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:30:04.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.199 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.199 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.199 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:04.199 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.199 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.199 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:04.199 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.200 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:30:04.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:30:04.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.200 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:30:04.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:30:04.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.200 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:04.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:30:04.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:04.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:30:04.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:30:04.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:30:04.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:30:04.202 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:30:04.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.202 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.202 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:30:04.202 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:30:04.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.202 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.202 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:30:04.202 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:30:04.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.203 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.203 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:30:04.203 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:30:04.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.203 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.203 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:30:04.203 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:30:04.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.203 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.203 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:30:04.203 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:30:04.203 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.204 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.204 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.204 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:04.204 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:04.204 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.951534081999995, 2024-09-17T07:30:04.204 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:30:04.204 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.204 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.204 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.204 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:04.204 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:04.204 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:30:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:30:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:30:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:30:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:30:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:30:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:30:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.207 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.207 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:30:04.207 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:30:04.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.207 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.207 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:30:04.207 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:30:04.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.207 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:04.207 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:30:04.207 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:30:04.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.208 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.208 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:30:04.208 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:30:04.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.208 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:04.208 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:30:04.208 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:30:04.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.208 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.208 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:30:04.208 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:30:04.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.209 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.209 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:30:04.209 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:30:04.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.209 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.209 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:30:04.209 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:30:04.209 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.209 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.209 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.209 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.210 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.210 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:04.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:04.212 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-17T07:30:04.212 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:04.209+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:04.229 INFO:teuthology.orchestra.run.smithi045.stdout:truncating image image_client.0-clone45 from 0x94ca0f8 (overlap 0x1227caf) to 0x1057037 2024-09-17T07:30:04.311 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:04.311+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:04.325 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-17T07:30:04.327 INFO:teuthology.orchestra.run.smithi045.stdout:5965 writesame 0x692eb89 thru 0x693cff8 data_size 0xac(0xe470 bytes) 2024-09-17T07:30:04.335 INFO:teuthology.orchestra.run.smithi045.stdout:5966 trunc from 0x94ca0f8 to 0x9993816 2024-09-17T07:30:04.341 INFO:teuthology.orchestra.run.smithi045.stdout:5967 writesame 0x6e15fd3 thru 0x6e18929 data_size 0x1(0x2957 bytes) 2024-09-17T07:30:04.344 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:04.344 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:04.345 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:04.345 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:04.348 INFO:teuthology.orchestra.run.smithi045.stdout:5968 punch from 0x211aa29 to 0x21280e0, (0xd6b7 bytes) 2024-09-17T07:30:04.349 INFO:teuthology.orchestra.run.smithi045.stdout:5970 trunc from 0x9993816 to 0xa974b7f 2024-09-17T07:30:04.355 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:04.355 INFO:teuthology.orchestra.run.smithi045.stdout:5971 read 0x709c3a7 thru 0x70a2c26 (0x6880 bytes) 2024-09-17T07:30:04.355 INFO:teuthology.orchestra.run.smithi045.stdout:5973 write 0x82dee78 thru 0x82e6493 (0x761c bytes) 2024-09-17T07:30:04.360 INFO:teuthology.orchestra.run.smithi045.stdout:5974 read 0x713bf9d thru 0x714b85a (0xf8be bytes) 2024-09-17T07:30:04.360 INFO:teuthology.orchestra.run.smithi045.stdout:5975 read 0x6e0c606 thru 0x6e180b5 (0xbab0 bytes) 2024-09-17T07:30:04.361 INFO:teuthology.orchestra.run.smithi045.stdout:5976 writesame 0x5730827 thru 0x5733326 data_size 0x10(0x2b00 bytes) 2024-09-17T07:30:04.365 INFO:teuthology.orchestra.run.smithi045.stdout:5978 writesame 0x930bac3 thru 0x93192f0 data_size 0xe(0xd82e bytes) 2024-09-17T07:30:04.372 INFO:teuthology.orchestra.run.smithi045.stdout:5979 trunc from 0xa974b7f to 0xc4507bf 2024-09-17T07:30:04.388 INFO:teuthology.orchestra.run.smithi045.stdout:5980 write 0xe426ac5 thru 0xe431779 (0xacb5 bytes) 2024-09-17T07:30:04.399 INFO:teuthology.orchestra.run.smithi045.stdout:5981 trunc from 0xe43177a to 0x5d2717c 2024-09-17T07:30:04.412 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:04.410+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:04.434 INFO:teuthology.orchestra.run.smithi045.stdout:5982 write 0xad1329c thru 0xad1c21b (0x8f80 bytes) 2024-09-17T07:30:04.436 INFO:teuthology.orchestra.run.smithi045.stdout:5983 writesame 0x7f07720 thru 0x7f14573 data_size 0x8b(0xce54 bytes) 2024-09-17T07:30:04.438 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:04.445 INFO:teuthology.orchestra.run.smithi045.stdout:5985 write 0xbfc22c5 thru 0xbfc9d95 (0x7ad1 bytes) 2024-09-17T07:30:04.447 INFO:teuthology.orchestra.run.smithi045.stdout:5986 write 0x3327b86 thru 0x333796d (0xfde8 bytes) 2024-09-17T07:30:04.448 INFO:teuthology.orchestra.run.smithi045.stdout:5987 read 0xbe493ba thru 0xbe57b99 (0xe7e0 bytes) 2024-09-17T07:30:04.448 INFO:teuthology.orchestra.run.smithi045.stdout:5988 read 0xacdf75b thru 0xaced013 (0xd8b9 bytes) 2024-09-17T07:30:04.449 INFO:teuthology.orchestra.run.smithi045.stdout:5989 write 0x24da935 thru 0x24e2001 (0x76cd bytes) 2024-09-17T07:30:04.450 INFO:teuthology.orchestra.run.smithi045.stdout:5990 read 0x6b440df thru 0x6b45b12 (0x1a34 bytes) 2024-09-17T07:30:04.450 INFO:teuthology.orchestra.run.smithi045.stdout:5991 read 0xe24d82 thru 0xe25205 (0x484 bytes) 2024-09-17T07:30:04.452 INFO:teuthology.orchestra.run.smithi045.stdout:5992 trunc from 0xbfc9d96 to 0xd895e1e 2024-09-17T07:30:04.467 INFO:teuthology.orchestra.run.smithi045.stdout:5993 trunc from 0xd895e1e to 0x5327768 2024-09-17T07:30:04.475 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:04.475 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:04.475 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:04.475 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:04.476 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:04.480 INFO:teuthology.orchestra.run.smithi045.stdout:compare and write disabled 2024-09-17T07:30:04.481 INFO:teuthology.orchestra.run.smithi045.stdout:5995 writesame 0x1c54819 thru 0x1c5f8f8 data_size 0x50(0xb0e0 bytes) 2024-09-17T07:30:04.482 INFO:teuthology.orchestra.run.smithi045.stdout:5996 read 0x138e7d0 thru 0x1397f39 (0x976a bytes) 2024-09-17T07:30:04.485 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:04.486 INFO:teuthology.orchestra.run.smithi045.stdout:5997 trunc from 0x5327768 to 0x1f8f994 2024-09-17T07:30:04.510 INFO:teuthology.orchestra.run.smithi045.stdout:5999 read 0x26d36d thru 0x26d810 (0x4a4 bytes) 2024-09-17T07:30:04.513 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:04.511+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:04.546 INFO:teuthology.orchestra.run.smithi045.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-17T07:30:04.598 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:04.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17069 3.0 e41/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 41'110, mlcod=40'108)", 2024-09-17T07:30:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 61.857407015, 2024-09-17T07:30:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011408908000000001, 2024-09-17T07:30:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739753+0000", 2024-09-17T07:30:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:30:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743532+0000", 2024-09-17T07:30:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037794510000000001 2024-09-17T07:30:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743557+0000", 2024-09-17T07:30:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5253e-05 2024-09-17T07:30:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743569+0000", 2024-09-17T07:30:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1161999999999999e-05 2024-09-17T07:30:04.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743650+0000", 2024-09-17T07:30:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1087999999999993e-05 2024-09-17T07:30:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743674+0000", 2024-09-17T07:30:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4562e-05 2024-09-17T07:30:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751095+0000", 2024-09-17T07:30:04.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074211140000000004 2024-09-17T07:30:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751161+0000", 2024-09-17T07:30:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5707e-05 2024-09-17T07:30:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17240 3.0 e42/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 42'113, mlcod=41'110)", 2024-09-17T07:30:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 61.371748531000001, 2024-09-17T07:30:04.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064392440000000002, 2024-09-17T07:30:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:04.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:30:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225418+0000", 2024-09-17T07:30:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8360000000000003e-06 2024-09-17T07:30:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225426+0000", 2024-09-17T07:30:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2670000000000006e-06 2024-09-17T07:30:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225434+0000", 2024-09-17T07:30:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0560000000000005e-06 2024-09-17T07:30:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225514+0000", 2024-09-17T07:30:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9979000000000001e-05 2024-09-17T07:30:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225535+0000", 2024-09-17T07:30:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0965e-05 2024-09-17T07:30:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231797+0000", 2024-09-17T07:30:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062621129999999997 2024-09-17T07:30:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231850+0000", 2024-09-17T07:30:04.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2611999999999997e-05 2024-09-17T07:30:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20054 3.9 e44/23 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:head v 44'131, mlcod=44'129)", 2024-09-17T07:30:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:30:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 59.329597415000002, 2024-09-17T07:30:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028510570000000002, 2024-09-17T07:30:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.608 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:30:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:30:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267563+0000", 2024-09-17T07:30:04.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.584e-06 2024-09-17T07:30:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268269+0000", 2024-09-17T07:30:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00070567399999999999 2024-09-17T07:30:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268287+0000", 2024-09-17T07:30:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.84e-05 2024-09-17T07:30:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268297+0000", 2024-09-17T07:30:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9059999999999997e-06 2024-09-17T07:30:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268368+0000", 2024-09-17T07:30:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0562999999999995e-05 2024-09-17T07:30:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268393+0000", 2024-09-17T07:30:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4958999999999999e-05 2024-09-17T07:30:04.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270330+0000", 2024-09-17T07:30:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001936644 2024-09-17T07:30:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270413+0000", 2024-09-17T07:30:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3326999999999997e-05 2024-09-17T07:30:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:32543 3.0 e55/23 3:03bfd97d:::rbd_data.11839a97b753.0000000000000007:head v 55'150, mlcod=54'148)", 2024-09-17T07:30:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 47.142522278000001, 2024-09-17T07:30:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031813480000000001, 2024-09-17T07:30:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5199999999999998e-07 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455083+0000", 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044629199999999999 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455098+0000", 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4183e-05 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455106+0000", 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455173+0000", 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6346000000000005e-05 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455200+0000", 2024-09-17T07:30:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7319000000000001e-05 2024-09-17T07:30:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457774+0000", 2024-09-17T07:30:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025742650000000001 2024-09-17T07:30:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457818+0000", 2024-09-17T07:30:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3891999999999999e-05 2024-09-17T07:30:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33735 3.1f e55/52 3:fea54097:::rbd_data.11839a97b753.0000000000000000:head v 55'260, mlcod=55'258)", 2024-09-17T07:30:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.77258965, 2024-09-17T07:30:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035215200000000002, 2024-09-17T07:30:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824571+0000", 2024-09-17T07:30:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.164e-06 2024-09-17T07:30:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825118+0000", 2024-09-17T07:30:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054696400000000004 2024-09-17T07:30:04.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825134+0000", 2024-09-17T07:30:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6342999999999999e-05 2024-09-17T07:30:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825142+0000", 2024-09-17T07:30:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5270000000000001e-06 2024-09-17T07:30:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825200+0000", 2024-09-17T07:30:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.893e-05 2024-09-17T07:30:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825218+0000", 2024-09-17T07:30:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7935999999999999e-05 2024-09-17T07:30:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828037+0000", 2024-09-17T07:30:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002818961 2024-09-17T07:30:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828091+0000", 2024-09-17T07:30:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3695e-05 2024-09-17T07:30:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33869 3.9 e55/23 3:907b81b5:::rbd_data.11839a97b753.0000000000000001:head v 55'171, mlcod=55'169)", 2024-09-17T07:30:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 46.755533827000001, 2024-09-17T07:30:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033990859999999999, 2024-09-17T07:30:04.621 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841626+0000", 2024-09-17T07:30:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.023e-06 2024-09-17T07:30:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842067+0000", 2024-09-17T07:30:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000440612 2024-09-17T07:30:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842080+0000", 2024-09-17T07:30:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.29e-05 2024-09-17T07:30:04.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842086+0000", 2024-09-17T07:30:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2090000000000002e-06 2024-09-17T07:30:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842144+0000", 2024-09-17T07:30:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7627999999999998e-05 2024-09-17T07:30:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842171+0000", 2024-09-17T07:30:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7214000000000001e-05 2024-09-17T07:30:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.844973+0000", 2024-09-17T07:30:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002802232 2024-09-17T07:30:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.845024+0000", 2024-09-17T07:30:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1267999999999998e-05 2024-09-17T07:30:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.625 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51432 3.0 e59/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 59'177, mlcod=58'174)", 2024-09-17T07:30:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.775936809999997, 2024-09-17T07:30:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010790086000000001, 2024-09-17T07:30:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821224+0000", 2024-09-17T07:30:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.511e-06 2024-09-17T07:30:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825174+0000", 2024-09-17T07:30:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0039497949999999999 2024-09-17T07:30:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825196+0000", 2024-09-17T07:30:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2643999999999999e-05 2024-09-17T07:30:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825206+0000", 2024-09-17T07:30:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4320000000000005e-06 2024-09-17T07:30:04.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825272+0000", 2024-09-17T07:30:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5829999999999998e-05 2024-09-17T07:30:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825299+0000", 2024-09-17T07:30:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7107e-05 2024-09-17T07:30:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.831939+0000", 2024-09-17T07:30:04.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0066398510000000004 2024-09-17T07:30:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.832012+0000", 2024-09-17T07:30:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3916000000000002e-05 2024-09-17T07:30:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51969 3.0 e59/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 59'179, mlcod=59'177)", 2024-09-17T07:30:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.667068432000001, 2024-09-17T07:30:04.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060297249999999997, 2024-09-17T07:30:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930093+0000", 2024-09-17T07:30:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.764e-06 2024-09-17T07:30:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931779+0000", 2024-09-17T07:30:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001686379 2024-09-17T07:30:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931802+0000", 2024-09-17T07:30:04.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2682000000000001e-05 2024-09-17T07:30:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931811+0000", 2024-09-17T07:30:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5640000000000002e-06 2024-09-17T07:30:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931877+0000", 2024-09-17T07:30:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6155999999999995e-05 2024-09-17T07:30:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931896+0000", 2024-09-17T07:30:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9111e-05 2024-09-17T07:30:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936063+0000", 2024-09-17T07:30:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004166711 2024-09-17T07:30:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936121+0000", 2024-09-17T07:30:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7358000000000002e-05 2024-09-17T07:30:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.634 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54417 3.0 e60/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 60'182, mlcod=59'179)", 2024-09-17T07:30:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.159551440999998, 2024-09-17T07:30:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0059885219999999996, 2024-09-17T07:30:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.635 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437609+0000", 2024-09-17T07:30:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5829999999999999e-06 2024-09-17T07:30:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437623+0000", 2024-09-17T07:30:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3348e-05 2024-09-17T07:30:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437643+0000", 2024-09-17T07:30:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0463999999999999e-05 2024-09-17T07:30:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437653+0000", 2024-09-17T07:30:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6290000000000005e-06 2024-09-17T07:30:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437719+0000", 2024-09-17T07:30:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6428999999999997e-05 2024-09-17T07:30:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437745+0000", 2024-09-17T07:30:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5690999999999998e-05 2024-09-17T07:30:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443549+0000", 2024-09-17T07:30:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0058038259999999998 2024-09-17T07:30:04.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443596+0000", 2024-09-17T07:30:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7552000000000001e-05 2024-09-17T07:30:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54419 3.0 e60/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 60'184, mlcod=59'179)", 2024-09-17T07:30:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.153199084000001, 2024-09-17T07:30:04.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004706844, 2024-09-17T07:30:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443961+0000", 2024-09-17T07:30:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:30:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443967+0000", 2024-09-17T07:30:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.268e-06 2024-09-17T07:30:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443983+0000", 2024-09-17T07:30:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5248999999999999e-05 2024-09-17T07:30:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443990+0000", 2024-09-17T07:30:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5120000000000002e-06 2024-09-17T07:30:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444056+0000", 2024-09-17T07:30:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6197000000000003e-05 2024-09-17T07:30:04.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444081+0000", 2024-09-17T07:30:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4791e-05 2024-09-17T07:30:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448616+0000", 2024-09-17T07:30:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045346129999999998 2024-09-17T07:30:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448667+0000", 2024-09-17T07:30:04.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-17T07:30:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54895 3.1f e60/52 3:fa1c1177:::rbd_data.1183ad87379e.0000000000000001:head v 60'283, mlcod=60'281)", 2024-09-17T07:30:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:30:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 40.805718087999999, 2024-09-17T07:30:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0029495210000000001, 2024-09-17T07:30:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:30:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:30:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791443+0000", 2024-09-17T07:30:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.153e-06 2024-09-17T07:30:04.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792619+0000", 2024-09-17T07:30:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001176011 2024-09-17T07:30:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792637+0000", 2024-09-17T07:30:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-05 2024-09-17T07:30:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792649+0000", 2024-09-17T07:30:04.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1712000000000001e-05 2024-09-17T07:30:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792722+0000", 2024-09-17T07:30:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3545000000000001e-05 2024-09-17T07:30:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792738+0000", 2024-09-17T07:30:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5223e-05 2024-09-17T07:30:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.794330+0000", 2024-09-17T07:30:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015922740000000001 2024-09-17T07:30:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.794391+0000", 2024-09-17T07:30:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0803e-05 2024-09-17T07:30:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:80794 3.9 e72/23 3:90311a61:::rbd_data.1183cb746bb8.0000000000000007:head v 72'222, mlcod=71'220)", 2024-09-17T07:30:04.648 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 27.165210200000001, 2024-09-17T07:30:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004371733, 2024-09-17T07:30:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0800000000000001e-07 2024-09-17T07:30:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434108+0000", 2024-09-17T07:30:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021587569999999999 2024-09-17T07:30:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434120+0000", 2024-09-17T07:30:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1821000000000001e-05 2024-09-17T07:30:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434128+0000", 2024-09-17T07:30:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7579999999999992e-06 2024-09-17T07:30:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434191+0000", 2024-09-17T07:30:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3202999999999998e-05 2024-09-17T07:30:04.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434218+0000", 2024-09-17T07:30:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6771000000000002e-05 2024-09-17T07:30:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436289+0000", 2024-09-17T07:30:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020717700000000001 2024-09-17T07:30:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436321+0000", 2024-09-17T07:30:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1445000000000001e-05 2024-09-17T07:30:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 75/74 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9, version: 36'125, data_included: [5478708~26316,5767168~524288,6553600~497972], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9@36'125, size: 10246384, copy_subset: [(0, 10246384)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7051572, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5478708, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:41.719003+0000", 2024-09-17T07:30:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.878156037, 2024-09-17T07:30:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025840260000000001, 2024-09-17T07:30:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.653 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.719003+0000", 2024-09-17T07:30:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.719003+0000", 2024-09-17T07:30:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.719004+0000", 2024-09-17T07:30:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0699999999999996e-07 2024-09-17T07:30:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.720057+0000", 2024-09-17T07:30:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010526800000000001 2024-09-17T07:30:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.720083+0000", 2024-09-17T07:30:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6237e-05 2024-09-17T07:30:04.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.720102+0000", 2024-09-17T07:30:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8918000000000001e-05 2024-09-17T07:30:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.720207+0000", 2024-09-17T07:30:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010558800000000001 2024-09-17T07:30:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.720224+0000", 2024-09-17T07:30:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6426000000000002e-05 2024-09-17T07:30:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.721587+0000", 2024-09-17T07:30:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00136337 2024-09-17T07:30:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 75/74 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head, version: 36'126, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head@36'126, size: 9431460, copy_subset: [], clone_subset: {3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9: [(0, 9431460)]}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:04.657 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.856302607, 2024-09-17T07:30:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041288289999999997, 2024-09-17T07:30:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:04.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740858+0000", 2024-09-17T07:30:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1599999999999999e-06 2024-09-17T07:30:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740885+0000", 2024-09-17T07:30:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.707e-05 2024-09-17T07:30:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740925+0000", 2024-09-17T07:30:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0649000000000003e-05 2024-09-17T07:30:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740957+0000", 2024-09-17T07:30:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1585999999999999e-05 2024-09-17T07:30:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741185+0000", 2024-09-17T07:30:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022759700000000001 2024-09-17T07:30:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741228+0000", 2024-09-17T07:30:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3705999999999998e-05 2024-09-17T07:30:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.744985+0000", 2024-09-17T07:30:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037570609999999999 2024-09-17T07:30:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.661 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85061 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'257, mlcod=79'255)", 2024-09-17T07:30:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.448471650999998, 2024-09-17T07:30:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033850550000000001, 2024-09-17T07:30:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9699999999999995e-07 2024-09-17T07:30:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148835+0000", 2024-09-17T07:30:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000146852 2024-09-17T07:30:04.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148845+0000", 2024-09-17T07:30:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0232e-05 2024-09-17T07:30:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148852+0000", 2024-09-17T07:30:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6490000000000002e-06 2024-09-17T07:30:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148913+0000", 2024-09-17T07:30:04.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1099999999999994e-05 2024-09-17T07:30:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148931+0000", 2024-09-17T07:30:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7938e-05 2024-09-17T07:30:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152025+0000", 2024-09-17T07:30:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0030942729999999998 2024-09-17T07:30:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152073+0000", 2024-09-17T07:30:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7213999999999999e-05 2024-09-17T07:30:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85063 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'259, mlcod=79'255)", 2024-09-17T07:30:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.447324696999999, 2024-09-17T07:30:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037722659999999998, 2024-09-17T07:30:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.666 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5799999999999999e-07 2024-09-17T07:30:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149841+0000", 2024-09-17T07:30:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4840000000000003e-06 2024-09-17T07:30:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149849+0000", 2024-09-17T07:30:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4670000000000004e-06 2024-09-17T07:30:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149854+0000", 2024-09-17T07:30:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4779999999999999e-06 2024-09-17T07:30:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149920+0000", 2024-09-17T07:30:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5760999999999996e-05 2024-09-17T07:30:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149949+0000", 2024-09-17T07:30:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8838000000000001e-05 2024-09-17T07:30:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153558+0000", 2024-09-17T07:30:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036092939999999999 2024-09-17T07:30:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153607+0000", 2024-09-17T07:30:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8386000000000001e-05 2024-09-17T07:30:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.670 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85085 3.5 e79/76 3:a50fe76b:::rbd_data.1183c869c667.0000000000000035:head v 79'401, mlcod=78'399)", 2024-09-17T07:30:04.671 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.151174+0000", 2024-09-17T07:30:04.671 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.445984899999999, 2024-09-17T07:30:04.671 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025105449999999999, 2024-09-17T07:30:04.671 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.671 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.671 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.671 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.671 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151174+0000", 2024-09-17T07:30:04.671 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151174+0000", 2024-09-17T07:30:04.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151175+0000", 2024-09-17T07:30:04.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9999999999999997e-07 2024-09-17T07:30:04.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151180+0000", 2024-09-17T07:30:04.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2650000000000004e-06 2024-09-17T07:30:04.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.673 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151189+0000", 2024-09-17T07:30:04.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0780000000000002e-06 2024-09-17T07:30:04.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.673 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151196+0000", 2024-09-17T07:30:04.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1600000000000003e-06 2024-09-17T07:30:04.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.673 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151267+0000", 2024-09-17T07:30:04.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1916999999999996e-05 2024-09-17T07:30:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151300+0000", 2024-09-17T07:30:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2975000000000002e-05 2024-09-17T07:30:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153644+0000", 2024-09-17T07:30:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00234329 2024-09-17T07:30:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153685+0000", 2024-09-17T07:30:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1159999999999999e-05 2024-09-17T07:30:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:94996 3.0 e82/23 3:074b4f56:::rbd_data.118311e0d6f8.0000000000000001:head v 82'715, mlcod=80'713)", 2024-09-17T07:30:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.981419104, 2024-09-17T07:30:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031499179999999998, 2024-09-17T07:30:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.675 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615741+0000", 2024-09-17T07:30:04.676 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999996e-07 2024-09-17T07:30:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616917+0000", 2024-09-17T07:30:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011763310000000001 2024-09-17T07:30:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616936+0000", 2024-09-17T07:30:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8624e-05 2024-09-17T07:30:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.677 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616948+0000", 2024-09-17T07:30:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.238e-05 2024-09-17T07:30:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617054+0000", 2024-09-17T07:30:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000106556 2024-09-17T07:30:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617081+0000", 2024-09-17T07:30:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.622e-05 2024-09-17T07:30:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618836+0000", 2024-09-17T07:30:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001755412 2024-09-17T07:30:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618890+0000", 2024-09-17T07:30:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3915000000000001e-05 2024-09-17T07:30:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.679 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:95007 3.5 e82/76 3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head v 82'422, mlcod=82'420)", 2024-09-17T07:30:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.973496403, 2024-09-17T07:30:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003939901, 2024-09-17T07:30:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623664+0000", 2024-09-17T07:30:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3799999999999999e-06 2024-09-17T07:30:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626104+0000", 2024-09-17T07:30:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002440149 2024-09-17T07:30:04.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626120+0000", 2024-09-17T07:30:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5962000000000001e-05 2024-09-17T07:30:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626127+0000", 2024-09-17T07:30:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8340000000000001e-06 2024-09-17T07:30:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626186+0000", 2024-09-17T07:30:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9234999999999998e-05 2024-09-17T07:30:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626208+0000", 2024-09-17T07:30:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1719e-05 2024-09-17T07:30:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627567+0000", 2024-09-17T07:30:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013587919999999999 2024-09-17T07:30:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627603+0000", 2024-09-17T07:30:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5830000000000001e-05 2024-09-17T07:30:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:104928 3.18 e87/76 3:1ef990b1:::rbd_data.1183dd51323.0000000000000006:head v 87'562, mlcod=86'559)", 2024-09-17T07:30:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.0547406919999993, 2024-09-17T07:30:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0061095869999999997, 2024-09-17T07:30:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:04.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542421+0000", 2024-09-17T07:30:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.069e-06 2024-09-17T07:30:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544190+0000", 2024-09-17T07:30:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017693539999999999 2024-09-17T07:30:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544215+0000", 2024-09-17T07:30:04.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5398999999999999e-05 2024-09-17T07:30:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544229+0000", 2024-09-17T07:30:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3865e-05 2024-09-17T07:30:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544322+0000", 2024-09-17T07:30:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2540000000000005e-05 2024-09-17T07:30:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:04.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544348+0000", 2024-09-17T07:30:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6197999999999999e-05 2024-09-17T07:30:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548475+0000", 2024-09-17T07:30:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041266000000000002 2024-09-17T07:30:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548528+0000", 2024-09-17T07:30:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3562e-05 2024-09-17T07:30:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.688 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:04.689 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:04.689 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:04.690 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:04.691 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:04.689+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:04.791 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:04.791+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:04.796 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:04.796 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:04.796 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:04.796 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:04.804 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:04.814 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:30:04.892 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:04.892+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:04.938 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:04.938 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:04.949 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:04.949 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:04.949 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:04.949 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:04.949 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:04.954 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:04.955 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:04.957 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:04.971 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:04.972 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:04.987 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:04.998 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:04.998+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:05.004 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:05.004 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:05.021 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:05.021 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:05.037 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:05.037 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:05.054 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:05.054 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:05.095 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:05.093+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:05.102 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:05.102 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:05.111 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:05.196 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:05.195+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:05.230 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:05.230 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:05.241 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:05.298 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:05.297+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:05.349 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:05.349 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:05.357 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:05.399 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:05.397+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:05.500 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:05.500 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:05.500 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:05.499+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:05.508 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:05.601 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:05.600+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:05.653 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:05.653 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:05.661 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:05.702 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:05.701+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:05.764 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:05.764 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:05.772 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:05.815 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:05.815+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:05.876 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:05.876 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:05.886 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:05.904 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:05.902+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:06.004 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:06.004+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:06.022 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-17T07:30:06.022 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:30:06.022 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T07:30:06.022 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T07:30:06.029 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:06.030 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:06.038 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:06.105 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:06.104+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:06.189 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:06.189 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:06.198 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:06.214 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T07:30:06.241 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:06.241+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:06.305 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:06.306 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:06.307 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:06.306+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:06.314 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:06.372 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:30:06.373 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:06.373 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 97, 2024-09-17T07:30:06.373 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "b283e6d5-0f4a-4271-a443-15e695e35364", 2024-09-17T07:30:06.373 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-17T07:28:12.276015+0000", 2024-09-17T07:30:06.373 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-17T07:30:06.151464+0000", 2024-09-17T07:30:06.373 INFO:teuthology.orchestra.run.smithi045.stdout: "last_up_change": "2024-09-17T07:29:58.120417+0000", 2024-09-17T07:30:06.373 INFO:teuthology.orchestra.run.smithi045.stdout: "last_in_change": "2024-09-17T07:29:39.311058+0000", 2024-09-17T07:30:06.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T07:30:06.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_num": 5799936, 2024-09-17T07:30:06.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_set": [ 2024-09-17T07:30:06.373 INFO:teuthology.orchestra.run.smithi045.stdout: "pglog_hardlimit", 2024-09-17T07:30:06.373 INFO:teuthology.orchestra.run.smithi045.stdout: "purged_snapdirs", 2024-09-17T07:30:06.374 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_deletes", 2024-09-17T07:30:06.374 INFO:teuthology.orchestra.run.smithi045.stdout: "sortbitwise" 2024-09-17T07:30:06.374 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:30:06.374 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_version": 6, 2024-09-17T07:30:06.374 INFO:teuthology.orchestra.run.smithi045.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T07:30:06.374 INFO:teuthology.orchestra.run.smithi045.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T07:30:06.374 INFO:teuthology.orchestra.run.smithi045.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T07:30:06.374 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_snapshot": "", 2024-09-17T07:30:06.374 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_max": 3, 2024-09-17T07:30:06.374 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd": 8, 2024-09-17T07:30:06.374 INFO:teuthology.orchestra.run.smithi045.stdout: "require_min_compat_client": "luminous", 2024-09-17T07:30:06.374 INFO:teuthology.orchestra.run.smithi045.stdout: "min_compat_client": "jewel", 2024-09-17T07:30:06.374 INFO:teuthology.orchestra.run.smithi045.stdout: "require_osd_release": "squid", 2024-09-17T07:30:06.374 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_crimson": false, 2024-09-17T07:30:06.375 INFO:teuthology.orchestra.run.smithi045.stdout: "pools": [ 2024-09-17T07:30:06.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.375 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 1, 2024-09-17T07:30:06.375 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": ".mgr", 2024-09-17T07:30:06.375 INFO:teuthology.orchestra.run.smithi045.stdout: "create_time": "2024-09-17T07:28:24.767696+0000", 2024-09-17T07:30:06.375 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-17T07:30:06.375 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_names": "hashpspool", 2024-09-17T07:30:06.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": 1, 2024-09-17T07:30:06.376 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 2, 2024-09-17T07:30:06.376 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": 1, 2024-09-17T07:30:06.376 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_rule": 0, 2024-09-17T07:30:06.376 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_count": 0, 2024-09-17T07:30:06.376 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_target": 0, 2024-09-17T07:30:06.376 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T07:30:06.376 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T07:30:06.376 INFO:teuthology.orchestra.run.smithi045.stdout: "is_stretch_pool": false, 2024-09-17T07:30:06.377 INFO:teuthology.orchestra.run.smithi045.stdout: "object_hash": 2, 2024-09-17T07:30:06.377 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscale_mode": "off", 2024-09-17T07:30:06.377 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": 1, 2024-09-17T07:30:06.377 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num": 1, 2024-09-17T07:30:06.377 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num_target": 1, 2024-09-17T07:30:06.377 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_target": 1, 2024-09-17T07:30:06.377 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_pending": 1, 2024-09-17T07:30:06.378 INFO:teuthology.orchestra.run.smithi045.stdout: "last_pg_merge_meta": { 2024-09-17T07:30:06.378 INFO:teuthology.orchestra.run.smithi045.stdout: "source_pgid": "0.0", 2024-09-17T07:30:06.378 INFO:teuthology.orchestra.run.smithi045.stdout: "ready_epoch": 0, 2024-09-17T07:30:06.378 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_started": 0, 2024-09-17T07:30:06.378 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_clean": 0, 2024-09-17T07:30:06.378 INFO:teuthology.orchestra.run.smithi045.stdout: "source_version": "0'0", 2024-09-17T07:30:06.378 INFO:teuthology.orchestra.run.smithi045.stdout: "target_version": "0'0" 2024-09-17T07:30:06.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.379 INFO:teuthology.orchestra.run.smithi045.stdout: "last_change": "18", 2024-09-17T07:30:06.379 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend": "0", 2024-09-17T07:30:06.379 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T07:30:06.379 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T07:30:06.379 INFO:teuthology.orchestra.run.smithi045.stdout: "auid": 0, 2024-09-17T07:30:06.379 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_mode": "selfmanaged", 2024-09-17T07:30:06.379 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_seq": 0, 2024-09-17T07:30:06.379 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_epoch": 0, 2024-09-17T07:30:06.379 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_snaps": [], 2024-09-17T07:30:06.379 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps": "[]", 2024-09-17T07:30:06.379 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_bytes": 0, 2024-09-17T07:30:06.379 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_objects": 0, 2024-09-17T07:30:06.379 INFO:teuthology.orchestra.run.smithi045.stdout: "tiers": [], 2024-09-17T07:30:06.380 INFO:teuthology.orchestra.run.smithi045.stdout: "tier_of": -1, 2024-09-17T07:30:06.380 INFO:teuthology.orchestra.run.smithi045.stdout: "read_tier": -1, 2024-09-17T07:30:06.380 INFO:teuthology.orchestra.run.smithi045.stdout: "write_tier": -1, 2024-09-17T07:30:06.380 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_mode": "none", 2024-09-17T07:30:06.380 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_bytes": 0, 2024-09-17T07:30:06.380 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_objects": 0, 2024-09-17T07:30:06.380 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T07:30:06.381 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T07:30:06.381 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T07:30:06.381 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_flush_age": 0, 2024-09-17T07:30:06.381 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_evict_age": 0, 2024-09-17T07:30:06.381 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profile": "", 2024-09-17T07:30:06.381 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_params": { 2024-09-17T07:30:06.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "none" 2024-09-17T07:30:06.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.381 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_period": 0, 2024-09-17T07:30:06.382 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_count": 0, 2024-09-17T07:30:06.382 INFO:teuthology.orchestra.run.smithi045.stdout: "use_gmt_hitset": true, 2024-09-17T07:30:06.382 INFO:teuthology.orchestra.run.smithi045.stdout: "min_read_recency_for_promote": 0, 2024-09-17T07:30:06.382 INFO:teuthology.orchestra.run.smithi045.stdout: "min_write_recency_for_promote": 0, 2024-09-17T07:30:06.382 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T07:30:06.382 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_search_last_n": 0, 2024-09-17T07:30:06.382 INFO:teuthology.orchestra.run.smithi045.stdout: "grade_table": [], 2024-09-17T07:30:06.382 INFO:teuthology.orchestra.run.smithi045.stdout: "stripe_width": 0, 2024-09-17T07:30:06.382 INFO:teuthology.orchestra.run.smithi045.stdout: "expected_num_objects": 0, 2024-09-17T07:30:06.382 INFO:teuthology.orchestra.run.smithi045.stdout: "fast_read": false, 2024-09-17T07:30:06.382 INFO:teuthology.orchestra.run.smithi045.stdout: "options": { 2024-09-17T07:30:06.382 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_max": 32, 2024-09-17T07:30:06.382 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_min": 1 2024-09-17T07:30:06.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.383 INFO:teuthology.orchestra.run.smithi045.stdout: "application_metadata": { 2024-09-17T07:30:06.383 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": {} 2024-09-17T07:30:06.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.383 INFO:teuthology.orchestra.run.smithi045.stdout: "read_balance": { 2024-09-17T07:30:06.383 INFO:teuthology.orchestra.run.smithi045.stdout: "score_type": "Fair distribution", 2024-09-17T07:30:06.383 INFO:teuthology.orchestra.run.smithi045.stdout: "score_acting": 8, 2024-09-17T07:30:06.383 INFO:teuthology.orchestra.run.smithi045.stdout: "score_stable": 8, 2024-09-17T07:30:06.383 INFO:teuthology.orchestra.run.smithi045.stdout: "optimal_score": 0.25, 2024-09-17T07:30:06.383 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_acting": 2, 2024-09-17T07:30:06.383 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_stable": 2, 2024-09-17T07:30:06.383 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity_weighted": 1, 2024-09-17T07:30:06.383 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity": 1, 2024-09-17T07:30:06.384 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity_weighted": 1 2024-09-17T07:30:06.384 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.384 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.384 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 2, 2024-09-17T07:30:06.384 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": "rbd", 2024-09-17T07:30:06.384 INFO:teuthology.orchestra.run.smithi045.stdout: "create_time": "2024-09-17T07:28:30.249487+0000", 2024-09-17T07:30:06.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 8193, 2024-09-17T07:30:06.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T07:30:06.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": 1, 2024-09-17T07:30:06.384 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 2, 2024-09-17T07:30:06.384 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": 1, 2024-09-17T07:30:06.384 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_rule": 0, 2024-09-17T07:30:06.385 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_count": 0, 2024-09-17T07:30:06.385 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_target": 0, 2024-09-17T07:30:06.385 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T07:30:06.385 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T07:30:06.385 INFO:teuthology.orchestra.run.smithi045.stdout: "is_stretch_pool": false, 2024-09-17T07:30:06.385 INFO:teuthology.orchestra.run.smithi045.stdout: "object_hash": 2, 2024-09-17T07:30:06.386 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscale_mode": "off", 2024-09-17T07:30:06.386 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": 8, 2024-09-17T07:30:06.386 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num": 8, 2024-09-17T07:30:06.386 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num_target": 8, 2024-09-17T07:30:06.386 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_target": 8, 2024-09-17T07:30:06.386 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_pending": 8, 2024-09-17T07:30:06.386 INFO:teuthology.orchestra.run.smithi045.stdout: "last_pg_merge_meta": { 2024-09-17T07:30:06.386 INFO:teuthology.orchestra.run.smithi045.stdout: "source_pgid": "0.0", 2024-09-17T07:30:06.386 INFO:teuthology.orchestra.run.smithi045.stdout: "ready_epoch": 0, 2024-09-17T07:30:06.386 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_started": 0, 2024-09-17T07:30:06.386 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_clean": 0, 2024-09-17T07:30:06.386 INFO:teuthology.orchestra.run.smithi045.stdout: "source_version": "0'0", 2024-09-17T07:30:06.387 INFO:teuthology.orchestra.run.smithi045.stdout: "target_version": "0'0" 2024-09-17T07:30:06.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.387 INFO:teuthology.orchestra.run.smithi045.stdout: "last_change": "22", 2024-09-17T07:30:06.387 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend": "0", 2024-09-17T07:30:06.387 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T07:30:06.387 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T07:30:06.387 INFO:teuthology.orchestra.run.smithi045.stdout: "auid": 0, 2024-09-17T07:30:06.388 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_mode": "selfmanaged", 2024-09-17T07:30:06.388 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_seq": 2, 2024-09-17T07:30:06.388 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_epoch": 22, 2024-09-17T07:30:06.388 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_snaps": [], 2024-09-17T07:30:06.388 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps": "[]", 2024-09-17T07:30:06.388 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_bytes": 0, 2024-09-17T07:30:06.388 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_objects": 0, 2024-09-17T07:30:06.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tiers": [], 2024-09-17T07:30:06.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tier_of": -1, 2024-09-17T07:30:06.388 INFO:teuthology.orchestra.run.smithi045.stdout: "read_tier": -1, 2024-09-17T07:30:06.388 INFO:teuthology.orchestra.run.smithi045.stdout: "write_tier": -1, 2024-09-17T07:30:06.389 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_mode": "none", 2024-09-17T07:30:06.389 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_bytes": 0, 2024-09-17T07:30:06.389 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_objects": 0, 2024-09-17T07:30:06.389 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T07:30:06.389 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T07:30:06.389 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T07:30:06.389 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_flush_age": 0, 2024-09-17T07:30:06.389 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_evict_age": 0, 2024-09-17T07:30:06.389 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profile": "", 2024-09-17T07:30:06.389 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_params": { 2024-09-17T07:30:06.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "none" 2024-09-17T07:30:06.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.389 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_period": 0, 2024-09-17T07:30:06.389 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_count": 0, 2024-09-17T07:30:06.390 INFO:teuthology.orchestra.run.smithi045.stdout: "use_gmt_hitset": true, 2024-09-17T07:30:06.390 INFO:teuthology.orchestra.run.smithi045.stdout: "min_read_recency_for_promote": 0, 2024-09-17T07:30:06.390 INFO:teuthology.orchestra.run.smithi045.stdout: "min_write_recency_for_promote": 0, 2024-09-17T07:30:06.390 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T07:30:06.390 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_search_last_n": 0, 2024-09-17T07:30:06.390 INFO:teuthology.orchestra.run.smithi045.stdout: "grade_table": [], 2024-09-17T07:30:06.390 INFO:teuthology.orchestra.run.smithi045.stdout: "stripe_width": 0, 2024-09-17T07:30:06.390 INFO:teuthology.orchestra.run.smithi045.stdout: "expected_num_objects": 0, 2024-09-17T07:30:06.390 INFO:teuthology.orchestra.run.smithi045.stdout: "fast_read": false, 2024-09-17T07:30:06.390 INFO:teuthology.orchestra.run.smithi045.stdout: "options": {}, 2024-09-17T07:30:06.390 INFO:teuthology.orchestra.run.smithi045.stdout: "application_metadata": { 2024-09-17T07:30:06.390 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd": {} 2024-09-17T07:30:06.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.391 INFO:teuthology.orchestra.run.smithi045.stdout: "read_balance": { 2024-09-17T07:30:06.391 INFO:teuthology.orchestra.run.smithi045.stdout: "score_type": "Fair distribution", 2024-09-17T07:30:06.391 INFO:teuthology.orchestra.run.smithi045.stdout: "score_acting": 2, 2024-09-17T07:30:06.391 INFO:teuthology.orchestra.run.smithi045.stdout: "score_stable": 2, 2024-09-17T07:30:06.391 INFO:teuthology.orchestra.run.smithi045.stdout: "optimal_score": 0.75, 2024-09-17T07:30:06.391 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_acting": 1.5, 2024-09-17T07:30:06.391 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_stable": 1.5, 2024-09-17T07:30:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity_weighted": 1, 2024-09-17T07:30:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity": 1, 2024-09-17T07:30:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity_weighted": 1 2024-09-17T07:30:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 3, 2024-09-17T07:30:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": "pool_client.0", 2024-09-17T07:30:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: "create_time": "2024-09-17T07:28:41.831004+0000", 2024-09-17T07:30:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 8193, 2024-09-17T07:30:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T07:30:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": 1, 2024-09-17T07:30:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 2, 2024-09-17T07:30:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": 1, 2024-09-17T07:30:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_rule": 0, 2024-09-17T07:30:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_count": 0, 2024-09-17T07:30:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_target": 0, 2024-09-17T07:30:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T07:30:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T07:30:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: "is_stretch_pool": false, 2024-09-17T07:30:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: "object_hash": 2, 2024-09-17T07:30:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscale_mode": "off", 2024-09-17T07:30:06.394 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": 32, 2024-09-17T07:30:06.394 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num": 32, 2024-09-17T07:30:06.394 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num_target": 32, 2024-09-17T07:30:06.394 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_target": 32, 2024-09-17T07:30:06.394 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_pending": 32, 2024-09-17T07:30:06.394 INFO:teuthology.orchestra.run.smithi045.stdout: "last_pg_merge_meta": { 2024-09-17T07:30:06.394 INFO:teuthology.orchestra.run.smithi045.stdout: "source_pgid": "0.0", 2024-09-17T07:30:06.394 INFO:teuthology.orchestra.run.smithi045.stdout: "ready_epoch": 0, 2024-09-17T07:30:06.394 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_started": 0, 2024-09-17T07:30:06.394 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_clean": 0, 2024-09-17T07:30:06.394 INFO:teuthology.orchestra.run.smithi045.stdout: "source_version": "0'0", 2024-09-17T07:30:06.394 INFO:teuthology.orchestra.run.smithi045.stdout: "target_version": "0'0" 2024-09-17T07:30:06.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.394 INFO:teuthology.orchestra.run.smithi045.stdout: "last_change": "97", 2024-09-17T07:30:06.395 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend": "0", 2024-09-17T07:30:06.395 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T07:30:06.395 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T07:30:06.395 INFO:teuthology.orchestra.run.smithi045.stdout: "auid": 0, 2024-09-17T07:30:06.395 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_mode": "selfmanaged", 2024-09-17T07:30:06.395 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_seq": 48, 2024-09-17T07:30:06.395 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_epoch": 97, 2024-09-17T07:30:06.395 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_snaps": [], 2024-09-17T07:30:06.395 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps": "[]", 2024-09-17T07:30:06.395 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_bytes": 0, 2024-09-17T07:30:06.395 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_objects": 0, 2024-09-17T07:30:06.395 INFO:teuthology.orchestra.run.smithi045.stdout: "tiers": [], 2024-09-17T07:30:06.395 INFO:teuthology.orchestra.run.smithi045.stdout: "tier_of": -1, 2024-09-17T07:30:06.395 INFO:teuthology.orchestra.run.smithi045.stdout: "read_tier": -1, 2024-09-17T07:30:06.396 INFO:teuthology.orchestra.run.smithi045.stdout: "write_tier": -1, 2024-09-17T07:30:06.396 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_mode": "none", 2024-09-17T07:30:06.396 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_bytes": 0, 2024-09-17T07:30:06.396 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_objects": 0, 2024-09-17T07:30:06.396 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T07:30:06.396 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T07:30:06.396 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T07:30:06.396 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_flush_age": 0, 2024-09-17T07:30:06.396 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_evict_age": 0, 2024-09-17T07:30:06.396 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profile": "", 2024-09-17T07:30:06.396 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_params": { 2024-09-17T07:30:06.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "none" 2024-09-17T07:30:06.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.396 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_period": 0, 2024-09-17T07:30:06.397 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_count": 0, 2024-09-17T07:30:06.397 INFO:teuthology.orchestra.run.smithi045.stdout: "use_gmt_hitset": true, 2024-09-17T07:30:06.397 INFO:teuthology.orchestra.run.smithi045.stdout: "min_read_recency_for_promote": 0, 2024-09-17T07:30:06.397 INFO:teuthology.orchestra.run.smithi045.stdout: "min_write_recency_for_promote": 0, 2024-09-17T07:30:06.397 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T07:30:06.397 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_search_last_n": 0, 2024-09-17T07:30:06.397 INFO:teuthology.orchestra.run.smithi045.stdout: "grade_table": [], 2024-09-17T07:30:06.397 INFO:teuthology.orchestra.run.smithi045.stdout: "stripe_width": 0, 2024-09-17T07:30:06.397 INFO:teuthology.orchestra.run.smithi045.stdout: "expected_num_objects": 0, 2024-09-17T07:30:06.397 INFO:teuthology.orchestra.run.smithi045.stdout: "fast_read": false, 2024-09-17T07:30:06.397 INFO:teuthology.orchestra.run.smithi045.stdout: "options": {}, 2024-09-17T07:30:06.397 INFO:teuthology.orchestra.run.smithi045.stdout: "application_metadata": { 2024-09-17T07:30:06.398 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd": {} 2024-09-17T07:30:06.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.398 INFO:teuthology.orchestra.run.smithi045.stdout: "read_balance": { 2024-09-17T07:30:06.398 INFO:teuthology.orchestra.run.smithi045.stdout: "score_type": "Fair distribution", 2024-09-17T07:30:06.398 INFO:teuthology.orchestra.run.smithi045.stdout: "score_acting": 1.7400000095367432, 2024-09-17T07:30:06.398 INFO:teuthology.orchestra.run.smithi045.stdout: "score_stable": 1.7400000095367432, 2024-09-17T07:30:06.398 INFO:teuthology.orchestra.run.smithi045.stdout: "optimal_score": 0.87999999523162842, 2024-09-17T07:30:06.398 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-17T07:30:06.398 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-17T07:30:06.399 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity_weighted": 1, 2024-09-17T07:30:06.399 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity": 1, 2024-09-17T07:30:06.399 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity_weighted": 1 2024-09-17T07:30:06.399 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.399 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.399 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:30:06.399 INFO:teuthology.orchestra.run.smithi045.stdout: "osds": [ 2024-09-17T07:30:06.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.399 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 0, 2024-09-17T07:30:06.399 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "7794e6a8-ebf4-4b7c-bf3d-63cd015234ef", 2024-09-17T07:30:06.399 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:06.399 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-17T07:30:06.399 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0.845458984375, 2024-09-17T07:30:06.400 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:06.400 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-17T07:30:06.400 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-17T07:30:06.400 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 14, 2024-09-17T07:30:06.400 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 76, 2024-09-17T07:30:06.400 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-17T07:30:06.400 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:06.400 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:06.400 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.400 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6800", 2024-09-17T07:30:06.400 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:06.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.401 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6801", 2024-09-17T07:30:06.401 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:06.401 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.401 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.401 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:06.401 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.402 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6802", 2024-09-17T07:30:06.402 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:06.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.402 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.402 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6803", 2024-09-17T07:30:06.402 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:06.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.402 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.402 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:06.402 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.402 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.403 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6806", 2024-09-17T07:30:06.403 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:06.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.403 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6807", 2024-09-17T07:30:06.403 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:06.403 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.403 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.403 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:06.403 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.404 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6804", 2024-09-17T07:30:06.404 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:06.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.404 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6805", 2024-09-17T07:30:06.404 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:06.404 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.404 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.404 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6801/2039818160", 2024-09-17T07:30:06.404 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6803/2039818160", 2024-09-17T07:30:06.405 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6807/2039818160", 2024-09-17T07:30:06.405 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6805/2039818160", 2024-09-17T07:30:06.405 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:06.405 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:06.405 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:06.405 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.405 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 1, 2024-09-17T07:30:06.405 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5", 2024-09-17T07:30:06.405 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:06.405 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-17T07:30:06.405 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-17T07:30:06.405 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:06.406 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-17T07:30:06.406 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-17T07:30:06.406 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 14, 2024-09-17T07:30:06.406 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 76, 2024-09-17T07:30:06.406 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-17T07:30:06.406 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:06.406 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:06.406 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.406 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.406 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6808", 2024-09-17T07:30:06.406 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:06.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.407 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6809", 2024-09-17T07:30:06.407 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:06.407 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.407 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.407 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:06.407 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.407 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6810", 2024-09-17T07:30:06.407 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:06.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.408 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6811", 2024-09-17T07:30:06.408 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:06.408 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.408 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.408 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:06.408 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.409 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6814", 2024-09-17T07:30:06.409 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:06.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.409 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.409 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6815", 2024-09-17T07:30:06.409 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:06.409 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.409 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.410 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:06.410 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.410 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6812", 2024-09-17T07:30:06.410 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:06.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.410 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6813", 2024-09-17T07:30:06.410 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:06.410 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.410 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.411 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6809/1158817756", 2024-09-17T07:30:06.411 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6811/1158817756", 2024-09-17T07:30:06.411 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6815/1158817756", 2024-09-17T07:30:06.411 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6813/1158817756", 2024-09-17T07:30:06.411 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:06.411 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:06.411 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:06.411 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.411 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 2, 2024-09-17T07:30:06.411 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "195a9db1-568d-459a-bb50-2a1bb2445f53", 2024-09-17T07:30:06.411 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:06.412 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-17T07:30:06.412 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-17T07:30:06.412 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:06.412 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-17T07:30:06.412 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-17T07:30:06.412 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 14, 2024-09-17T07:30:06.412 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 73, 2024-09-17T07:30:06.412 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-17T07:30:06.412 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:06.412 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:06.412 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.413 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6824", 2024-09-17T07:30:06.413 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:06.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.413 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.413 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6825", 2024-09-17T07:30:06.413 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:06.413 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.413 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.413 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:06.413 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.414 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6826", 2024-09-17T07:30:06.414 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:06.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.414 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6827", 2024-09-17T07:30:06.414 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:06.414 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.414 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.415 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:06.415 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.415 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6830", 2024-09-17T07:30:06.415 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:06.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.415 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6831", 2024-09-17T07:30:06.415 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:06.415 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.415 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.416 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:06.416 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.416 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6828", 2024-09-17T07:30:06.416 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:06.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.416 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6829", 2024-09-17T07:30:06.416 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:06.416 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.417 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.417 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6825/3298886784", 2024-09-17T07:30:06.417 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6827/3298886784", 2024-09-17T07:30:06.417 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6831/3298886784", 2024-09-17T07:30:06.417 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6829/3298886784", 2024-09-17T07:30:06.417 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:06.417 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:06.417 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:06.418 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.418 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 3, 2024-09-17T07:30:06.418 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "b3fda98b-a10a-4354-905e-1b37a5124b98", 2024-09-17T07:30:06.418 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:06.418 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 0, 2024-09-17T07:30:06.419 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0, 2024-09-17T07:30:06.419 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:06.419 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 14, 2024-09-17T07:30:06.419 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 36, 2024-09-17T07:30:06.419 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 90, 2024-09-17T07:30:06.419 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 23, 2024-09-17T07:30:06.419 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 37, 2024-09-17T07:30:06.419 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:06.420 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:06.420 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.420 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.420 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6816", 2024-09-17T07:30:06.421 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:06.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.421 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6817", 2024-09-17T07:30:06.421 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:06.421 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.422 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.422 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:06.422 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.422 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6818", 2024-09-17T07:30:06.422 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:06.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.423 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.423 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6819", 2024-09-17T07:30:06.423 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:06.423 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.423 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.423 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:06.423 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.424 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6822", 2024-09-17T07:30:06.424 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:06.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.425 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6823", 2024-09-17T07:30:06.425 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:06.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.425 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.425 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:06.425 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.426 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6820", 2024-09-17T07:30:06.426 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:06.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.426 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6821", 2024-09-17T07:30:06.426 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:06.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.427 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.427 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6817/724784918", 2024-09-17T07:30:06.427 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6819/724784918", 2024-09-17T07:30:06.427 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6823/724784918", 2024-09-17T07:30:06.427 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6821/724784918", 2024-09-17T07:30:06.427 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:06.427 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:06.427 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:06.427 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.427 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 4, 2024-09-17T07:30:06.428 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "6ab67b48-2d83-4e68-8898-302a1adc0a52", 2024-09-17T07:30:06.428 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:06.428 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-17T07:30:06.428 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0.783294677734375, 2024-09-17T07:30:06.428 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:06.428 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-17T07:30:06.428 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-17T07:30:06.428 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 14, 2024-09-17T07:30:06.428 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 51, 2024-09-17T07:30:06.428 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-17T07:30:06.428 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:06.428 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:06.429 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.429 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6824", 2024-09-17T07:30:06.429 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:06.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.429 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6825", 2024-09-17T07:30:06.429 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:06.429 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.429 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.429 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:06.430 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.430 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.430 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6826", 2024-09-17T07:30:06.430 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:06.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.430 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.430 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6827", 2024-09-17T07:30:06.430 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:06.430 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.430 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.430 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:06.431 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.431 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6830", 2024-09-17T07:30:06.431 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:06.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.431 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6831", 2024-09-17T07:30:06.431 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:06.431 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.431 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.431 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:06.432 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.432 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6828", 2024-09-17T07:30:06.432 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:06.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.432 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6829", 2024-09-17T07:30:06.432 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:06.432 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.432 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.433 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.192:6825/1644784047", 2024-09-17T07:30:06.433 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.192:6827/1644784047", 2024-09-17T07:30:06.433 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.192:6831/1644784047", 2024-09-17T07:30:06.433 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.192:6829/1644784047", 2024-09-17T07:30:06.433 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:06.433 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:06.433 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:06.433 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.433 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 5, 2024-09-17T07:30:06.433 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "ae3c787c-63d2-4e88-8f99-6d4f19837a3a", 2024-09-17T07:30:06.434 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:06.434 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-17T07:30:06.434 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-17T07:30:06.434 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:06.434 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-17T07:30:06.434 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-17T07:30:06.434 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 14, 2024-09-17T07:30:06.434 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 77, 2024-09-17T07:30:06.434 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-17T07:30:06.434 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:06.434 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:06.434 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.435 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6816", 2024-09-17T07:30:06.435 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:06.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.435 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6817", 2024-09-17T07:30:06.435 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:06.435 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.435 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.435 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:06.435 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.436 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6818", 2024-09-17T07:30:06.436 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:06.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.436 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6819", 2024-09-17T07:30:06.436 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:06.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.436 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.436 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:06.436 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.437 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6822", 2024-09-17T07:30:06.437 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:06.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.437 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6823", 2024-09-17T07:30:06.437 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:06.437 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.437 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.438 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:06.438 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.438 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.438 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6820", 2024-09-17T07:30:06.438 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:06.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.438 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.438 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6821", 2024-09-17T07:30:06.438 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:06.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.438 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.439 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.192:6817/4077426583", 2024-09-17T07:30:06.439 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.192:6819/4077426583", 2024-09-17T07:30:06.439 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.192:6823/4077426583", 2024-09-17T07:30:06.439 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.192:6821/4077426583", 2024-09-17T07:30:06.439 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:06.439 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:06.439 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:06.439 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.439 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 6, 2024-09-17T07:30:06.439 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "f792f18d-ad9e-49a8-b194-24522398fb25", 2024-09-17T07:30:06.439 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:06.439 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-17T07:30:06.440 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-17T07:30:06.440 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:06.440 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-17T07:30:06.440 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-17T07:30:06.440 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 14, 2024-09-17T07:30:06.440 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 76, 2024-09-17T07:30:06.440 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-17T07:30:06.440 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:06.440 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:06.440 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.440 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.440 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6808", 2024-09-17T07:30:06.441 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:06.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.441 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6809", 2024-09-17T07:30:06.441 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:06.441 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.441 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.441 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:06.441 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.442 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.442 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6810", 2024-09-17T07:30:06.442 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:06.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.442 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.442 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6811", 2024-09-17T07:30:06.442 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:06.442 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.442 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.443 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:06.443 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.443 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6814", 2024-09-17T07:30:06.443 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:06.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.443 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6815", 2024-09-17T07:30:06.443 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:06.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.443 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.444 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:06.444 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.444 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.444 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6812", 2024-09-17T07:30:06.444 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:06.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.444 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.444 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6813", 2024-09-17T07:30:06.444 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:06.444 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.445 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.445 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.192:6809/2922229639", 2024-09-17T07:30:06.445 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.192:6811/2922229639", 2024-09-17T07:30:06.445 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.192:6815/2922229639", 2024-09-17T07:30:06.445 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.192:6813/2922229639", 2024-09-17T07:30:06.445 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:06.445 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:06.445 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:06.445 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.445 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 7, 2024-09-17T07:30:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "4a589d43-b0b9-42de-989c-46928d693c24", 2024-09-17T07:30:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-17T07:30:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0.8525848388671875, 2024-09-17T07:30:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-17T07:30:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-17T07:30:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 14, 2024-09-17T07:30:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 76, 2024-09-17T07:30:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-17T07:30:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.447 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6800", 2024-09-17T07:30:06.447 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:06.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.447 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6801", 2024-09-17T07:30:06.447 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:06.447 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.447 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.447 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:06.447 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.448 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6802", 2024-09-17T07:30:06.448 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:06.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.448 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6803", 2024-09-17T07:30:06.448 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:06.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.448 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.448 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:06.448 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.449 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6806", 2024-09-17T07:30:06.449 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:06.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.449 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6807", 2024-09-17T07:30:06.449 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:06.449 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.449 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.450 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:06.450 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:06.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.450 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:06.450 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6804", 2024-09-17T07:30:06.450 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:06.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.450 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:06.450 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6805", 2024-09-17T07:30:06.450 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:06.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.450 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.451 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.192:6801/780298944", 2024-09-17T07:30:06.451 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.192:6803/780298944", 2024-09-17T07:30:06.451 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.192:6807/780298944", 2024-09-17T07:30:06.451 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.192:6805/780298944", 2024-09-17T07:30:06.451 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:06.451 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:06.451 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:06.451 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.451 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.451 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:30:06.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_xinfo": [ 2024-09-17T07:30:06.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.451 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 0, 2024-09-17T07:30:06.451 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:06.451 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:06.452 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:06.452 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:06.452 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-17T07:30:06.452 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:20.586514+0000", 2024-09-17T07:30:06.452 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-17T07:30:06.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.452 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 1, 2024-09-17T07:30:06.452 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:06.452 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:06.452 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:06.452 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:06.452 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-17T07:30:06.453 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:20.833505+0000", 2024-09-17T07:30:06.453 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-17T07:30:06.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.453 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 2, 2024-09-17T07:30:06.453 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:06.453 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:06.453 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:06.453 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:06.453 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-17T07:30:06.453 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:21.059899+0000", 2024-09-17T07:30:06.453 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-17T07:30:06.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.454 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 3, 2024-09-17T07:30:06.454 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:06.454 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:06.454 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:06.454 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:06.454 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 65536, 2024-09-17T07:30:06.454 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:20.499470+0000", 2024-09-17T07:30:06.454 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 37 2024-09-17T07:30:06.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.454 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 4, 2024-09-17T07:30:06.454 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:06.454 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-17T07:30:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:21.353342+0000", 2024-09-17T07:30:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-17T07:30:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 5, 2024-09-17T07:30:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-17T07:30:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:20.586673+0000", 2024-09-17T07:30:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-17T07:30:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 6, 2024-09-17T07:30:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-17T07:30:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:20.767847+0000", 2024-09-17T07:30:06.457 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-17T07:30:06.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.457 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 7, 2024-09-17T07:30:06.457 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:06.457 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:06.457 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:06.458 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:06.458 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-17T07:30:06.458 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:20.859615+0000", 2024-09-17T07:30:06.458 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-17T07:30:06.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.458 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:30:06.458 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_upmap": [], 2024-09-17T07:30:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_upmap_items": [], 2024-09-17T07:30:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_upmap_primaries": [], 2024-09-17T07:30:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_temp": [], 2024-09-17T07:30:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_temp": [], 2024-09-17T07:30:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklist": {}, 2024-09-17T07:30:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "range_blocklist": {}, 2024-09-17T07:30:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profiles": { 2024-09-17T07:30:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "default": { 2024-09-17T07:30:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-failure-domain": "osd", 2024-09-17T07:30:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "k": "2", 2024-09-17T07:30:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "m": "1", 2024-09-17T07:30:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "plugin": "jerasure", 2024-09-17T07:30:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: "technique": "reed_sol_van" 2024-09-17T07:30:06.459 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:06.460 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps_queue": [ 2024-09-17T07:30:06.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.460 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 3, 2024-09-17T07:30:06.460 INFO:teuthology.orchestra.run.smithi045.stdout: "snaps": [ 2024-09-17T07:30:06.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.460 INFO:teuthology.orchestra.run.smithi045.stdout: "begin": 47, 2024-09-17T07:30:06.460 INFO:teuthology.orchestra.run.smithi045.stdout: "length": 2 2024-09-17T07:30:06.460 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.460 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:30:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: "new_removed_snaps": [ 2024-09-17T07:30:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 3, 2024-09-17T07:30:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: "snaps": [ 2024-09-17T07:30:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: "begin": 47, 2024-09-17T07:30:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: "length": 1 2024-09-17T07:30:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:30:06.461 INFO:teuthology.orchestra.run.smithi045.stdout: "new_purged_snaps": [], 2024-09-17T07:30:06.462 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_node_flags": {}, 2024-09-17T07:30:06.462 INFO:teuthology.orchestra.run.smithi045.stdout: "device_class_flags": {}, 2024-09-17T07:30:06.462 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": { 2024-09-17T07:30:06.462 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode_enabled": false, 2024-09-17T07:30:06.462 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_bucket_count": 0, 2024-09-17T07:30:06.462 INFO:teuthology.orchestra.run.smithi045.stdout: "degraded_stretch_mode": 0, 2024-09-17T07:30:06.462 INFO:teuthology.orchestra.run.smithi045.stdout: "recovering_stretch_mode": 0, 2024-09-17T07:30:06.462 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode_bucket": 0 2024-09-17T07:30:06.462 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:06.462 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:06.465 INFO:tasks.thrashosds.thrasher:j is {'epoch': 97, 'fsid': 'b283e6d5-0f4a-4271-a443-15e695e35364', 'created': '2024-09-17T07:28:12.276015+0000', 'modified': '2024-09-17T07:30:06.151464+0000', 'last_up_change': '2024-09-17T07:29:58.120417+0000', 'last_in_change': '2024-09-17T07:29:39.311058+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T07:28:24.767696+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T07:28:30.249487+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.75, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T07:28:41.831004+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 48, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '7794e6a8-ebf4-4b7c-bf3d-63cd015234ef', 'up': 1, 'in': 1, 'weight': 0.845458984375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6800', 'nonce': 2039818160}, {'type': 'v1', 'addr': '172.21.15.45:6801', 'nonce': 2039818160}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6802', 'nonce': 2039818160}, {'type': 'v1', 'addr': '172.21.15.45:6803', 'nonce': 2039818160}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6806', 'nonce': 2039818160}, {'type': 'v1', 'addr': '172.21.15.45:6807', 'nonce': 2039818160}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6804', 'nonce': 2039818160}, {'type': 'v1', 'addr': '172.21.15.45:6805', 'nonce': 2039818160}]}, 'public_addr': '172.21.15.45:6801/2039818160', 'cluster_addr': '172.21.15.45:6803/2039818160', 'heartbeat_back_addr': '172.21.15.45:6807/2039818160', 'heartbeat_front_addr': '172.21.15.45:6805/2039818160', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6808', 'nonce': 1158817756}, {'type': 'v1', 'addr': '172.21.15.45:6809', 'nonce': 1158817756}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6810', 'nonce': 1158817756}, {'type': 'v1', 'addr': '172.21.15.45:6811', 'nonce': 1158817756}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6814', 'nonce': 1158817756}, {'type': 'v1', 'addr': '172.21.15.45:6815', 'nonce': 1158817756}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6812', 'nonce': 1158817756}, {'type': 'v1', 'addr': '172.21.15.45:6813', 'nonce': 1158817756}]}, 'public_addr': '172.21.15.45:6809/1158817756', 'cluster_addr': '172.21.15.45:6811/1158817756', 'heartbeat_back_addr': '172.21.15.45:6815/1158817756', 'heartbeat_front_addr': '172.21.15.45:6813/1158817756', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '195a9db1-568d-459a-bb50-2a1bb2445f53', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 73, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6824', 'nonce': 3298886784}, {'type': 'v1', 'addr': '172.21.15.45:6825', 'nonce': 3298886784}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6826', 'nonce': 3298886784}, {'type': 'v1', 'addr': '172.21.15.45:6827', 'nonce': 3298886784}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6830', 'nonce': 3298886784}, {'type': 'v1', 'addr': '172.21.15.45:6831', 'nonce': 3298886784}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6828', 'nonce': 3298886784}, {'type': 'v1', 'addr': '172.21.15.45:6829', 'nonce': 3298886784}]}, 'public_addr': '172.21.15.45:6825/3298886784', 'cluster_addr': '172.21.15.45:6827/3298886784', 'heartbeat_back_addr': '172.21.15.45:6831/3298886784', 'heartbeat_front_addr': '172.21.15.45:6829/3298886784', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b3fda98b-a10a-4354-905e-1b37a5124b98', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 36, 'up_from': 90, 'up_thru': 23, 'down_at': 37, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6816', 'nonce': 724784918}, {'type': 'v1', 'addr': '172.21.15.45:6817', 'nonce': 724784918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6818', 'nonce': 724784918}, {'type': 'v1', 'addr': '172.21.15.45:6819', 'nonce': 724784918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6822', 'nonce': 724784918}, {'type': 'v1', 'addr': '172.21.15.45:6823', 'nonce': 724784918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6820', 'nonce': 724784918}, {'type': 'v1', 'addr': '172.21.15.45:6821', 'nonce': 724784918}]}, 'public_addr': '172.21.15.45:6817/724784918', 'cluster_addr': '172.21.15.45:6819/724784918', 'heartbeat_back_addr': '172.21.15.45:6823/724784918', 'heartbeat_front_addr': '172.21.15.45:6821/724784918', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6ab67b48-2d83-4e68-8898-302a1adc0a52', 'up': 1, 'in': 1, 'weight': 0.783294677734375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6824', 'nonce': 1644784047}, {'type': 'v1', 'addr': '172.21.15.192:6825', 'nonce': 1644784047}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6826', 'nonce': 1644784047}, {'type': 'v1', 'addr': '172.21.15.192:6827', 'nonce': 1644784047}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6830', 'nonce': 1644784047}, {'type': 'v1', 'addr': '172.21.15.192:6831', 'nonce': 1644784047}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6828', 'nonce': 1644784047}, {'type': 'v1', 'addr': '172.21.15.192:6829', 'nonce': 1644784047}]}, 'public_addr': '172.21.15.192:6825/1644784047', 'cluster_addr': '172.21.15.192:6827/1644784047', 'heartbeat_back_addr': '172.21.15.192:6831/1644784047', 'heartbeat_front_addr': '172.21.15.192:6829/1644784047', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ae3c787c-63d2-4e88-8f99-6d4f19837a3a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 4077426583}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 4077426583}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 4077426583}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 4077426583}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 4077426583}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 4077426583}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 4077426583}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 4077426583}]}, 'public_addr': '172.21.15.192:6817/4077426583', 'cluster_addr': '172.21.15.192:6819/4077426583', 'heartbeat_back_addr': '172.21.15.192:6823/4077426583', 'heartbeat_front_addr': '172.21.15.192:6821/4077426583', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'f792f18d-ad9e-49a8-b194-24522398fb25', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 2922229639}, {'type': 'v1', 'addr': '172.21.15.192:6809', 'nonce': 2922229639}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6810', 'nonce': 2922229639}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 2922229639}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6814', 'nonce': 2922229639}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 2922229639}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 2922229639}, {'type': 'v1', 'addr': '172.21.15.192:6813', 'nonce': 2922229639}]}, 'public_addr': '172.21.15.192:6809/2922229639', 'cluster_addr': '172.21.15.192:6811/2922229639', 'heartbeat_back_addr': '172.21.15.192:6815/2922229639', 'heartbeat_front_addr': '172.21.15.192:6813/2922229639', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '4a589d43-b0b9-42de-989c-46928d693c24', 'up': 1, 'in': 1, 'weight': 0.8525848388671875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 780298944}, {'type': 'v1', 'addr': '172.21.15.192:6801', 'nonce': 780298944}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6802', 'nonce': 780298944}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 780298944}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6806', 'nonce': 780298944}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 780298944}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 780298944}, {'type': 'v1', 'addr': '172.21.15.192:6805', 'nonce': 780298944}]}, 'public_addr': '172.21.15.192:6801/780298944', 'cluster_addr': '172.21.15.192:6803/780298944', 'heartbeat_back_addr': '172.21.15.192:6807/780298944', 'heartbeat_front_addr': '172.21.15.192:6805/780298944', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T07:28:20.586514+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T07:28:20.833505+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T07:28:21.059899+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-17T07:28:20.499470+0000', 'dead_epoch': 37}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T07:28:21.353342+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T07:28:20.586673+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T07:28:20.767847+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T07:28:20.859615+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 47, 'length': 2}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 47, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T07:30:06.465 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:30:06.466 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:06.466 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:06.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:06.467 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:06.462+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:06.564 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:06.563+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:06.614 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:06.614 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:06.625 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:06.667 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:06.665+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:06.767 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:06.767+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:06.774 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:06.774 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:06.783 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:06.806 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:30:06.807 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:30:06.819 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-17T07:30:06.769409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2292011591,"num_objects":2329,"num_object_clones":560,"num_object_copies":4658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2329,"num_whiteouts":1075,"num_read":152429,"num_read_kb":1195659,"num_write":20836,"num_write_kb":792863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1204,"num_bytes_recovered":158168283,"num_keys_recovered":162,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16902,"ondisk_log_size":16902,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":444272,"kb_used_data":122564,"kb_used_omap":10,"kb_used_meta":320885,"kb_avail":660158608,"statfs":{"total":676457349120,"available":676002414592,"internally_reserved":0,"allocated":125505536,"data_stored":1295901098,"data_compressed":86775611,"data_compressed_allocated":89694208,"data_compressed_original":1265941805,"omap_allocated":11159,"internal_metadata":328586345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":359932655,"num_objects":90,"num_object_clones":30,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":12,"num_read":6547,"num_read_kb":16019,"num_write":948,"num_write_kb":9740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000671"},"pg_stats":[{"pgid":"3.1f","version":"95'666","reported_seq":3806,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.312967+0000","last_change":"2024-09-17T07:29:15.076965+0000","last_active":"2024-09-17T07:30:04.312967+0000","last_peered":"2024-09-17T07:30:04.312967+0000","last_clean":"2024-09-17T07:30:04.312967+0000","last_became_active":"2024-09-17T07:29:15.075896+0000","last_became_peered":"2024-09-17T07:29:15.075896+0000","last_unstale":"2024-09-17T07:30:04.312967+0000","last_undegraded":"2024-09-17T07:30:04.312967+0000","last_fullsized":"2024-09-17T07:30:04.312967+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:29.990608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013177429999999999,"stat_sum":{"num_bytes":120381287,"num_objects":87,"num_object_clones":23,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":38,"num_read":4929,"num_read_kb":33545,"num_write":812,"num_write_kb":41881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":12072365,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"95'452","reported_seq":2597,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.158901+0000","last_change":"2024-09-17T07:29:43.080474+0000","last_active":"2024-09-17T07:30:04.158901+0000","last_peered":"2024-09-17T07:30:04.158901+0000","last_clean":"2024-09-17T07:30:04.158901+0000","last_became_active":"2024-09-17T07:29:43.079885+0000","last_became_peered":"2024-09-17T07:29:43.079885+0000","last_unstale":"2024-09-17T07:30:04.158901+0000","last_undegraded":"2024-09-17T07:30:04.158901+0000","last_fullsized":"2024-09-17T07:30:04.158901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:00.295437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010259710000000001,"stat_sum":{"num_bytes":62323065,"num_objects":80,"num_object_clones":13,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":47,"num_read":3275,"num_read_kb":29909,"num_write":463,"num_write_kb":15578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":6814699,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"90'474","reported_seq":1977,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.150116+0000","last_change":"2024-09-17T07:28:45.076749+0000","last_active":"2024-09-17T07:30:04.150116+0000","last_peered":"2024-09-17T07:30:04.150116+0000","last_clean":"2024-09-17T07:30:04.150116+0000","last_became_active":"2024-09-17T07:28:43.078355+0000","last_became_peered":"2024-09-17T07:28:43.078355+0000","last_unstale":"2024-09-17T07:30:04.150116+0000","last_undegraded":"2024-09-17T07:30:04.150116+0000","last_fullsized":"2024-09-17T07:30:04.150116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:32.585253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602014,"stat_sum":{"num_bytes":85355155,"num_objects":77,"num_object_clones":16,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":43,"num_read":1877,"num_read_kb":28662,"num_write":531,"num_write_kb":27120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"95'650","reported_seq":4804,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.298639+0000","last_change":"2024-09-17T07:28:45.076729+0000","last_active":"2024-09-17T07:30:04.298639+0000","last_peered":"2024-09-17T07:30:04.298639+0000","last_clean":"2024-09-17T07:30:04.298639+0000","last_became_active":"2024-09-17T07:28:43.078480+0000","last_became_peered":"2024-09-17T07:28:43.078480+0000","last_unstale":"2024-09-17T07:30:04.298639+0000","last_undegraded":"2024-09-17T07:30:04.298639+0000","last_fullsized":"2024-09-17T07:30:04.298639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:11:11.148343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060216499999999999,"stat_sum":{"num_bytes":45256740,"num_objects":55,"num_object_clones":13,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":24,"num_read":6665,"num_read_kb":41612,"num_write":885,"num_write_kb":38700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"95'474","reported_seq":3730,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.150351+0000","last_change":"2024-09-17T07:29:14.076063+0000","last_active":"2024-09-17T07:30:04.150351+0000","last_peered":"2024-09-17T07:30:04.150351+0000","last_clean":"2024-09-17T07:30:04.150351+0000","last_became_active":"2024-09-17T07:29:14.074906+0000","last_became_peered":"2024-09-17T07:29:14.074906+0000","last_unstale":"2024-09-17T07:30:04.150351+0000","last_undegraded":"2024-09-17T07:30:04.150351+0000","last_fullsized":"2024-09-17T07:30:04.150351+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:37:05.416942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013284480000000001,"stat_sum":{"num_bytes":54656619,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":26,"num_read":5097,"num_read_kb":51823,"num_write":511,"num_write_kb":22857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":15469929,"num_keys_recovered":21,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"95'782","reported_seq":13289,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.307287+0000","last_change":"2024-09-17T07:28:45.080739+0000","last_active":"2024-09-17T07:30:04.307287+0000","last_peered":"2024-09-17T07:30:04.307287+0000","last_clean":"2024-09-17T07:30:04.307287+0000","last_became_active":"2024-09-17T07:28:43.076870+0000","last_became_peered":"2024-09-17T07:28:43.076870+0000","last_unstale":"2024-09-17T07:30:04.307287+0000","last_undegraded":"2024-09-17T07:30:04.307287+0000","last_fullsized":"2024-09-17T07:30:04.307287+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:38:43.673644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079048599999999997,"stat_sum":{"num_bytes":93809979,"num_objects":77,"num_object_clones":16,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":30,"num_read":15815,"num_read_kb":60173,"num_write":1089,"num_write_kb":20572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"95'389","reported_seq":4931,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.149877+0000","last_change":"2024-09-17T07:29:43.081894+0000","last_active":"2024-09-17T07:30:04.149877+0000","last_peered":"2024-09-17T07:30:04.149877+0000","last_clean":"2024-09-17T07:30:04.149877+0000","last_became_active":"2024-09-17T07:29:43.080228+0000","last_became_peered":"2024-09-17T07:29:43.080228+0000","last_unstale":"2024-09-17T07:30:04.149877+0000","last_undegraded":"2024-09-17T07:30:04.149877+0000","last_fullsized":"2024-09-17T07:30:04.149877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:39.343209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062180999999999998,"stat_sum":{"num_bytes":51105738,"num_objects":72,"num_object_clones":15,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":40,"num_read":4320,"num_read_kb":37310,"num_write":380,"num_write_kb":43089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":12758046,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"95'599","reported_seq":3437,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.150070+0000","last_change":"2024-09-17T07:29:43.078464+0000","last_active":"2024-09-17T07:30:04.150070+0000","last_peered":"2024-09-17T07:30:04.150070+0000","last_clean":"2024-09-17T07:30:04.150070+0000","last_became_active":"2024-09-17T07:29:43.077187+0000","last_became_peered":"2024-09-17T07:29:43.077187+0000","last_unstale":"2024-09-17T07:30:04.150070+0000","last_undegraded":"2024-09-17T07:30:04.150070+0000","last_fullsized":"2024-09-17T07:30:04.150070+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:14.214051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064275900000000002,"stat_sum":{"num_bytes":71258727,"num_objects":82,"num_object_clones":22,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":36,"num_read":5065,"num_read_kb":24562,"num_write":691,"num_write_kb":16574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":13130874,"num_keys_recovered":82,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"90'348","reported_seq":3513,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-17T07:30:03.152597+0000","last_change":"2024-09-17T07:28:45.088416+0000","last_active":"2024-09-17T07:30:03.152597+0000","last_peered":"2024-09-17T07:30:03.152597+0000","last_clean":"2024-09-17T07:30:03.152597+0000","last_became_active":"2024-09-17T07:28:43.080428+0000","last_became_peered":"2024-09-17T07:28:43.080428+0000","last_unstale":"2024-09-17T07:30:03.152597+0000","last_undegraded":"2024-09-17T07:30:03.152597+0000","last_fullsized":"2024-09-17T07:30:03.152597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:31:55.517417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000830013,"stat_sum":{"num_bytes":60179927,"num_objects":66,"num_object_clones":15,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":37,"num_read":4406,"num_read_kb":22246,"num_write":317,"num_write_kb":13518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"92'371","reported_seq":3227,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.198838+0000","last_change":"2024-09-17T07:28:45.077267+0000","last_active":"2024-09-17T07:30:04.198838+0000","last_peered":"2024-09-17T07:30:04.198838+0000","last_clean":"2024-09-17T07:30:04.198838+0000","last_became_active":"2024-09-17T07:28:43.082313+0000","last_became_peered":"2024-09-17T07:28:43.082313+0000","last_unstale":"2024-09-17T07:30:04.198838+0000","last_undegraded":"2024-09-17T07:30:04.198838+0000","last_fullsized":"2024-09-17T07:30:04.198838+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:25.981720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084992300000000004,"stat_sum":{"num_bytes":46284558,"num_objects":64,"num_object_clones":12,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":27,"num_read":5045,"num_read_kb":57423,"num_write":352,"num_write_kb":31831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"94'307","reported_seq":3452,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.152418+0000","last_change":"2024-09-17T07:29:43.081583+0000","last_active":"2024-09-17T07:30:04.152418+0000","last_peered":"2024-09-17T07:30:04.152418+0000","last_clean":"2024-09-17T07:30:04.152418+0000","last_became_active":"2024-09-17T07:29:43.080880+0000","last_became_peered":"2024-09-17T07:29:43.080880+0000","last_unstale":"2024-09-17T07:30:04.152418+0000","last_undegraded":"2024-09-17T07:30:04.152418+0000","last_fullsized":"2024-09-17T07:30:04.152418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:35.192399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005702,"stat_sum":{"num_bytes":63166129,"num_objects":69,"num_object_clones":18,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":30,"num_read":3398,"num_read_kb":32195,"num_write":329,"num_write_kb":22735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":17427308,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"93'395","reported_seq":1431,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.149906+0000","last_change":"2024-09-17T07:28:45.080921+0000","last_active":"2024-09-17T07:30:04.149906+0000","last_peered":"2024-09-17T07:30:04.149906+0000","last_clean":"2024-09-17T07:30:04.149906+0000","last_became_active":"2024-09-17T07:28:43.081203+0000","last_became_peered":"2024-09-17T07:28:43.081203+0000","last_unstale":"2024-09-17T07:30:04.149906+0000","last_undegraded":"2024-09-17T07:30:04.149906+0000","last_fullsized":"2024-09-17T07:30:04.149906+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:58.766905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091011499999999997,"stat_sum":{"num_bytes":47172180,"num_objects":67,"num_object_clones":19,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":33,"num_read":1367,"num_read_kb":17945,"num_write":449,"num_write_kb":24884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":170,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.158721+0000","last_change":"2024-09-17T07:29:12.072080+0000","last_active":"2024-09-17T07:30:04.158721+0000","last_peered":"2024-09-17T07:30:04.158721+0000","last_clean":"2024-09-17T07:30:04.158721+0000","last_became_active":"2024-09-17T07:29:12.071738+0000","last_became_peered":"2024-09-17T07:29:12.071738+0000","last_unstale":"2024-09-17T07:30:04.158721+0000","last_undegraded":"2024-09-17T07:30:04.158721+0000","last_fullsized":"2024-09-17T07:30:04.158721+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:37.818818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"93'906","reported_seq":2208,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-17T07:30:03.152535+0000","last_change":"2024-09-17T07:28:45.088312+0000","last_active":"2024-09-17T07:30:03.152535+0000","last_peered":"2024-09-17T07:30:03.152535+0000","last_clean":"2024-09-17T07:30:03.152535+0000","last_became_active":"2024-09-17T07:28:43.079229+0000","last_became_peered":"2024-09-17T07:28:43.079229+0000","last_unstale":"2024-09-17T07:30:03.152535+0000","last_undegraded":"2024-09-17T07:30:03.152535+0000","last_fullsized":"2024-09-17T07:30:03.152535+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:25:42.246676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080697400000000002,"stat_sum":{"num_bytes":44549574,"num_objects":70,"num_object_clones":18,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":33,"num_read":4550,"num_read_kb":48690,"num_write":1434,"num_write_kb":29287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":166,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.149873+0000","last_change":"2024-09-17T07:28:33.361412+0000","last_active":"2024-09-17T07:30:04.149873+0000","last_peered":"2024-09-17T07:30:04.149873+0000","last_clean":"2024-09-17T07:30:04.149873+0000","last_became_active":"2024-09-17T07:28:31.362639+0000","last_became_peered":"2024-09-17T07:28:31.362639+0000","last_unstale":"2024-09-17T07:30:04.149873+0000","last_undegraded":"2024-09-17T07:30:04.149873+0000","last_fullsized":"2024-09-17T07:30:04.149873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:42.297390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049448899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"94'466","reported_seq":3075,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.199463+0000","last_change":"2024-09-17T07:29:43.081963+0000","last_active":"2024-09-17T07:30:04.199463+0000","last_peered":"2024-09-17T07:30:04.199463+0000","last_clean":"2024-09-17T07:30:04.199463+0000","last_became_active":"2024-09-17T07:29:43.081197+0000","last_became_peered":"2024-09-17T07:29:43.081197+0000","last_unstale":"2024-09-17T07:30:04.199463+0000","last_undegraded":"2024-09-17T07:30:04.199463+0000","last_fullsized":"2024-09-17T07:30:04.199463+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:20.183400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011396100000000001,"stat_sum":{"num_bytes":101632038,"num_objects":93,"num_object_clones":26,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":40,"num_read":3696,"num_read_kb":36727,"num_write":558,"num_write_kb":22611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":6377674,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":165,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.152543+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:30:04.152543+0000","last_peered":"2024-09-17T07:30:04.152543+0000","last_clean":"2024-09-17T07:30:04.152543+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:30:04.152543+0000","last_undegraded":"2024-09-17T07:30:04.152543+0000","last_fullsized":"2024-09-17T07:30:04.152543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"95'935","reported_seq":2901,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.150007+0000","last_change":"2024-09-17T07:28:45.077202+0000","last_active":"2024-09-17T07:30:04.150007+0000","last_peered":"2024-09-17T07:30:04.150007+0000","last_clean":"2024-09-17T07:30:04.150007+0000","last_became_active":"2024-09-17T07:28:43.239387+0000","last_became_peered":"2024-09-17T07:28:43.239387+0000","last_unstale":"2024-09-17T07:30:04.150007+0000","last_undegraded":"2024-09-17T07:30:04.150007+0000","last_fullsized":"2024-09-17T07:30:04.150007+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:54.764970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000576977,"stat_sum":{"num_bytes":40145198,"num_objects":69,"num_object_clones":17,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":33,"num_read":6367,"num_read_kb":16470,"num_write":1354,"num_write_kb":10787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.150066+0000","last_change":"2024-09-17T07:28:33.361447+0000","last_active":"2024-09-17T07:30:04.150066+0000","last_peered":"2024-09-17T07:30:04.150066+0000","last_clean":"2024-09-17T07:30:04.150066+0000","last_became_active":"2024-09-17T07:28:31.567277+0000","last_became_peered":"2024-09-17T07:28:31.567277+0000","last_unstale":"2024-09-17T07:30:04.150066+0000","last_undegraded":"2024-09-17T07:30:04.150066+0000","last_fullsized":"2024-09-17T07:30:04.150066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:09.089091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056035799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"93'486","reported_seq":2531,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.150129+0000","last_change":"2024-09-17T07:29:43.082601+0000","last_active":"2024-09-17T07:30:04.150129+0000","last_peered":"2024-09-17T07:30:04.150129+0000","last_clean":"2024-09-17T07:30:04.150129+0000","last_became_active":"2024-09-17T07:29:43.080970+0000","last_became_peered":"2024-09-17T07:29:43.080970+0000","last_unstale":"2024-09-17T07:30:04.150129+0000","last_undegraded":"2024-09-17T07:30:04.150129+0000","last_fullsized":"2024-09-17T07:30:04.150129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:23.735383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079629899999999999,"stat_sum":{"num_bytes":78497410,"num_objects":80,"num_object_clones":19,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":36,"num_read":2612,"num_read_kb":46248,"num_write":587,"num_write_kb":27527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":17428070,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"94'400","reported_seq":1712,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.150234+0000","last_change":"2024-09-17T07:28:45.080688+0000","last_active":"2024-09-17T07:30:04.150234+0000","last_peered":"2024-09-17T07:30:04.150234+0000","last_clean":"2024-09-17T07:30:04.150234+0000","last_became_active":"2024-09-17T07:28:43.078767+0000","last_became_peered":"2024-09-17T07:28:43.078767+0000","last_unstale":"2024-09-17T07:30:04.150234+0000","last_undegraded":"2024-09-17T07:30:04.150234+0000","last_fullsized":"2024-09-17T07:30:04.150234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:30.754890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000727296,"stat_sum":{"num_bytes":55804136,"num_objects":71,"num_object_clones":13,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":36,"num_read":1744,"num_read_kb":34919,"num_write":393,"num_write_kb":16532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":168,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.149622+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:30:04.149622+0000","last_peered":"2024-09-17T07:30:04.149622+0000","last_clean":"2024-09-17T07:30:04.149622+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:30:04.149622+0000","last_undegraded":"2024-09-17T07:30:04.149622+0000","last_fullsized":"2024-09-17T07:30:04.149622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"86'749","reported_seq":10844,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.150093+0000","last_change":"2024-09-17T07:28:45.076896+0000","last_active":"2024-09-17T07:30:04.150093+0000","last_peered":"2024-09-17T07:30:04.150093+0000","last_clean":"2024-09-17T07:30:04.150093+0000","last_became_active":"2024-09-17T07:28:43.239409+0000","last_became_peered":"2024-09-17T07:28:43.239409+0000","last_unstale":"2024-09-17T07:30:04.150093+0000","last_undegraded":"2024-09-17T07:30:04.150093+0000","last_fullsized":"2024-09-17T07:30:04.150093+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:56.818168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055657199999999997,"stat_sum":{"num_bytes":121953719,"num_objects":81,"num_object_clones":17,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":44,"num_read":12549,"num_read_kb":91656,"num_write":1194,"num_write_kb":58356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":171,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-17T07:30:03.161088+0000","last_change":"2024-09-17T07:29:40.324743+0000","last_active":"2024-09-17T07:30:03.161088+0000","last_peered":"2024-09-17T07:30:03.161088+0000","last_clean":"2024-09-17T07:30:03.161088+0000","last_became_active":"2024-09-17T07:29:40.324487+0000","last_became_peered":"2024-09-17T07:29:40.324487+0000","last_unstale":"2024-09-17T07:30:03.161088+0000","last_undegraded":"2024-09-17T07:30:03.161088+0000","last_fullsized":"2024-09-17T07:30:03.161088+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:07:57.322104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"93'351","reported_seq":3875,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-17T07:30:03.152388+0000","last_change":"2024-09-17T07:29:43.083597+0000","last_active":"2024-09-17T07:30:03.152388+0000","last_peered":"2024-09-17T07:30:03.152388+0000","last_clean":"2024-09-17T07:30:03.152388+0000","last_became_active":"2024-09-17T07:29:43.082561+0000","last_became_peered":"2024-09-17T07:29:43.082561+0000","last_unstale":"2024-09-17T07:30:03.152388+0000","last_undegraded":"2024-09-17T07:30:03.152388+0000","last_fullsized":"2024-09-17T07:30:03.152388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:37.267897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070743599999999998,"stat_sum":{"num_bytes":59542457,"num_objects":62,"num_object_clones":13,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":25,"num_read":3604,"num_read_kb":51528,"num_write":377,"num_write_kb":27749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":17043150,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":165,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.152177+0000","last_change":"2024-09-17T07:28:33.951319+0000","last_active":"2024-09-17T07:30:04.152177+0000","last_peered":"2024-09-17T07:30:04.152177+0000","last_clean":"2024-09-17T07:30:04.152177+0000","last_became_active":"2024-09-17T07:28:31.364065+0000","last_became_peered":"2024-09-17T07:28:31.364065+0000","last_unstale":"2024-09-17T07:30:04.152177+0000","last_undegraded":"2024-09-17T07:30:04.152177+0000","last_fullsized":"2024-09-17T07:30:04.152177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:40.816064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'317","reported_seq":1745,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.296854+0000","last_change":"2024-09-17T07:29:44.081418+0000","last_active":"2024-09-17T07:30:04.296854+0000","last_peered":"2024-09-17T07:30:04.296854+0000","last_clean":"2024-09-17T07:30:04.296854+0000","last_became_active":"2024-09-17T07:29:44.080665+0000","last_became_peered":"2024-09-17T07:29:44.080665+0000","last_unstale":"2024-09-17T07:30:04.296854+0000","last_undegraded":"2024-09-17T07:30:04.296854+0000","last_fullsized":"2024-09-17T07:30:04.296854+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:51:44.093027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075432199999999998,"stat_sum":{"num_bytes":40262290,"num_objects":60,"num_object_clones":13,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":27,"num_read":2258,"num_read_kb":8662,"num_write":312,"num_write_kb":6305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":3397150,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":167,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.149646+0000","last_change":"2024-09-17T07:28:33.361820+0000","last_active":"2024-09-17T07:30:04.149646+0000","last_peered":"2024-09-17T07:30:04.149646+0000","last_clean":"2024-09-17T07:30:04.149646+0000","last_became_active":"2024-09-17T07:28:31.359703+0000","last_became_peered":"2024-09-17T07:28:31.359703+0000","last_unstale":"2024-09-17T07:30:04.149646+0000","last_undegraded":"2024-09-17T07:30:04.149646+0000","last_fullsized":"2024-09-17T07:30:04.149646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:59.468099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000487639,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":256,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.152395+0000","last_change":"2024-09-17T07:28:26.547435+0000","last_active":"2024-09-17T07:30:04.152395+0000","last_peered":"2024-09-17T07:30:04.152395+0000","last_clean":"2024-09-17T07:30:04.152395+0000","last_became_active":"2024-09-17T07:28:26.546300+0000","last_became_peered":"2024-09-17T07:28:26.546300+0000","last_unstale":"2024-09-17T07:30:04.152395+0000","last_undegraded":"2024-09-17T07:30:04.152395+0000","last_fullsized":"2024-09-17T07:30:04.152395+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:02:23.225041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"95'452","reported_seq":1930,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.149976+0000","last_change":"2024-09-17T07:29:43.082284+0000","last_active":"2024-09-17T07:30:04.149976+0000","last_peered":"2024-09-17T07:30:04.149976+0000","last_clean":"2024-09-17T07:30:04.149976+0000","last_became_active":"2024-09-17T07:29:43.080250+0000","last_became_peered":"2024-09-17T07:29:43.080250+0000","last_unstale":"2024-09-17T07:30:04.149976+0000","last_undegraded":"2024-09-17T07:30:04.149976+0000","last_fullsized":"2024-09-17T07:30:04.149976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:02.814146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057675599999999999,"stat_sum":{"num_bytes":119193038,"num_objects":74,"num_object_clones":21,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":36,"num_read":1593,"num_read_kb":24325,"num_write":502,"num_write_kb":33950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":12317557,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"95'440","reported_seq":2737,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.150224+0000","last_change":"2024-09-17T07:28:45.080237+0000","last_active":"2024-09-17T07:30:04.150224+0000","last_peered":"2024-09-17T07:30:04.150224+0000","last_clean":"2024-09-17T07:30:04.150224+0000","last_became_active":"2024-09-17T07:28:43.239554+0000","last_became_peered":"2024-09-17T07:28:43.239554+0000","last_unstale":"2024-09-17T07:30:04.150224+0000","last_undegraded":"2024-09-17T07:30:04.150224+0000","last_fullsized":"2024-09-17T07:30:04.150224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:40.608791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000366203,"stat_sum":{"num_bytes":55853496,"num_objects":75,"num_object_clones":21,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":31,"num_read":4052,"num_read_kb":20685,"num_write":472,"num_write_kb":12184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"95'570","reported_seq":7557,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.242449+0000","last_change":"2024-09-17T07:28:45.554483+0000","last_active":"2024-09-17T07:30:04.242449+0000","last_peered":"2024-09-17T07:30:04.242449+0000","last_clean":"2024-09-17T07:30:04.242449+0000","last_became_active":"2024-09-17T07:28:43.076652+0000","last_became_peered":"2024-09-17T07:28:43.076652+0000","last_unstale":"2024-09-17T07:30:04.242449+0000","last_undegraded":"2024-09-17T07:30:04.242449+0000","last_fullsized":"2024-09-17T07:30:04.242449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:29:30.466139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031203200000000001,"stat_sum":{"num_bytes":143555262,"num_objects":102,"num_object_clones":28,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":49,"num_read":8723,"num_read_kb":51898,"num_write":584,"num_write_kb":21066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"94'469","reported_seq":2256,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-17T07:30:03.288422+0000","last_change":"2024-09-17T07:29:43.084922+0000","last_active":"2024-09-17T07:30:03.288422+0000","last_peered":"2024-09-17T07:30:03.288422+0000","last_clean":"2024-09-17T07:30:03.288422+0000","last_became_active":"2024-09-17T07:29:43.083061+0000","last_became_peered":"2024-09-17T07:29:43.083061+0000","last_unstale":"2024-09-17T07:30:03.288422+0000","last_undegraded":"2024-09-17T07:30:03.288422+0000","last_fullsized":"2024-09-17T07:30:03.288422+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:45.843085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079276200000000002,"stat_sum":{"num_bytes":78825628,"num_objects":65,"num_object_clones":17,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":27,"num_read":2185,"num_read_kb":29658,"num_write":607,"num_write_kb":15593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":3965421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"95'728","reported_seq":4515,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.247594+0000","last_change":"2024-09-17T07:29:44.082752+0000","last_active":"2024-09-17T07:30:04.247594+0000","last_peered":"2024-09-17T07:30:04.247594+0000","last_clean":"2024-09-17T07:30:04.247594+0000","last_became_active":"2024-09-17T07:29:44.080788+0000","last_became_peered":"2024-09-17T07:29:44.080788+0000","last_unstale":"2024-09-17T07:30:04.247594+0000","last_undegraded":"2024-09-17T07:30:04.247594+0000","last_fullsized":"2024-09-17T07:30:04.247594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:44.226665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080871800000000005,"stat_sum":{"num_bytes":30104086,"num_objects":76,"num_object_clones":16,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":29,"num_read":5740,"num_read_kb":27404,"num_write":1000,"num_write_kb":14703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":7797775,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"95'570","reported_seq":3746,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.276573+0000","last_change":"2024-09-17T07:28:45.078543+0000","last_active":"2024-09-17T07:30:04.276573+0000","last_peered":"2024-09-17T07:30:04.276573+0000","last_clean":"2024-09-17T07:30:04.276573+0000","last_became_active":"2024-09-17T07:28:43.080816+0000","last_became_peered":"2024-09-17T07:28:43.080816+0000","last_unstale":"2024-09-17T07:30:04.276573+0000","last_undegraded":"2024-09-17T07:30:04.276573+0000","last_fullsized":"2024-09-17T07:30:04.276573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:46:43.676382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052799200000000002,"stat_sum":{"num_bytes":113291038,"num_objects":68,"num_object_clones":18,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":26,"num_read":5233,"num_read_kb":48299,"num_write":729,"num_write_kb":31189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"95'378","reported_seq":7083,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.302488+0000","last_change":"2024-09-17T07:29:12.182769+0000","last_active":"2024-09-17T07:30:04.302488+0000","last_peered":"2024-09-17T07:30:04.302488+0000","last_clean":"2024-09-17T07:30:04.302488+0000","last_became_active":"2024-09-17T07:29:12.076559+0000","last_became_peered":"2024-09-17T07:29:12.076559+0000","last_unstale":"2024-09-17T07:30:04.302488+0000","last_undegraded":"2024-09-17T07:30:04.302488+0000","last_fullsized":"2024-09-17T07:30:04.302488+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:07:28.898021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072888599999999999,"stat_sum":{"num_bytes":110124036,"num_objects":71,"num_object_clones":21,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":35,"num_read":6887,"num_read_kb":55464,"num_write":358,"num_write_kb":41313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":12168265,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"90'704","reported_seq":4355,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.152399+0000","last_change":"2024-09-17T07:28:45.077964+0000","last_active":"2024-09-17T07:30:04.152399+0000","last_peered":"2024-09-17T07:30:04.152399+0000","last_clean":"2024-09-17T07:30:04.152399+0000","last_became_active":"2024-09-17T07:28:43.081133+0000","last_became_peered":"2024-09-17T07:28:43.081133+0000","last_unstale":"2024-09-17T07:30:04.152399+0000","last_undegraded":"2024-09-17T07:30:04.152399+0000","last_fullsized":"2024-09-17T07:30:04.152399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:02.649206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001137037,"stat_sum":{"num_bytes":66601976,"num_objects":81,"num_object_clones":18,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":40,"num_read":6001,"num_read_kb":22818,"num_write":837,"num_write_kb":18442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"94'309","reported_seq":2107,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.158715+0000","last_change":"2024-09-17T07:28:45.554473+0000","last_active":"2024-09-17T07:30:04.158715+0000","last_peered":"2024-09-17T07:30:04.158715+0000","last_clean":"2024-09-17T07:30:04.158715+0000","last_became_active":"2024-09-17T07:28:43.077360+0000","last_became_peered":"2024-09-17T07:28:43.077360+0000","last_unstale":"2024-09-17T07:30:04.158715+0000","last_undegraded":"2024-09-17T07:30:04.158715+0000","last_fullsized":"2024-09-17T07:30:04.158715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:22.496648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030798799999999998,"stat_sum":{"num_bytes":38876108,"num_objects":64,"num_object_clones":14,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":28,"num_read":2480,"num_read_kb":25893,"num_write":396,"num_write_kb":12223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"94'674","reported_seq":2547,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.307704+0000","last_change":"2024-09-17T07:28:45.078484+0000","last_active":"2024-09-17T07:30:04.307704+0000","last_peered":"2024-09-17T07:30:04.307704+0000","last_clean":"2024-09-17T07:30:04.307704+0000","last_became_active":"2024-09-17T07:28:43.080785+0000","last_became_peered":"2024-09-17T07:28:43.080785+0000","last_unstale":"2024-09-17T07:30:04.307704+0000","last_undegraded":"2024-09-17T07:30:04.307704+0000","last_fullsized":"2024-09-17T07:30:04.307704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:42.283877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076953700000000004,"stat_sum":{"num_bytes":77310727,"num_objects":63,"num_object_clones":17,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":29,"num_read":4554,"num_read_kb":44563,"num_write":859,"num_write_kb":26398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"88'325","reported_seq":1531,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-17T07:30:03.152437+0000","last_change":"2024-09-17T07:28:45.088053+0000","last_active":"2024-09-17T07:30:03.152437+0000","last_peered":"2024-09-17T07:30:03.152437+0000","last_clean":"2024-09-17T07:30:03.152437+0000","last_became_active":"2024-09-17T07:28:43.081466+0000","last_became_peered":"2024-09-17T07:28:43.081466+0000","last_unstale":"2024-09-17T07:30:03.152437+0000","last_undegraded":"2024-09-17T07:30:03.152437+0000","last_fullsized":"2024-09-17T07:30:03.152437+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:26.673024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067203100000000002,"stat_sum":{"num_bytes":54836951,"num_objects":79,"num_object_clones":21,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":36,"num_read":1861,"num_read_kb":44241,"num_write":355,"num_write_kb":29078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"93'691","reported_seq":2888,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-17T07:30:04.152475+0000","last_change":"2024-09-17T07:28:45.078262+0000","last_active":"2024-09-17T07:30:04.152475+0000","last_peered":"2024-09-17T07:30:04.152475+0000","last_clean":"2024-09-17T07:30:04.152475+0000","last_became_active":"2024-09-17T07:28:43.080664+0000","last_became_peered":"2024-09-17T07:28:43.080664+0000","last_unstale":"2024-09-17T07:30:04.152475+0000","last_undegraded":"2024-09-17T07:30:04.152475+0000","last_fullsized":"2024-09-17T07:30:04.152475+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:28.283196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066264099999999995,"stat_sum":{"num_bytes":59511932,"num_objects":58,"num_object_clones":14,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":24,"num_read":4415,"num_read_kb":28051,"num_write":1016,"num_write_kb":16601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2291421204,"num_objects":2325,"num_object_clones":560,"num_object_copies":4650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2325,"num_whiteouts":1075,"num_read":152363,"num_read_kb":1195604,"num_write":20733,"num_write_kb":791233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1204,"num_bytes_recovered":158168283,"num_keys_recovered":162,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":120508416,"data_stored":1334075352,"data_compressed":89048574,"data_compressed_allocated":92028928,"data_compressed_original":1307075422,"omap_allocated":7902,"internal_metadata":0},"log_size":16827,"ondisk_log_size":16827,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65196,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74032,"kb_used_data":24128,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94297808,"statfs":{"total":96636764160,"available":96560955392,"internally_reserved":0,"allocated":24707072,"data_stored":278893670,"data_compressed":18689122,"data_compressed_allocated":18444288,"data_compressed_original":273963674,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45328,"kb_used_data":9972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326512,"statfs":{"total":96636764160,"available":96590348288,"internally_reserved":0,"allocated":10211328,"data_stored":88401385,"data_compressed":5943445,"data_compressed_allocated":6344704,"data_compressed_original":84977795,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68484,"kb_used_data":19252,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94303356,"statfs":{"total":96636764160,"available":96566636544,"internally_reserved":0,"allocated":19714048,"data_stored":215829709,"data_compressed":14356201,"data_compressed_allocated":14479360,"data_compressed_original":211334093,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81252,"kb_used_data":23696,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94290588,"statfs":{"total":96636764160,"available":96553562112,"internally_reserved":0,"allocated":24264704,"data_stored":217376802,"data_compressed":15010170,"data_compressed_allocated":16920576,"data_compressed_original":211258761,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":0,"seq":386547056642,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542166,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40524,"kb_used_data":13492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331316,"statfs":{"total":96636764160,"available":96595267584,"internally_reserved":0,"allocated":13815808,"data_stored":142828870,"data_compressed":9719070,"data_compressed_allocated":10067968,"data_compressed_original":139728117,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87044,"kb_used_data":15600,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94284796,"statfs":{"total":96636764160,"available":96547631104,"internally_reserved":0,"allocated":15974400,"data_stored":175550575,"data_compressed":11734136,"data_compressed_allocated":11558912,"data_compressed_original":172140284,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47608,"kb_used_data":16424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324232,"statfs":{"total":96636764160,"available":96588013568,"internally_reserved":0,"allocated":16818176,"data_stored":177020087,"data_compressed":11323467,"data_compressed_allocated":11878400,"data_compressed_original":172539081,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15659008,"data_stored":175943343,"data_compressed":11294113,"data_compressed_allocated":11833344,"data_compressed_original":171987598,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14958592,"data_stored":175140411,"data_compressed":11734136,"data_compressed_allocated":11558912,"data_compressed_original":172140284,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12705792,"data_stored":142314412,"data_compressed":9719070,"data_compressed_allocated":10067968,"data_compressed_original":139728117,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2805760,"data_stored":42339805,"data_compressed":2323573,"data_compressed_allocated":2420736,"data_compressed_original":42025068,"omap_allocated":7902,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":23158784,"data_stored":217100192,"data_compressed":15015351,"data_compressed_allocated":16920576,"data_compressed_original":211365257,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":18608128,"data_stored":215315285,"data_compressed":14356201,"data_compressed_allocated":14479360,"data_compressed_original":211334093,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":9027584,"data_stored":88043365,"data_compressed":5949606,"data_compressed_allocated":6348800,"data_compressed_original":85104771,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":23584768,"data_stored":277878539,"data_compressed":18656524,"data_compressed_allocated":18399232,"data_compressed_original":273390234,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:30:06.821 INFO:tasks.thrashosds.thrasher:Setting 2.4 to [5, 3] 2024-09-17T07:30:06.821 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.4', '5', '3'] 2024-09-17T07:30:06.822 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.4 5 3 2024-09-17T07:30:06.891 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:06.891 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:06.891 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:06.889+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:06.899 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:06.973 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:06.971+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:07.048 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:07.048 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:07.057 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:07.065 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:30:07.073 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:07.072+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:07.162 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-17T07:30:07.174 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:07.173+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:07.175 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:07.176 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:07.188 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:07.237 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:07.259 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:07.275 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:07.274+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:07.276 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:07.276 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:07.293 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:07.310 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:07.310 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:07.327 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:07.327 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:07.341 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:07.341 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:07.341 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:07.341 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:07.344 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:07.344 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:07.350 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:07.362 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:07.362 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:07.376 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:07.375+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:07.469 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:07.469 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:07.469 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:07.469 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:07.469 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:07.478 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:07.478 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:07.476+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:07.578 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:07.577+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:07.628 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:07.628 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:07.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:07.629 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:07.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.629 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.629 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:07.629 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493866836999999, 2024-09-17T07:30:07.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:30:07.629 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.629 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.629 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.629 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.629 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.629 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.629 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:30:07.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.630 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:07.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:07.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:30:07.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:30:07.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:30:07.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:30:07.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:30:07.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:30:07.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:30:07.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:30:07.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:30:07.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:30:07.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:30:07.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:30:07.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:30:07.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:30:07.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:30:07.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:30:07.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:30:07.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:30:07.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:30:07.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:30:07.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:30:07.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:30:07.634 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.634 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.634 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.635 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.635 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:07.635 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493829939999998, 2024-09-17T07:30:07.635 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:30:07.635 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.635 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.635 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.635 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.635 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.635 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:30:07.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.635 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:07.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:07.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:30:07.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:30:07.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:30:07.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:30:07.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:30:07.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:30:07.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:30:07.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:30:07.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:30:07.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:30:07.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:07.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:30:07.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:30:07.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:30:07.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:30:07.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:30:07.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:30:07.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:30:07.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:30:07.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:30:07.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:30:07.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:30:07.640 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.640 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.640 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.640 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.640 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:07.640 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493803972999999, 2024-09-17T07:30:07.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:30:07.641 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.641 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.641 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.641 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.641 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.641 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.641 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:30:07.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.641 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:07.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:30:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:30:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:30:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:30:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:30:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:30:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:30:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:30:07.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:30:07.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.644 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:30:07.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:30:07.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.644 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:30:07.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:30:07.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:30:07.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:30:07.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:30:07.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:30:07.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:30:07.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:30:07.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:30:07.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:30:07.646 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.646 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.646 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.646 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.646 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:07.646 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493779377999999, 2024-09-17T07:30:07.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:30:07.646 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.646 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.646 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.647 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.647 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.647 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.647 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:30:07.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.647 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.647 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:07.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.647 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:07.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.648 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:07.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:30:07.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.648 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:30:07.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:30:07.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.648 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:30:07.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:30:07.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:30:07.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:30:07.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:30:07.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:30:07.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:30:07.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:30:07.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:30:07.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:30:07.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:30:07.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:30:07.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:30:07.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:30:07.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:30:07.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:30:07.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:30:07.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:30:07.651 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.652 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.652 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.652 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.652 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:07.652 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493755891000006, 2024-09-17T07:30:07.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:30:07.652 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.652 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.652 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.652 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.652 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.653 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:30:07.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.653 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:07.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:07.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:30:07.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:07.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:30:07.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:30:07.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:30:07.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:30:07.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:30:07.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:30:07.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:30:07.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:30:07.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:07.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:30:07.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:30:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:30:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:30:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:30:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:30:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:30:07.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:30:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:30:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:30:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:30:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493742152999999, 2024-09-17T07:30:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:30:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:30:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:07.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:07.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:07.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:30:07.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:30:07.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:30:07.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:30:07.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:30:07.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:30:07.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:30:07.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:07.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:30:07.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:07.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:30:07.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:30:07.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:30:07.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:30:07.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:30:07.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:30:07.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:30:07.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:30:07.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:30:07.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:30:07.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:30:07.663 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.663 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.663 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.663 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.663 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:07.663 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493711963999999, 2024-09-17T07:30:07.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:30:07.664 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.664 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.664 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.664 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.664 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.664 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.664 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:30:07.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.664 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:07.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:07.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:30:07.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:30:07.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:30:07.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:30:07.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:07.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:30:07.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:30:07.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:30:07.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:30:07.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:30:07.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:30:07.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:30:07.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:30:07.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:30:07.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:30:07.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:07.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:30:07.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:30:07.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:30:07.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:30:07.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:30:07.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:30:07.670 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.670 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.670 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.670 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.670 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:07.670 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493675459000002, 2024-09-17T07:30:07.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:30:07.670 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.670 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.670 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.670 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.671 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.671 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:30:07.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.671 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.671 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:07.671 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.671 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:07.671 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:07.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:30:07.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:30:07.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:30:07.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:30:07.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:30:07.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.673 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:30:07.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:30:07.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.673 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:30:07.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:30:07.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.673 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:07.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:30:07.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:30:07.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:30:07.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:30:07.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:30:07.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:30:07.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:30:07.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.675 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.675 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:30:07.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:30:07.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.675 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.675 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:30:07.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:30:07.675 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.675 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.675 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.676 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.676 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:07.676 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493647699999997, 2024-09-17T07:30:07.676 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:30:07.676 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.676 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.676 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.676 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.676 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.676 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.676 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:30:07.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.676 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.677 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.677 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:07.677 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.677 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.677 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:07.677 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.677 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.677 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:30:07.677 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:30:07.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.678 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.678 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:30:07.678 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:30:07.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.678 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.678 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:30:07.678 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:30:07.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.678 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.678 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:30:07.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:30:07.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:07.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:30:07.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:30:07.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:30:07.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:30:07.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:30:07.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:30:07.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:07.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:30:07.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:30:07.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.681 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:30:07.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:30:07.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.681 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.681 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:30:07.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:30:07.681 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.681 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.681 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.681 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.681 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:07.682 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493625539999996, 2024-09-17T07:30:07.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:30:07.682 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.682 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.682 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.682 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.682 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.682 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:30:07.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.682 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:07.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.683 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:07.683 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.683 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:07.683 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:30:07.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.684 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:30:07.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:30:07.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.684 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.684 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:30:07.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:07.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.684 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.684 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:30:07.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:30:07.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.684 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.685 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:30:07.685 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:30:07.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.685 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.685 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:30:07.685 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:30:07.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.685 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.685 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:30:07.685 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:30:07.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.686 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:30:07.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:30:07.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.686 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:30:07.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:30:07.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.686 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:30:07.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:30:07.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:30:07.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:30:07.687 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.687 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.687 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.687 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.687 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:07.687 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493612378999998, 2024-09-17T07:30:07.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:30:07.687 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.687 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.688 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.688 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.688 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.688 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.688 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:30:07.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.688 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:07.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.689 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:07.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.689 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:07.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:30:07.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.689 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:30:07.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:30:07.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.690 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:30:07.690 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:30:07.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.690 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.690 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:30:07.690 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:30:07.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.690 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.690 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:30:07.690 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:30:07.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.690 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.691 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:07.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:30:07.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.691 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.691 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:30:07.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:30:07.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.691 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.691 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:30:07.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:30:07.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.692 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.692 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:30:07.692 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:30:07.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.692 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.692 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:30:07.692 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:30:07.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.692 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.692 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:30:07.692 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:30:07.692 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.692 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.693 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.693 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:07.693 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493593790999995, 2024-09-17T07:30:07.693 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:30:07.693 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.693 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.693 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.693 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.693 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.693 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:30:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:30:07.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:30:07.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:30:07.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:30:07.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:07.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:30:07.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:07.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:30:07.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:07.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:30:07.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:07.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:30:07.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.697 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.697 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:30:07.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:30:07.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.697 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.697 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:30:07.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:30:07.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.697 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.697 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:30:07.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:30:07.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.698 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.698 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.698 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:30:07.698 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:30:07.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.698 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.698 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.698 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:30:07.698 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:30:07.698 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.698 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.698 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.699 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.699 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:07.699 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493560979999998, 2024-09-17T07:30:07.699 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:30:07.699 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.699 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.699 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.699 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.699 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.699 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.699 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:30:07.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.699 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.700 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.700 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:07.700 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.700 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.700 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.700 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:07.700 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.700 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.700 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.700 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:07.700 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:30:07.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.701 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.701 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:30:07.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:30:07.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.701 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.701 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:30:07.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:30:07.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.701 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.701 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:30:07.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:30:07.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.702 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.702 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:30:07.702 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:30:07.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.702 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.702 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:07.702 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:07.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.702 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.702 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:30:07.702 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:30:07.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.703 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.703 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:30:07.703 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:07.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.703 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.703 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:30:07.703 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:30:07.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.703 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.703 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:30:07.704 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:30:07.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.704 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.704 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.704 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:30:07.704 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:30:07.704 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.704 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.704 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.704 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.704 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.704 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:07.704 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493541922999995, 2024-09-17T07:30:07.704 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:30:07.705 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.705 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.705 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.705 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.705 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.705 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.705 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:30:07.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.705 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.705 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.705 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:07.705 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.706 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.706 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:07.706 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.706 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.706 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:30:07.706 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:30:07.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.706 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.706 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:30:07.706 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:30:07.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.707 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.707 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:30:07.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:30:07.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.707 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.707 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:30:07.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:30:07.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.707 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.707 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:30:07.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:30:07.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.708 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.708 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.708 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:30:07.708 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:30:07.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.708 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.708 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.708 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:30:07.708 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:30:07.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.708 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.708 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.709 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:30:07.709 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:30:07.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.709 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.709 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:30:07.709 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:30:07.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.709 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.709 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:30:07.709 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:30:07.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.709 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.710 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:30:07.710 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:30:07.710 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.710 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.710 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.710 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.710 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:07.710 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493519813999995, 2024-09-17T07:30:07.710 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:30:07.710 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.710 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.710 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.711 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.711 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.711 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.711 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:30:07.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.711 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.711 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.711 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.711 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:07.711 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.711 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.711 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.711 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:07.711 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.712 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.712 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.712 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:30:07.712 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:30:07.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.712 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.712 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.712 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:07.712 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:30:07.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.712 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.712 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.712 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:30:07.713 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:30:07.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.713 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.713 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:30:07.713 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:07.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.713 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.713 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:30:07.713 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:30:07.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.713 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.713 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:30:07.714 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:30:07.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.714 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.714 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:30:07.714 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:30:07.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.714 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.714 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:30:07.714 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:30:07.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.714 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.715 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:30:07.715 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:30:07.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.715 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.715 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.715 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:30:07.715 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:30:07.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.715 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.715 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.715 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:30:07.715 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:30:07.715 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.715 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.715 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.716 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.716 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.716 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:07.716 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493493706999999, 2024-09-17T07:30:07.716 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:30:07.716 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.716 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.716 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.716 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.716 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.716 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.716 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:30:07.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.717 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.717 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.717 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.717 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:07.717 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.717 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.717 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.717 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:07.717 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.717 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.717 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.717 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:30:07.717 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:30:07.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.718 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.718 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.718 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:07.718 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:30:07.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.718 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.718 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.718 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:30:07.718 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:07.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.718 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.718 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.718 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:30:07.718 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:30:07.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.719 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.719 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.719 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:30:07.719 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:30:07.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.719 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.719 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.719 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:30:07.719 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:30:07.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.719 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.719 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.719 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:30:07.720 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:30:07.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.720 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.720 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.720 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:30:07.720 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:30:07.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.720 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.720 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.720 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:30:07.720 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:30:07.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.720 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.720 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.720 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:30:07.721 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:30:07.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.721 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.721 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.721 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:30:07.721 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:30:07.721 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.721 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.721 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.721 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.721 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.721 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:07.721 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493464562, 2024-09-17T07:30:07.722 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:30:07.722 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.722 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.722 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.722 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.722 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.722 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.722 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:30:07.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.722 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.722 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.722 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.722 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:07.722 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.723 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.723 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:07.723 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.723 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.723 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:07.723 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:30:07.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.723 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.723 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:30:07.723 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:07.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.724 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.724 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:30:07.724 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:30:07.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.724 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.724 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:30:07.724 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:30:07.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.724 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.724 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:30:07.724 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:30:07.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.725 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.725 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.725 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:30:07.725 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:30:07.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.725 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.725 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.725 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:30:07.725 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:30:07.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.725 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.725 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.725 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:30:07.726 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:30:07.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.726 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.726 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:30:07.726 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:30:07.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.726 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.726 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:30:07.726 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:30:07.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.726 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.727 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:30:07.727 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:30:07.727 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.727 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.727 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.727 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.727 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.727 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:07.727 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493432759000001, 2024-09-17T07:30:07.727 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:30:07.727 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.727 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.727 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.727 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.728 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.728 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.728 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:30:07.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.728 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.728 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.728 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.728 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:07.728 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.728 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.728 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.728 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:07.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.729 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.729 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:30:07.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:30:07.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.729 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.729 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:07.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:30:07.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.729 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:30:07.730 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:30:07.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.730 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:30:07.730 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:30:07.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.730 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:30:07.730 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:30:07.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.730 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:30:07.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:30:07.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.731 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:30:07.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:30:07.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.731 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:30:07.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:30:07.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.732 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:30:07.732 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:30:07.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.732 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:30:07.732 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:30:07.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.732 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:30:07.732 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:30:07.732 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.733 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.733 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.733 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.733 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.733 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:07.733 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493405812000006, 2024-09-17T07:30:07.734 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:30:07.734 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.734 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.734 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.734 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.734 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.734 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:30:07.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.734 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.734 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.734 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.734 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:07.734 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:07.735 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:30:07.735 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:30:07.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:30:07.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:30:07.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.736 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.736 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:07.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:30:07.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.736 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.736 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:07.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:30:07.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.736 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.737 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:30:07.737 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:30:07.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.737 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.737 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:30:07.737 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:30:07.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.737 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.737 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:30:07.737 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:30:07.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.738 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.738 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:30:07.738 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:30:07.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.738 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.738 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:30:07.738 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:30:07.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.738 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.738 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:30:07.738 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:30:07.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.739 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.739 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.739 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:30:07.739 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:30:07.739 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.739 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.739 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.739 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.739 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:07.740 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:07.740 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.493364862999996, 2024-09-17T07:30:07.740 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:30:07.740 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:07.740 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:07.740 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:07.740 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:07.740 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:07.740 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:07.740 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:30:07.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.740 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:07.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.740 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:07.741 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:07.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.741 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:07.741 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:07.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:07.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.741 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:07.741 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:30:07.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:30:07.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.742 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:07.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:30:07.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:30:07.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.742 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:07.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:30:07.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:30:07.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.742 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:07.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:30:07.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:30:07.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:30:07.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:30:07.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:07.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:30:07.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:30:07.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:30:07.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:30:07.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:07.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:30:07.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:30:07.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:07.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:30:07.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:30:07.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:07.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:30:07.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:30:07.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:07.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:07.745 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:07.745 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:30:07.745 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:30:07.745 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.745 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.745 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.745 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:07.745 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:07.745 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:07.747 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:07.747 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:07.746+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:07.847 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:07.846+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:07.895 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:07.895 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:07.895 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:07.895 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:07.903 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:07.948 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:07.947+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:08.049 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:08.048+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:08.050 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:08.050 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:08.050 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:08.051 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:08.051 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:08.059 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:08.150 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:08.149+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:08.166 INFO:teuthology.orchestra.run.smithi045.stderr:set 2.4 pg_upmap mapping to [5,3] 2024-09-17T07:30:08.171 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17069 3.0 e41/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 41'110, mlcod=40'108)", 2024-09-17T07:30:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.429810893999999, 2024-09-17T07:30:08.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011408908000000001, 2024-09-17T07:30:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:08.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739753+0000", 2024-09-17T07:30:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:30:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743532+0000", 2024-09-17T07:30:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037794510000000001 2024-09-17T07:30:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.173 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743557+0000", 2024-09-17T07:30:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5253e-05 2024-09-17T07:30:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743569+0000", 2024-09-17T07:30:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1161999999999999e-05 2024-09-17T07:30:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743650+0000", 2024-09-17T07:30:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1087999999999993e-05 2024-09-17T07:30:08.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743674+0000", 2024-09-17T07:30:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4562e-05 2024-09-17T07:30:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751095+0000", 2024-09-17T07:30:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074211140000000004 2024-09-17T07:30:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.175 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751161+0000", 2024-09-17T07:30:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5707e-05 2024-09-17T07:30:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17240 3.0 e42/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 42'113, mlcod=41'110)", 2024-09-17T07:30:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 64.944152410000001, 2024-09-17T07:30:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064392440000000002, 2024-09-17T07:30:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.176 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.177 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:30:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225418+0000", 2024-09-17T07:30:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8360000000000003e-06 2024-09-17T07:30:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225426+0000", 2024-09-17T07:30:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2670000000000006e-06 2024-09-17T07:30:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.179 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225434+0000", 2024-09-17T07:30:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0560000000000005e-06 2024-09-17T07:30:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.180 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225514+0000", 2024-09-17T07:30:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9979000000000001e-05 2024-09-17T07:30:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.181 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225535+0000", 2024-09-17T07:30:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0965e-05 2024-09-17T07:30:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231797+0000", 2024-09-17T07:30:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062621129999999997 2024-09-17T07:30:08.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231850+0000", 2024-09-17T07:30:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2611999999999997e-05 2024-09-17T07:30:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.183 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:20054 3.9 e44/23 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:head v 44'131, mlcod=44'129)", 2024-09-17T07:30:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:30:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 62.902001294000002, 2024-09-17T07:30:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028510570000000002, 2024-09-17T07:30:08.184 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:30:08.185 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267562+0000", 2024-09-17T07:30:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.267563+0000", 2024-09-17T07:30:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.584e-06 2024-09-17T07:30:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.187 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268269+0000", 2024-09-17T07:30:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00070567399999999999 2024-09-17T07:30:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268287+0000", 2024-09-17T07:30:08.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.84e-05 2024-09-17T07:30:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268297+0000", 2024-09-17T07:30:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9059999999999997e-06 2024-09-17T07:30:08.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268368+0000", 2024-09-17T07:30:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0562999999999995e-05 2024-09-17T07:30:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.190 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.268393+0000", 2024-09-17T07:30:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4958999999999999e-05 2024-09-17T07:30:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270330+0000", 2024-09-17T07:30:08.191 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001936644 2024-09-17T07:30:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:05.270413+0000", 2024-09-17T07:30:08.192 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3326999999999997e-05 2024-09-17T07:30:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.193 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:32543 3.0 e55/23 3:03bfd97d:::rbd_data.11839a97b753.0000000000000007:head v 55'150, mlcod=54'148)", 2024-09-17T07:30:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.714926157000001, 2024-09-17T07:30:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031813480000000001, 2024-09-17T07:30:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.194 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5199999999999998e-07 2024-09-17T07:30:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455083+0000", 2024-09-17T07:30:08.197 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044629199999999999 2024-09-17T07:30:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455098+0000", 2024-09-17T07:30:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4183e-05 2024-09-17T07:30:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.198 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455106+0000", 2024-09-17T07:30:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:30:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455173+0000", 2024-09-17T07:30:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6346000000000005e-05 2024-09-17T07:30:08.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455200+0000", 2024-09-17T07:30:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7319000000000001e-05 2024-09-17T07:30:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457774+0000", 2024-09-17T07:30:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025742650000000001 2024-09-17T07:30:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457818+0000", 2024-09-17T07:30:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3891999999999999e-05 2024-09-17T07:30:08.204 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.205 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33735 3.1f e55/52 3:fea54097:::rbd_data.11839a97b753.0000000000000000:head v 55'260, mlcod=55'258)", 2024-09-17T07:30:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.344993529, 2024-09-17T07:30:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035215200000000002, 2024-09-17T07:30:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.206 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824571+0000", 2024-09-17T07:30:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.164e-06 2024-09-17T07:30:08.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825118+0000", 2024-09-17T07:30:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054696400000000004 2024-09-17T07:30:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.209 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825134+0000", 2024-09-17T07:30:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6342999999999999e-05 2024-09-17T07:30:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825142+0000", 2024-09-17T07:30:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5270000000000001e-06 2024-09-17T07:30:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825200+0000", 2024-09-17T07:30:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.893e-05 2024-09-17T07:30:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.211 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825218+0000", 2024-09-17T07:30:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7935999999999999e-05 2024-09-17T07:30:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.212 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828037+0000", 2024-09-17T07:30:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002818961 2024-09-17T07:30:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828091+0000", 2024-09-17T07:30:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3695e-05 2024-09-17T07:30:08.213 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33869 3.9 e55/23 3:907b81b5:::rbd_data.11839a97b753.0000000000000001:head v 55'171, mlcod=55'169)", 2024-09-17T07:30:08.214 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.327937706, 2024-09-17T07:30:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033990859999999999, 2024-09-17T07:30:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.215 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.217 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841626+0000", 2024-09-17T07:30:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.023e-06 2024-09-17T07:30:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.218 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842067+0000", 2024-09-17T07:30:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000440612 2024-09-17T07:30:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842080+0000", 2024-09-17T07:30:08.219 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.29e-05 2024-09-17T07:30:08.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.220 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.220 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842086+0000", 2024-09-17T07:30:08.220 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2090000000000002e-06 2024-09-17T07:30:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842144+0000", 2024-09-17T07:30:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7627999999999998e-05 2024-09-17T07:30:08.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842171+0000", 2024-09-17T07:30:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7214000000000001e-05 2024-09-17T07:30:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.222 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.844973+0000", 2024-09-17T07:30:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002802232 2024-09-17T07:30:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.223 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.845024+0000", 2024-09-17T07:30:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1267999999999998e-05 2024-09-17T07:30:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51432 3.0 e59/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 59'177, mlcod=58'174)", 2024-09-17T07:30:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 45.348340688999997, 2024-09-17T07:30:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010790086000000001, 2024-09-17T07:30:08.225 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.226 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:08.227 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.228 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.228 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821224+0000", 2024-09-17T07:30:08.228 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.511e-06 2024-09-17T07:30:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825174+0000", 2024-09-17T07:30:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0039497949999999999 2024-09-17T07:30:08.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825196+0000", 2024-09-17T07:30:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2643999999999999e-05 2024-09-17T07:30:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825206+0000", 2024-09-17T07:30:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4320000000000005e-06 2024-09-17T07:30:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825272+0000", 2024-09-17T07:30:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5829999999999998e-05 2024-09-17T07:30:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825299+0000", 2024-09-17T07:30:08.232 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7107e-05 2024-09-17T07:30:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.831939+0000", 2024-09-17T07:30:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0066398510000000004 2024-09-17T07:30:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.832012+0000", 2024-09-17T07:30:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3916000000000002e-05 2024-09-17T07:30:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51969 3.0 e59/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 59'179, mlcod=59'177)", 2024-09-17T07:30:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 45.239472311, 2024-09-17T07:30:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060297249999999997, 2024-09-17T07:30:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930093+0000", 2024-09-17T07:30:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.764e-06 2024-09-17T07:30:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931779+0000", 2024-09-17T07:30:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001686379 2024-09-17T07:30:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931802+0000", 2024-09-17T07:30:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2682000000000001e-05 2024-09-17T07:30:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931811+0000", 2024-09-17T07:30:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5640000000000002e-06 2024-09-17T07:30:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931877+0000", 2024-09-17T07:30:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6155999999999995e-05 2024-09-17T07:30:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931896+0000", 2024-09-17T07:30:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9111e-05 2024-09-17T07:30:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936063+0000", 2024-09-17T07:30:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004166711 2024-09-17T07:30:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936121+0000", 2024-09-17T07:30:08.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7358000000000002e-05 2024-09-17T07:30:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54417 3.0 e60/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 60'182, mlcod=59'179)", 2024-09-17T07:30:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 44.731955319999997, 2024-09-17T07:30:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0059885219999999996, 2024-09-17T07:30:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437609+0000", 2024-09-17T07:30:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5829999999999999e-06 2024-09-17T07:30:08.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437623+0000", 2024-09-17T07:30:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3348e-05 2024-09-17T07:30:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437643+0000", 2024-09-17T07:30:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0463999999999999e-05 2024-09-17T07:30:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437653+0000", 2024-09-17T07:30:08.245 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6290000000000005e-06 2024-09-17T07:30:08.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.245 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.245 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437719+0000", 2024-09-17T07:30:08.245 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6428999999999997e-05 2024-09-17T07:30:08.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.245 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.245 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437745+0000", 2024-09-17T07:30:08.245 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5690999999999998e-05 2024-09-17T07:30:08.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.245 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443549+0000", 2024-09-17T07:30:08.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0058038259999999998 2024-09-17T07:30:08.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443596+0000", 2024-09-17T07:30:08.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7552000000000001e-05 2024-09-17T07:30:08.246 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.246 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.246 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.246 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54419 3.0 e60/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 60'184, mlcod=59'179)", 2024-09-17T07:30:08.246 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:08.247 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 44.725602963, 2024-09-17T07:30:08.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004706844, 2024-09-17T07:30:08.247 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.247 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.247 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.247 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:08.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:08.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.248 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.248 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443961+0000", 2024-09-17T07:30:08.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:30:08.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.248 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.248 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443967+0000", 2024-09-17T07:30:08.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.268e-06 2024-09-17T07:30:08.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.249 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.249 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443983+0000", 2024-09-17T07:30:08.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5248999999999999e-05 2024-09-17T07:30:08.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.249 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.249 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443990+0000", 2024-09-17T07:30:08.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5120000000000002e-06 2024-09-17T07:30:08.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.249 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.249 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444056+0000", 2024-09-17T07:30:08.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6197000000000003e-05 2024-09-17T07:30:08.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444081+0000", 2024-09-17T07:30:08.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4791e-05 2024-09-17T07:30:08.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448616+0000", 2024-09-17T07:30:08.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045346129999999998 2024-09-17T07:30:08.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448667+0000", 2024-09-17T07:30:08.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-17T07:30:08.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.251 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.251 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.251 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54895 3.1f e60/52 3:fa1c1177:::rbd_data.1183ad87379e.0000000000000001:head v 60'283, mlcod=60'281)", 2024-09-17T07:30:08.251 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:30:08.251 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 44.378121966999998, 2024-09-17T07:30:08.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0029495210000000001, 2024-09-17T07:30:08.251 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.251 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.251 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:30:08.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791441+0000", 2024-09-17T07:30:08.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.791443+0000", 2024-09-17T07:30:08.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.153e-06 2024-09-17T07:30:08.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792619+0000", 2024-09-17T07:30:08.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001176011 2024-09-17T07:30:08.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792637+0000", 2024-09-17T07:30:08.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-05 2024-09-17T07:30:08.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792649+0000", 2024-09-17T07:30:08.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1712000000000001e-05 2024-09-17T07:30:08.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792722+0000", 2024-09-17T07:30:08.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3545000000000001e-05 2024-09-17T07:30:08.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.792738+0000", 2024-09-17T07:30:08.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5223e-05 2024-09-17T07:30:08.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.794330+0000", 2024-09-17T07:30:08.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015922740000000001 2024-09-17T07:30:08.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.794391+0000", 2024-09-17T07:30:08.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0803e-05 2024-09-17T07:30:08.255 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.255 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.255 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.255 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:80794 3.9 e72/23 3:90311a61:::rbd_data.1183cb746bb8.0000000000000007:head v 72'222, mlcod=71'220)", 2024-09-17T07:30:08.255 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:08.255 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 30.737614079, 2024-09-17T07:30:08.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004371733, 2024-09-17T07:30:08.256 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.256 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.256 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:08.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:08.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.257 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:08.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0800000000000001e-07 2024-09-17T07:30:08.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.257 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434108+0000", 2024-09-17T07:30:08.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021587569999999999 2024-09-17T07:30:08.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.258 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434120+0000", 2024-09-17T07:30:08.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1821000000000001e-05 2024-09-17T07:30:08.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.258 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.258 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434128+0000", 2024-09-17T07:30:08.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7579999999999992e-06 2024-09-17T07:30:08.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.258 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.258 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434191+0000", 2024-09-17T07:30:08.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3202999999999998e-05 2024-09-17T07:30:08.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434218+0000", 2024-09-17T07:30:08.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6771000000000002e-05 2024-09-17T07:30:08.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436289+0000", 2024-09-17T07:30:08.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020717700000000001 2024-09-17T07:30:08.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436321+0000", 2024-09-17T07:30:08.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1445000000000001e-05 2024-09-17T07:30:08.260 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.260 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.260 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.260 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 75/74 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9, version: 36'125, data_included: [5478708~26316,5767168~524288,6553600~497972], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9@36'125, size: 10246384, copy_subset: [(0, 10246384)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7051572, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5478708, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:08.260 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:41.719003+0000", 2024-09-17T07:30:08.260 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 26.450559916, 2024-09-17T07:30:08.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025840260000000001, 2024-09-17T07:30:08.260 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.260 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.260 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:08.260 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.261 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.261 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.719003+0000", 2024-09-17T07:30:08.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.261 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.261 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.719003+0000", 2024-09-17T07:30:08.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.261 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.261 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.719004+0000", 2024-09-17T07:30:08.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0699999999999996e-07 2024-09-17T07:30:08.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.720057+0000", 2024-09-17T07:30:08.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010526800000000001 2024-09-17T07:30:08.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.720083+0000", 2024-09-17T07:30:08.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6237e-05 2024-09-17T07:30:08.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.720102+0000", 2024-09-17T07:30:08.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8918000000000001e-05 2024-09-17T07:30:08.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.720207+0000", 2024-09-17T07:30:08.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010558800000000001 2024-09-17T07:30:08.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.720224+0000", 2024-09-17T07:30:08.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6426000000000002e-05 2024-09-17T07:30:08.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.721587+0000", 2024-09-17T07:30:08.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00136337 2024-09-17T07:30:08.264 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.264 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.264 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.264 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 75/74 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head, version: 36'126, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head@36'126, size: 9431460, copy_subset: [], clone_subset: {3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9: [(0, 9431460)]}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:08.264 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:08.264 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 26.428706485999999, 2024-09-17T07:30:08.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041288289999999997, 2024-09-17T07:30:08.264 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.265 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:08.265 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:08.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:08.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740858+0000", 2024-09-17T07:30:08.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1599999999999999e-06 2024-09-17T07:30:08.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740885+0000", 2024-09-17T07:30:08.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.707e-05 2024-09-17T07:30:08.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740925+0000", 2024-09-17T07:30:08.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0649000000000003e-05 2024-09-17T07:30:08.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740957+0000", 2024-09-17T07:30:08.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1585999999999999e-05 2024-09-17T07:30:08.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741185+0000", 2024-09-17T07:30:08.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022759700000000001 2024-09-17T07:30:08.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741228+0000", 2024-09-17T07:30:08.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3705999999999998e-05 2024-09-17T07:30:08.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.744985+0000", 2024-09-17T07:30:08.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037570609999999999 2024-09-17T07:30:08.268 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.268 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.268 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.268 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85061 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'257, mlcod=79'255)", 2024-09-17T07:30:08.268 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:08.268 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 23.020875530000001, 2024-09-17T07:30:08.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033850550000000001, 2024-09-17T07:30:08.269 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.269 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.269 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.269 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:08.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:08.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:08.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9699999999999995e-07 2024-09-17T07:30:08.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148835+0000", 2024-09-17T07:30:08.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000146852 2024-09-17T07:30:08.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148845+0000", 2024-09-17T07:30:08.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0232e-05 2024-09-17T07:30:08.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148852+0000", 2024-09-17T07:30:08.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6490000000000002e-06 2024-09-17T07:30:08.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148913+0000", 2024-09-17T07:30:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1099999999999994e-05 2024-09-17T07:30:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148931+0000", 2024-09-17T07:30:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7938e-05 2024-09-17T07:30:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152025+0000", 2024-09-17T07:30:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0030942729999999998 2024-09-17T07:30:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152073+0000", 2024-09-17T07:30:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7213999999999999e-05 2024-09-17T07:30:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85063 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'259, mlcod=79'255)", 2024-09-17T07:30:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 23.019728575999999, 2024-09-17T07:30:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037722659999999998, 2024-09-17T07:30:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:08.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5799999999999999e-07 2024-09-17T07:30:08.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149841+0000", 2024-09-17T07:30:08.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4840000000000003e-06 2024-09-17T07:30:08.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149849+0000", 2024-09-17T07:30:08.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4670000000000004e-06 2024-09-17T07:30:08.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149854+0000", 2024-09-17T07:30:08.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4779999999999999e-06 2024-09-17T07:30:08.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149920+0000", 2024-09-17T07:30:08.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5760999999999996e-05 2024-09-17T07:30:08.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149949+0000", 2024-09-17T07:30:08.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8838000000000001e-05 2024-09-17T07:30:08.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153558+0000", 2024-09-17T07:30:08.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036092939999999999 2024-09-17T07:30:08.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153607+0000", 2024-09-17T07:30:08.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8386000000000001e-05 2024-09-17T07:30:08.277 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.277 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.277 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.278 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85085 3.5 e79/76 3:a50fe76b:::rbd_data.1183c869c667.0000000000000035:head v 79'401, mlcod=78'399)", 2024-09-17T07:30:08.278 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.151174+0000", 2024-09-17T07:30:08.278 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 23.018388778999999, 2024-09-17T07:30:08.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025105449999999999, 2024-09-17T07:30:08.278 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.278 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.278 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151174+0000", 2024-09-17T07:30:08.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151174+0000", 2024-09-17T07:30:08.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151175+0000", 2024-09-17T07:30:08.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9999999999999997e-07 2024-09-17T07:30:08.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151180+0000", 2024-09-17T07:30:08.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2650000000000004e-06 2024-09-17T07:30:08.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151189+0000", 2024-09-17T07:30:08.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0780000000000002e-06 2024-09-17T07:30:08.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151196+0000", 2024-09-17T07:30:08.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1600000000000003e-06 2024-09-17T07:30:08.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151267+0000", 2024-09-17T07:30:08.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1916999999999996e-05 2024-09-17T07:30:08.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.151300+0000", 2024-09-17T07:30:08.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2975000000000002e-05 2024-09-17T07:30:08.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153644+0000", 2024-09-17T07:30:08.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00234329 2024-09-17T07:30:08.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153685+0000", 2024-09-17T07:30:08.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1159999999999999e-05 2024-09-17T07:30:08.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.282 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.282 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:94996 3.0 e82/23 3:074b4f56:::rbd_data.118311e0d6f8.0000000000000001:head v 82'715, mlcod=80'713)", 2024-09-17T07:30:08.282 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:08.282 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.553822983, 2024-09-17T07:30:08.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031499179999999998, 2024-09-17T07:30:08.282 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.283 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:08.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:08.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615741+0000", 2024-09-17T07:30:08.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999996e-07 2024-09-17T07:30:08.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616917+0000", 2024-09-17T07:30:08.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011763310000000001 2024-09-17T07:30:08.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616936+0000", 2024-09-17T07:30:08.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8624e-05 2024-09-17T07:30:08.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616948+0000", 2024-09-17T07:30:08.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.238e-05 2024-09-17T07:30:08.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617054+0000", 2024-09-17T07:30:08.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000106556 2024-09-17T07:30:08.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617081+0000", 2024-09-17T07:30:08.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.622e-05 2024-09-17T07:30:08.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618836+0000", 2024-09-17T07:30:08.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001755412 2024-09-17T07:30:08.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618890+0000", 2024-09-17T07:30:08.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3915000000000001e-05 2024-09-17T07:30:08.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.286 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.287 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:95007 3.5 e82/76 3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head v 82'422, mlcod=82'420)", 2024-09-17T07:30:08.287 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:08.287 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.545900282000002, 2024-09-17T07:30:08.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003939901, 2024-09-17T07:30:08.287 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.287 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.287 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:08.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:08.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623664+0000", 2024-09-17T07:30:08.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3799999999999999e-06 2024-09-17T07:30:08.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626104+0000", 2024-09-17T07:30:08.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002440149 2024-09-17T07:30:08.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626120+0000", 2024-09-17T07:30:08.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5962000000000001e-05 2024-09-17T07:30:08.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626127+0000", 2024-09-17T07:30:08.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8340000000000001e-06 2024-09-17T07:30:08.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626186+0000", 2024-09-17T07:30:08.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9234999999999998e-05 2024-09-17T07:30:08.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626208+0000", 2024-09-17T07:30:08.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1719e-05 2024-09-17T07:30:08.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627567+0000", 2024-09-17T07:30:08.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013587919999999999 2024-09-17T07:30:08.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627603+0000", 2024-09-17T07:30:08.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5830000000000001e-05 2024-09-17T07:30:08.291 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.291 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.291 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.291 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:104928 3.18 e87/76 3:1ef990b1:::rbd_data.1183dd51323.0000000000000006:head v 87'562, mlcod=86'559)", 2024-09-17T07:30:08.291 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:08.291 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 12.627144571000001, 2024-09-17T07:30:08.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0061095869999999997, 2024-09-17T07:30:08.291 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:08.291 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:08.292 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.292 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:08.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:08.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:08.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:08.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:08.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:08.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:08.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542421+0000", 2024-09-17T07:30:08.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.069e-06 2024-09-17T07:30:08.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:08.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544190+0000", 2024-09-17T07:30:08.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017693539999999999 2024-09-17T07:30:08.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:08.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544215+0000", 2024-09-17T07:30:08.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5398999999999999e-05 2024-09-17T07:30:08.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544229+0000", 2024-09-17T07:30:08.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3865e-05 2024-09-17T07:30:08.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:08.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544322+0000", 2024-09-17T07:30:08.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2540000000000005e-05 2024-09-17T07:30:08.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:08.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544348+0000", 2024-09-17T07:30:08.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6197999999999999e-05 2024-09-17T07:30:08.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:08.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548475+0000", 2024-09-17T07:30:08.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041266000000000002 2024-09-17T07:30:08.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:08.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:08.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:08.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548528+0000", 2024-09-17T07:30:08.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3562e-05 2024-09-17T07:30:08.295 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.295 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.295 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.295 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:08.295 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:08.296 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:08.297 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:08.298 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:08.296+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:08.398 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:08.397+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:08.402 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:08.402 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:08.403 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:08.403 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:08.413 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:08.499 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:08.498+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:08.517 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:08.518 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:08.526 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:08.600 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:08.598+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:08.632 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:30525 3.1b 3:def7c8f0:::rbd_data.118381ffbbab.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T07:30:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.108581671000003, 2024-09-17T07:30:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55515850300000003, 2024-09-17T07:30:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30525 2024-09-17T07:30:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522300+0000", 2024-09-17T07:30:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:30:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522305+0000", 2024-09-17T07:30:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-17T07:30:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522318+0000", 2024-09-17T07:30:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3030999999999999e-05 2024-09-17T07:30:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522326+0000", 2024-09-17T07:30:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4200000000000001e-06 2024-09-17T07:30:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522385+0000", 2024-09-17T07:30:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9005000000000002e-05 2024-09-17T07:30:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522393+0000", 2024-09-17T07:30:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-06 2024-09-17T07:30:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075347+0000", 2024-09-17T07:30:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55295417800000002 2024-09-17T07:30:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075502+0000", 2024-09-17T07:30:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154272 2024-09-17T07:30:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075630+0000", 2024-09-17T07:30:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012816300000000001 2024-09-17T07:30:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077221+0000", 2024-09-17T07:30:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015912929999999999 2024-09-17T07:30:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077353+0000", 2024-09-17T07:30:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013215699999999999 2024-09-17T07:30:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077413+0000", 2024-09-17T07:30:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0034999999999997e-05 2024-09-17T07:30:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077457+0000", 2024-09-17T07:30:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4017999999999998e-05 2024-09-17T07:30:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44181 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13922304~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.993403022999999, 2024-09-17T07:30:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046010502000000002, 2024-09-17T07:30:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44181 2024-09-17T07:30:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637478+0000", 2024-09-17T07:30:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:30:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637480+0000", 2024-09-17T07:30:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.843e-06 2024-09-17T07:30:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637487+0000", 2024-09-17T07:30:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6159999999999996e-06 2024-09-17T07:30:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637492+0000", 2024-09-17T07:30:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5360000000000003e-06 2024-09-17T07:30:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683351+0000", 2024-09-17T07:30:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045858758999999999 2024-09-17T07:30:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683422+0000", 2024-09-17T07:30:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1705999999999994e-05 2024-09-17T07:30:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683488+0000", 2024-09-17T07:30:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5612999999999997e-05 2024-09-17T07:30:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44182 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13938688~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.993382351999998, 2024-09-17T07:30:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046153118999999999, 2024-09-17T07:30:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44182 2024-09-17T07:30:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:30:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637506+0000", 2024-09-17T07:30:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1520000000000003e-06 2024-09-17T07:30:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637512+0000", 2024-09-17T07:30:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-09-17T07:30:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637517+0000", 2024-09-17T07:30:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-09-17T07:30:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683514+0000", 2024-09-17T07:30:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045997616999999998 2024-09-17T07:30:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683584+0000", 2024-09-17T07:30:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9942000000000002e-05 2024-09-17T07:30:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683651+0000", 2024-09-17T07:30:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6951000000000003e-05 2024-09-17T07:30:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44183 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13955072~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.993357119999999, 2024-09-17T07:30:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046339793999999997, 2024-09-17T07:30:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44183 2024-09-17T07:30:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639583+0000", 2024-09-17T07:30:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002059412 2024-09-17T07:30:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639597+0000", 2024-09-17T07:30:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3832000000000001e-05 2024-09-17T07:30:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639607+0000", 2024-09-17T07:30:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485e-05 2024-09-17T07:30:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639614+0000", 2024-09-17T07:30:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0849999999999999e-06 2024-09-17T07:30:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683705+0000", 2024-09-17T07:30:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044091261 2024-09-17T07:30:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683792+0000", 2024-09-17T07:30:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6197000000000002e-05 2024-09-17T07:30:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683863+0000", 2024-09-17T07:30:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1521999999999997e-05 2024-09-17T07:30:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44199 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14217216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.989839568000001, 2024-09-17T07:30:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045995623999999999, 2024-09-17T07:30:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44199 2024-09-17T07:30:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:30:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641043+0000", 2024-09-17T07:30:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.624e-06 2024-09-17T07:30:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641048+0000", 2024-09-17T07:30:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4029999999999998e-06 2024-09-17T07:30:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641053+0000", 2024-09-17T07:30:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6670000000000004e-06 2024-09-17T07:30:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686918+0000", 2024-09-17T07:30:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045865131000000003 2024-09-17T07:30:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686986+0000", 2024-09-17T07:30:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8299000000000005e-05 2024-09-17T07:30:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687036+0000", 2024-09-17T07:30:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0164999999999998e-05 2024-09-17T07:30:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44200 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14233600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.989821739, 2024-09-17T07:30:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046183984999999997, 2024-09-17T07:30:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44200 2024-09-17T07:30:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641061+0000", 2024-09-17T07:30:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.767e-06 2024-09-17T07:30:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641067+0000", 2024-09-17T07:30:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7379999999999999e-06 2024-09-17T07:30:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641071+0000", 2024-09-17T07:30:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-09-17T07:30:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687078+0000", 2024-09-17T07:30:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046007095999999997 2024-09-17T07:30:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687158+0000", 2024-09-17T07:30:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0374e-05 2024-09-17T07:30:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687243+0000", 2024-09-17T07:30:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4221999999999994e-05 2024-09-17T07:30:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44201 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14249984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.989804093000004, 2024-09-17T07:30:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046303182999999998, 2024-09-17T07:30:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44201 2024-09-17T07:30:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641120+0000", 2024-09-17T07:30:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3933000000000001e-05 2024-09-17T07:30:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641123+0000", 2024-09-17T07:30:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.191e-06 2024-09-17T07:30:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641133+0000", 2024-09-17T07:30:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4020000000000008e-06 2024-09-17T07:30:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641138+0000", 2024-09-17T07:30:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6559999999999997e-06 2024-09-17T07:30:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687279+0000", 2024-09-17T07:30:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046140278999999999 2024-09-17T07:30:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687338+0000", 2024-09-17T07:30:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9268999999999998e-05 2024-09-17T07:30:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687379+0000", 2024-09-17T07:30:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1452999999999998e-05 2024-09-17T07:30:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44202 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14266368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.989733327000003, 2024-09-17T07:30:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046484299, 2024-09-17T07:30:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44202 2024-09-17T07:30:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642810+0000", 2024-09-17T07:30:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016627980000000001 2024-09-17T07:30:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642815+0000", 2024-09-17T07:30:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.091e-06 2024-09-17T07:30:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642829+0000", 2024-09-17T07:30:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4047e-05 2024-09-17T07:30:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642844+0000", 2024-09-17T07:30:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4515000000000001e-05 2024-09-17T07:30:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687426+0000", 2024-09-17T07:30:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044582201000000002 2024-09-17T07:30:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687529+0000", 2024-09-17T07:30:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103078 2024-09-17T07:30:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687631+0000", 2024-09-17T07:30:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102569 2024-09-17T07:30:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44209 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14381056~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.987835175000001, 2024-09-17T07:30:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045944444000000001, 2024-09-17T07:30:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44209 2024-09-17T07:30:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643046+0000", 2024-09-17T07:30:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:30:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643048+0000", 2024-09-17T07:30:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:30:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643054+0000", 2024-09-17T07:30:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.032e-06 2024-09-17T07:30:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643065+0000", 2024-09-17T07:30:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1861999999999999e-05 2024-09-17T07:30:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688806+0000", 2024-09-17T07:30:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045740617999999997 2024-09-17T07:30:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688927+0000", 2024-09-17T07:30:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000121327 2024-09-17T07:30:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688990+0000", 2024-09-17T07:30:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2293999999999997e-05 2024-09-17T07:30:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44210 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14397440~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.987808248999997, 2024-09-17T07:30:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046104635999999997, 2024-09-17T07:30:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44210 2024-09-17T07:30:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643073+0000", 2024-09-17T07:30:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.77e-07 2024-09-17T07:30:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643075+0000", 2024-09-17T07:30:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-09-17T07:30:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643082+0000", 2024-09-17T07:30:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7789999999999996e-06 2024-09-17T07:30:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643087+0000", 2024-09-17T07:30:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.967e-06 2024-09-17T07:30:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689022+0000", 2024-09-17T07:30:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045934349999999999 2024-09-17T07:30:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689112+0000", 2024-09-17T07:30:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0039999999999999e-05 2024-09-17T07:30:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689177+0000", 2024-09-17T07:30:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5041000000000003e-05 2024-09-17T07:30:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44211 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14413824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.987787116, 2024-09-17T07:30:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046309003000000001, 2024-09-17T07:30:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44211 2024-09-17T07:30:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643094+0000", 2024-09-17T07:30:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:30:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643095+0000", 2024-09-17T07:30:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-17T07:30:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643101+0000", 2024-09-17T07:30:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8730000000000004e-06 2024-09-17T07:30:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643106+0000", 2024-09-17T07:30:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6859999999999997e-06 2024-09-17T07:30:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689220+0000", 2024-09-17T07:30:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046113945000000003 2024-09-17T07:30:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689309+0000", 2024-09-17T07:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9560000000000003e-05 2024-09-17T07:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689402+0000", 2024-09-17T07:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2786000000000002e-05 2024-09-17T07:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44212 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14430208~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.987742554, 2024-09-17T07:30:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046448912000000002, 2024-09-17T07:30:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44212 2024-09-17T07:30:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2200000000000004e-07 2024-09-17T07:30:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643141+0000", 2024-09-17T07:30:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5210000000000001e-06 2024-09-17T07:30:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643149+0000", 2024-09-17T07:30:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6490000000000008e-06 2024-09-17T07:30:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643161+0000", 2024-09-17T07:30:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2241e-05 2024-09-17T07:30:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689423+0000", 2024-09-17T07:30:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046262323000000001 2024-09-17T07:30:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689505+0000", 2024-09-17T07:30:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2103999999999999e-05 2024-09-17T07:30:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689587+0000", 2024-09-17T07:30:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1452000000000006e-05 2024-09-17T07:30:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44213 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14446592~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.987711054999998, 2024-09-17T07:30:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046601780000000002, 2024-09-17T07:30:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44213 2024-09-17T07:30:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643170+0000", 2024-09-17T07:30:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-17T07:30:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643172+0000", 2024-09-17T07:30:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1660000000000001e-06 2024-09-17T07:30:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643180+0000", 2024-09-17T07:30:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-09-17T07:30:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643185+0000", 2024-09-17T07:30:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-09-17T07:30:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689622+0000", 2024-09-17T07:30:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046436742000000003 2024-09-17T07:30:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689697+0000", 2024-09-17T07:30:08.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5179999999999995e-05 2024-09-17T07:30:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689771+0000", 2024-09-17T07:30:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3928000000000001e-05 2024-09-17T07:30:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44214 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14462976~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:08.693 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.987687995000002, 2024-09-17T07:30:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046736038000000001, 2024-09-17T07:30:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44214 2024-09-17T07:30:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643193+0000", 2024-09-17T07:30:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:30:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643195+0000", 2024-09-17T07:30:08.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0640000000000001e-06 2024-09-17T07:30:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643201+0000", 2024-09-17T07:30:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2859999999999996e-06 2024-09-17T07:30:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643206+0000", 2024-09-17T07:30:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-17T07:30:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689814+0000", 2024-09-17T07:30:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046607742000000001 2024-09-17T07:30:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689886+0000", 2024-09-17T07:30:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2020999999999994e-05 2024-09-17T07:30:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689928+0000", 2024-09-17T07:30:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2771000000000001e-05 2024-09-17T07:30:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.697 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44215 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14479360~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.987644097999997, 2024-09-17T07:30:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046868543999999998, 2024-09-17T07:30:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44215 2024-09-17T07:30:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.698 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643237+0000", 2024-09-17T07:30:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:30:08.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643245+0000", 2024-09-17T07:30:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3270000000000001e-06 2024-09-17T07:30:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643254+0000", 2024-09-17T07:30:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6049999999999998e-06 2024-09-17T07:30:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643260+0000", 2024-09-17T07:30:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7849999999999997e-06 2024-09-17T07:30:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689961+0000", 2024-09-17T07:30:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046701196 2024-09-17T07:30:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690036+0000", 2024-09-17T07:30:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5153000000000004e-05 2024-09-17T07:30:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690105+0000", 2024-09-17T07:30:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8874000000000005e-05 2024-09-17T07:30:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44216 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14495744~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.987612384999998, 2024-09-17T07:30:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047003047999999999, 2024-09-17T07:30:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.702 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44216 2024-09-17T07:30:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643269+0000", 2024-09-17T07:30:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:30:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643271+0000", 2024-09-17T07:30:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-09-17T07:30:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643278+0000", 2024-09-17T07:30:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8850000000000001e-06 2024-09-17T07:30:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643282+0000", 2024-09-17T07:30:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5979999999999999e-06 2024-09-17T07:30:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690133+0000", 2024-09-17T07:30:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850470999999998 2024-09-17T07:30:08.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690205+0000", 2024-09-17T07:30:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2442999999999997e-05 2024-09-17T07:30:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690271+0000", 2024-09-17T07:30:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5874e-05 2024-09-17T07:30:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44217 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14512128~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.987591373000001, 2024-09-17T07:30:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047166485000000001, 2024-09-17T07:30:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44217 2024-09-17T07:30:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-17T07:30:08.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643291+0000", 2024-09-17T07:30:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.962e-06 2024-09-17T07:30:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643297+0000", 2024-09-17T07:30:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0700000000000003e-06 2024-09-17T07:30:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643302+0000", 2024-09-17T07:30:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:30:08.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690308+0000", 2024-09-17T07:30:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047005354999999999 2024-09-17T07:30:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690385+0000", 2024-09-17T07:30:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7541999999999997e-05 2024-09-17T07:30:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690455+0000", 2024-09-17T07:30:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0328999999999997e-05 2024-09-17T07:30:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44218 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14528512~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.987571696000003, 2024-09-17T07:30:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047321015000000001, 2024-09-17T07:30:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.711 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44218 2024-09-17T07:30:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:08.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-17T07:30:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643311+0000", 2024-09-17T07:30:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.933e-06 2024-09-17T07:30:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643318+0000", 2024-09-17T07:30:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3980000000000004e-06 2024-09-17T07:30:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643322+0000", 2024-09-17T07:30:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7609999999999998e-06 2024-09-17T07:30:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690484+0000", 2024-09-17T07:30:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047161428999999998 2024-09-17T07:30:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690564+0000", 2024-09-17T07:30:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9845000000000005e-05 2024-09-17T07:30:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690630+0000", 2024-09-17T07:30:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6021999999999999e-05 2024-09-17T07:30:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44219 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14544896~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.715 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.987551328999999, 2024-09-17T07:30:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048128708999999999, 2024-09-17T07:30:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44219 2024-09-17T07:30:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:30:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643337+0000", 2024-09-17T07:30:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.109e-06 2024-09-17T07:30:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643350+0000", 2024-09-17T07:30:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3046e-05 2024-09-17T07:30:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643356+0000", 2024-09-17T07:30:08.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8930000000000002e-06 2024-09-17T07:30:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690660+0000", 2024-09-17T07:30:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047304061000000001 2024-09-17T07:30:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691372+0000", 2024-09-17T07:30:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00071281299999999997 2024-09-17T07:30:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691458+0000", 2024-09-17T07:30:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5352000000000005e-05 2024-09-17T07:30:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44220 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14561280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.987517281000002, 2024-09-17T07:30:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048309849000000002, 2024-09-17T07:30:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:08.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44220 2024-09-17T07:30:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:08.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646869+0000", 2024-09-17T07:30:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003506267 2024-09-17T07:30:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646874+0000", 2024-09-17T07:30:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7500000000000003e-06 2024-09-17T07:30:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646887+0000", 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3115999999999999e-05 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646894+0000", 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5289999999999997e-06 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691547+0000", 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044652743000000002 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:08.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691622+0000", 2024-09-17T07:30:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5037999999999995e-05 2024-09-17T07:30:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691673+0000", 2024-09-17T07:30:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-17T07:30:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:08.724 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:08.726 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:08.726 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:08.725+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:08.826 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:08.825+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:08.871 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:08.874 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:08.874 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:08.874 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:08.878 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:08.931 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:08.930+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:08.988 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:08.989 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:08.989 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:08.989 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:08.989 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:08.998 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:09.040 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:09.038+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:09.129 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:09.127+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:09.144 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:09.144 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:09.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:09.144 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:09.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.144 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19985 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5275648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.144 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:09.144 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.890321501000003, 2024-09-17T07:30:09.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071819998999999995, 2024-09-17T07:30:09.145 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.145 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.145 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.145 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19985 2024-09-17T07:30:09.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.145 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:09.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:09.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:09.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:09.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252493+0000", 2024-09-17T07:30:09.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6300000000000004e-07 2024-09-17T07:30:09.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252496+0000", 2024-09-17T07:30:09.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-17T07:30:09.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252499+0000", 2024-09-17T07:30:09.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:30:09.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324036+0000", 2024-09-17T07:30:09.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071536587999999998 2024-09-17T07:30:09.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324118+0000", 2024-09-17T07:30:09.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2336999999999995e-05 2024-09-17T07:30:09.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324312+0000", 2024-09-17T07:30:09.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000193709 2024-09-17T07:30:09.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.149 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.149 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.149 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19986 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5505024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.149 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:09.149 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.890310349000003, 2024-09-17T07:30:09.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072083761999999996, 2024-09-17T07:30:09.150 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.150 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.150 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.150 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.150 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19986 2024-09-17T07:30:09.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.150 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:09.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:09.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:09.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:30:09.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252508+0000", 2024-09-17T07:30:09.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:30:09.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252511+0000", 2024-09-17T07:30:09.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.145e-06 2024-09-17T07:30:09.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252514+0000", 2024-09-17T07:30:09.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5569999999999998e-06 2024-09-17T07:30:09.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324326+0000", 2024-09-17T07:30:09.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071812681000000003 2024-09-17T07:30:09.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324372+0000", 2024-09-17T07:30:09.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5893000000000003e-05 2024-09-17T07:30:09.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324587+0000", 2024-09-17T07:30:09.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214608 2024-09-17T07:30:09.154 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.154 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.154 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.154 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19987 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5734400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.154 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:09.154 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.890296730999999, 2024-09-17T07:30:09.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072276676999999998, 2024-09-17T07:30:09.154 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.154 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.154 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19987 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:09.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:09.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252518+0000", 2024-09-17T07:30:09.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-17T07:30:09.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252522+0000", 2024-09-17T07:30:09.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2440000000000004e-06 2024-09-17T07:30:09.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252525+0000", 2024-09-17T07:30:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5919999999999999e-06 2024-09-17T07:30:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324621+0000", 2024-09-17T07:30:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072096442999999996 2024-09-17T07:30:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324699+0000", 2024-09-17T07:30:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8061999999999999e-05 2024-09-17T07:30:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324793+0000", 2024-09-17T07:30:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4207000000000001e-05 2024-09-17T07:30:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19988 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.890285759999998, 2024-09-17T07:30:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072585798000000007, 2024-09-17T07:30:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19988 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:09.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:09.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252529+0000", 2024-09-17T07:30:09.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:30:09.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252532+0000", 2024-09-17T07:30:09.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9579999999999999e-06 2024-09-17T07:30:09.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252534+0000", 2024-09-17T07:30:09.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5730000000000002e-06 2024-09-17T07:30:09.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324825+0000", 2024-09-17T07:30:09.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072290757999999997 2024-09-17T07:30:09.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324874+0000", 2024-09-17T07:30:09.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9360000000000002e-05 2024-09-17T07:30:09.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325113+0000", 2024-09-17T07:30:09.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000239025 2024-09-17T07:30:09.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.162 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.162 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.162 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19989 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.162 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:09.162 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.890276139000001, 2024-09-17T07:30:09.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072963494000000004, 2024-09-17T07:30:09.163 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.163 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.163 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.163 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.163 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19989 2024-09-17T07:30:09.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.163 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:09.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:09.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:30:09.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:30:09.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:30:09.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2900000000000002e-07 2024-09-17T07:30:09.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252542+0000", 2024-09-17T07:30:09.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8600000000000003e-06 2024-09-17T07:30:09.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252545+0000", 2024-09-17T07:30:09.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.605e-06 2024-09-17T07:30:09.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325144+0000", 2024-09-17T07:30:09.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072599460000000005 2024-09-17T07:30:09.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325223+0000", 2024-09-17T07:30:09.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8473999999999994e-05 2024-09-17T07:30:09.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325501+0000", 2024-09-17T07:30:09.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000278063 2024-09-17T07:30:09.166 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.166 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.166 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.166 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19990 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6422528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.167 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:09.167 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.890265667999998, 2024-09-17T07:30:09.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073241127000000003, 2024-09-17T07:30:09.167 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.167 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.167 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.167 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.167 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19990 2024-09-17T07:30:09.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.167 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252552+0000", 2024-09-17T07:30:09.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4719999999999997e-06 2024-09-17T07:30:09.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252557+0000", 2024-09-17T07:30:09.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3039999999999998e-06 2024-09-17T07:30:09.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252560+0000", 2024-09-17T07:30:09.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8830000000000002e-06 2024-09-17T07:30:09.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325529+0000", 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072968880999999999 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325574+0000", 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466000000000001e-05 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325789+0000", 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021490999999999999 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.171 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19991 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6651904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.171 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:09.171 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.890246366, 2024-09-17T07:30:09.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073720552999999994, 2024-09-17T07:30:09.171 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.171 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.171 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.171 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.171 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.171 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.171 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19991 2024-09-17T07:30:09.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.171 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:09.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4699999999999998e-07 2024-09-17T07:30:09.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252568+0000", 2024-09-17T07:30:09.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5300000000000002e-07 2024-09-17T07:30:09.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252572+0000", 2024-09-17T07:30:09.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9160000000000003e-06 2024-09-17T07:30:09.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252575+0000", 2024-09-17T07:30:09.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:30:09.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325820+0000", 2024-09-17T07:30:09.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073245597999999995 2024-09-17T07:30:09.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325901+0000", 2024-09-17T07:30:09.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0913000000000003e-05 2024-09-17T07:30:09.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326288+0000", 2024-09-17T07:30:09.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038646900000000002 2024-09-17T07:30:09.175 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.175 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.175 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.175 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19992 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6881280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.175 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:09.175 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.890236012999999, 2024-09-17T07:30:09.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074054683999999996, 2024-09-17T07:30:09.175 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.176 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.176 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.176 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19992 2024-09-17T07:30:09.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.176 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:09.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:09.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252578+0000", 2024-09-17T07:30:09.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:30:09.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252579+0000", 2024-09-17T07:30:09.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2500000000000004e-07 2024-09-17T07:30:09.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252582+0000", 2024-09-17T07:30:09.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-17T07:30:09.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252585+0000", 2024-09-17T07:30:09.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.542e-06 2024-09-17T07:30:09.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326315+0000", 2024-09-17T07:30:09.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073729921000000004 2024-09-17T07:30:09.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326359+0000", 2024-09-17T07:30:09.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4095999999999999e-05 2024-09-17T07:30:09.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326632+0000", 2024-09-17T07:30:09.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027314499999999997 2024-09-17T07:30:09.179 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.179 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.179 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.180 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19993 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7110656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.180 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:09.180 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.890225708999999, 2024-09-17T07:30:09.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074425142999999999, 2024-09-17T07:30:09.180 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.180 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.180 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.180 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.180 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.180 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.180 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19993 2024-09-17T07:30:09.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.180 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:09.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:09.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:09.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:30:09.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252589+0000", 2024-09-17T07:30:09.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.33e-07 2024-09-17T07:30:09.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252593+0000", 2024-09-17T07:30:09.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:30:09.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252595+0000", 2024-09-17T07:30:09.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.621e-06 2024-09-17T07:30:09.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326666+0000", 2024-09-17T07:30:09.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074070787999999999 2024-09-17T07:30:09.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326752+0000", 2024-09-17T07:30:09.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5748000000000006e-05 2024-09-17T07:30:09.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327013+0000", 2024-09-17T07:30:09.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026098400000000002 2024-09-17T07:30:09.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.184 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.184 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19994 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7340032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.184 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:09.185 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.890215249999997, 2024-09-17T07:30:09.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074765920999999999, 2024-09-17T07:30:09.185 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.185 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.185 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.185 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.185 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19994 2024-09-17T07:30:09.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.185 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:09.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:09.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:09.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:30:09.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252599+0000", 2024-09-17T07:30:09.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2099999999999995e-07 2024-09-17T07:30:09.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252603+0000", 2024-09-17T07:30:09.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:30:09.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252606+0000", 2024-09-17T07:30:09.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-09-17T07:30:09.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327047+0000", 2024-09-17T07:30:09.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074441050999999994 2024-09-17T07:30:09.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327121+0000", 2024-09-17T07:30:09.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4283999999999996e-05 2024-09-17T07:30:09.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327364+0000", 2024-09-17T07:30:09.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000243155 2024-09-17T07:30:09.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.189 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.189 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19995 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7569408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.189 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:09.189 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.890204552999997, 2024-09-17T07:30:09.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075172694999999998, 2024-09-17T07:30:09.189 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.189 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.190 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.190 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19995 2024-09-17T07:30:09.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.190 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:09.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:09.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:09.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:30:09.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252610+0000", 2024-09-17T07:30:09.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6799999999999999e-07 2024-09-17T07:30:09.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252614+0000", 2024-09-17T07:30:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000002e-06 2024-09-17T07:30:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252616+0000", 2024-09-17T07:30:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.419e-06 2024-09-17T07:30:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327407+0000", 2024-09-17T07:30:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074790617000000004 2024-09-17T07:30:09.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327498+0000", 2024-09-17T07:30:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0758000000000001e-05 2024-09-17T07:30:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327782+0000", 2024-09-17T07:30:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028395599999999998 2024-09-17T07:30:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7798784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.890194237000003, 2024-09-17T07:30:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075605832999999997, 2024-09-17T07:30:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:30:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:09.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:30:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:30:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:30:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:30:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:30:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:30:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:30:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327844+0000", 2024-09-17T07:30:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075217709999999993 2024-09-17T07:30:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327927+0000", 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2628999999999998e-05 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328225+0000", 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029800600000000002 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8028160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:09.198 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.890183884000002, 2024-09-17T07:30:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076061237000000004, 2024-09-17T07:30:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:30:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:30:09.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:30:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:30:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:30:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:30:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:30:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328268+0000", 2024-09-17T07:30:09.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075626323999999995 2024-09-17T07:30:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328352+0000", 2024-09-17T07:30:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3578999999999994e-05 2024-09-17T07:30:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328691+0000", 2024-09-17T07:30:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033915099999999999 2024-09-17T07:30:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8257536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.890168432999999, 2024-09-17T07:30:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076441903000000005, 2024-09-17T07:30:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:30:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:30:09.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:30:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:30:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:30:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:30:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:30:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:30:09.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328747+0000", 2024-09-17T07:30:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076094192000000005 2024-09-17T07:30:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328824+0000", 2024-09-17T07:30:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7096999999999997e-05 2024-09-17T07:30:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.329087+0000", 2024-09-17T07:30:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026249999999999998 2024-09-17T07:30:09.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20029 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:30:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.880083444999997, 2024-09-17T07:30:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071783696999999994, 2024-09-17T07:30:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.207 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20029 2024-09-17T07:30:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:30:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:30:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262735+0000", 2024-09-17T07:30:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.955e-06 2024-09-17T07:30:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262737+0000", 2024-09-17T07:30:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0090000000000002e-06 2024-09-17T07:30:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262743+0000", 2024-09-17T07:30:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2890000000000003e-06 2024-09-17T07:30:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262748+0000", 2024-09-17T07:30:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:30:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334163+0000", 2024-09-17T07:30:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071415336999999995 2024-09-17T07:30:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334243+0000", 2024-09-17T07:30:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9548000000000005e-05 2024-09-17T07:30:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334514+0000", 2024-09-17T07:30:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027086800000000001 2024-09-17T07:30:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7356416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.880052274000001, 2024-09-17T07:30:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072172616999999994, 2024-09-17T07:30:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.211 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:30:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:30:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:30:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:30:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:30:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:30:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:30:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:30:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:30:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334558+0000", 2024-09-17T07:30:09.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071780790999999997 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334647+0000", 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9448000000000001e-05 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334934+0000", 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028654599999999999 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7585792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.215 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.880030562000002, 2024-09-17T07:30:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072606135000000002, 2024-09-17T07:30:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:30:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:30:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:30:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:30:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:30:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:30:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:30:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:30:09.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335116+0000", 2024-09-17T07:30:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072323897999999998 2024-09-17T07:30:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335166+0000", 2024-09-17T07:30:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9882000000000001e-05 2024-09-17T07:30:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335389+0000", 2024-09-17T07:30:09.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022334300000000001 2024-09-17T07:30:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7815168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.880018362999998, 2024-09-17T07:30:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073051230999999994, 2024-09-17T07:30:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.220 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:30:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:30:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:30:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:30:09.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:30:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:30:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:30:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335431+0000", 2024-09-17T07:30:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072627852000000007 2024-09-17T07:30:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335525+0000", 2024-09-17T07:30:09.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3708000000000005e-05 2024-09-17T07:30:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335846+0000", 2024-09-17T07:30:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032182000000000001 2024-09-17T07:30:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.880007505000002, 2024-09-17T07:30:09.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073284199999999994, 2024-09-17T07:30:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:30:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.225 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:09.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:30:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:30:09.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:30:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:30:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:30:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:30:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:30:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335863+0000", 2024-09-17T07:30:09.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073048921000000003 2024-09-17T07:30:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335904+0000", 2024-09-17T07:30:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1545000000000003e-05 2024-09-17T07:30:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336090+0000", 2024-09-17T07:30:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185787 2024-09-17T07:30:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.231 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8273920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.879993245000001, 2024-09-17T07:30:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073670478999999997, 2024-09-17T07:30:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:30:09.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:30:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:30:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:30:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:30:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:30:09.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:30:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:30:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336126+0000", 2024-09-17T07:30:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073297254000000006 2024-09-17T07:30:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336211+0000", 2024-09-17T07:30:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4728000000000006e-05 2024-09-17T07:30:09.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336491+0000", 2024-09-17T07:30:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027979399999999998 2024-09-17T07:30:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:09.236 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:09.236 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:09.236 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T07:30:09.238 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:09.236+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:09.239 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:09.339 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:09.338+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:09.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:30:09.384 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:09.385 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:09.385 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:09.385 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:09.393 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:09.477 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:09.476+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:09.499 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:09.499 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:09.516 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:09.516 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:09.532 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:09.532 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:09.537 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:09.538 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:09.538 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:09.538 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:09.538 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:09.541 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:09.539+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:09.546 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:09.548 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:09.548 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:09.564 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:09.581 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:09.581 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:09.597 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:09.614 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:09.614 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:09.667 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:09.665+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:09.689 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:09.689 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:09.698 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:09.743 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:09.742+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:09.801 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:09.801 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:09.809 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:09.843 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:09.843+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:09.912 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:09.912 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:09.920 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:09.964 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:09.963+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:10.024 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:10.024 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:10.033 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:10.077 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:10.075+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:10.142 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:10.142 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:10.146 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:10.145+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:10.151 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:10.175 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-17T07:30:10.247 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:10.246+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:10.293 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:10.293 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:10.302 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:10.348 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:10.347+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:10.449 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:10.448+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:10.449 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:10.449 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:10.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:10.550 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:10.548+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:10.566 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:10.566 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:10.574 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:10.651 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:10.649+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:10.681 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:10.681 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:10.690 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:10.752 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:10.751+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:10.837 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:10.837 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:10.846 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:10.853 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:10.852+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:10.953 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:10.953 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:10.954 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:10.952+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:10.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:11.055 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:11.054+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:11.110 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:11.110 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:11.119 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:11.156 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:11.155+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:11.257 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:11.256+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:11.276 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:11.277 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:11.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:11.358 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:11.356+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:11.434 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:11.434 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:11.443 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:11.486 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:11.485+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:11.549 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:11.549 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:11.558 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:11.561 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:11.559+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:11.626 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:30:11.661 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:11.659+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:11.707 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:11.707 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:11.716 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:11.762 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:11.760+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:11.766 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:11.766 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:11.783 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:11.783 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:11.799 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:11.800 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:11.816 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:11.816 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:11.821 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:11.821 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:11.829 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:11.832 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:11.833 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:11.850 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:11.850 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:11.866 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:11.867 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:11.873 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:11.872+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:11.885 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:11.885 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:11.932 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:11.933 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:11.933 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:11.933 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:11.933 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:11.941 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:11.963 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:11.963+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:12.064 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:12.063+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:12.096 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:30525 3.1b 3:def7c8f0:::rbd_data.118381ffbbab.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T07:30:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 58.572031373999998, 2024-09-17T07:30:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55515850300000003, 2024-09-17T07:30:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30525 2024-09-17T07:30:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522300+0000", 2024-09-17T07:30:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:30:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522305+0000", 2024-09-17T07:30:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-17T07:30:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522318+0000", 2024-09-17T07:30:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3030999999999999e-05 2024-09-17T07:30:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522326+0000", 2024-09-17T07:30:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4200000000000001e-06 2024-09-17T07:30:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522385+0000", 2024-09-17T07:30:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9005000000000002e-05 2024-09-17T07:30:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522393+0000", 2024-09-17T07:30:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-06 2024-09-17T07:30:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075347+0000", 2024-09-17T07:30:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55295417800000002 2024-09-17T07:30:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075502+0000", 2024-09-17T07:30:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154272 2024-09-17T07:30:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075630+0000", 2024-09-17T07:30:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012816300000000001 2024-09-17T07:30:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077221+0000", 2024-09-17T07:30:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015912929999999999 2024-09-17T07:30:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077353+0000", 2024-09-17T07:30:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013215699999999999 2024-09-17T07:30:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077413+0000", 2024-09-17T07:30:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0034999999999997e-05 2024-09-17T07:30:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077457+0000", 2024-09-17T07:30:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4017999999999998e-05 2024-09-17T07:30:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44181 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13922304~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.456852726000001, 2024-09-17T07:30:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046010502000000002, 2024-09-17T07:30:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44181 2024-09-17T07:30:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637478+0000", 2024-09-17T07:30:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:30:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637480+0000", 2024-09-17T07:30:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.843e-06 2024-09-17T07:30:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637487+0000", 2024-09-17T07:30:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6159999999999996e-06 2024-09-17T07:30:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637492+0000", 2024-09-17T07:30:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5360000000000003e-06 2024-09-17T07:30:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683351+0000", 2024-09-17T07:30:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045858758999999999 2024-09-17T07:30:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683422+0000", 2024-09-17T07:30:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1705999999999994e-05 2024-09-17T07:30:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683488+0000", 2024-09-17T07:30:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5612999999999997e-05 2024-09-17T07:30:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44182 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13938688~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.456832055, 2024-09-17T07:30:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046153118999999999, 2024-09-17T07:30:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44182 2024-09-17T07:30:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:30:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637506+0000", 2024-09-17T07:30:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1520000000000003e-06 2024-09-17T07:30:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637512+0000", 2024-09-17T07:30:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-09-17T07:30:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637517+0000", 2024-09-17T07:30:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-09-17T07:30:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683514+0000", 2024-09-17T07:30:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045997616999999998 2024-09-17T07:30:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683584+0000", 2024-09-17T07:30:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9942000000000002e-05 2024-09-17T07:30:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683651+0000", 2024-09-17T07:30:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6951000000000003e-05 2024-09-17T07:30:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44183 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13955072~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.456806823000001, 2024-09-17T07:30:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046339793999999997, 2024-09-17T07:30:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44183 2024-09-17T07:30:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639583+0000", 2024-09-17T07:30:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002059412 2024-09-17T07:30:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639597+0000", 2024-09-17T07:30:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3832000000000001e-05 2024-09-17T07:30:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639607+0000", 2024-09-17T07:30:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485e-05 2024-09-17T07:30:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639614+0000", 2024-09-17T07:30:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0849999999999999e-06 2024-09-17T07:30:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683705+0000", 2024-09-17T07:30:12.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044091261 2024-09-17T07:30:12.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683792+0000", 2024-09-17T07:30:12.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6197000000000002e-05 2024-09-17T07:30:12.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683863+0000", 2024-09-17T07:30:12.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1521999999999997e-05 2024-09-17T07:30:12.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.116 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.116 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44199 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14217216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.116 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:12.116 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.453289271000003, 2024-09-17T07:30:12.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045995623999999999, 2024-09-17T07:30:12.116 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.116 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.117 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.117 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44199 2024-09-17T07:30:12.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.117 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:12.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:12.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:12.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:30:12.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641043+0000", 2024-09-17T07:30:12.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.624e-06 2024-09-17T07:30:12.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641048+0000", 2024-09-17T07:30:12.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4029999999999998e-06 2024-09-17T07:30:12.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641053+0000", 2024-09-17T07:30:12.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6670000000000004e-06 2024-09-17T07:30:12.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686918+0000", 2024-09-17T07:30:12.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045865131000000003 2024-09-17T07:30:12.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686986+0000", 2024-09-17T07:30:12.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8299000000000005e-05 2024-09-17T07:30:12.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687036+0000", 2024-09-17T07:30:12.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0164999999999998e-05 2024-09-17T07:30:12.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.120 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.121 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44200 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14233600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.121 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:12.121 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.453271442000002, 2024-09-17T07:30:12.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046183984999999997, 2024-09-17T07:30:12.121 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.121 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.121 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.121 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44200 2024-09-17T07:30:12.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.121 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:12.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:12.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:12.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:12.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641061+0000", 2024-09-17T07:30:12.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.767e-06 2024-09-17T07:30:12.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641067+0000", 2024-09-17T07:30:12.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7379999999999999e-06 2024-09-17T07:30:12.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641071+0000", 2024-09-17T07:30:12.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-09-17T07:30:12.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687078+0000", 2024-09-17T07:30:12.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046007095999999997 2024-09-17T07:30:12.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687158+0000", 2024-09-17T07:30:12.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0374e-05 2024-09-17T07:30:12.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687243+0000", 2024-09-17T07:30:12.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4221999999999994e-05 2024-09-17T07:30:12.125 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.125 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.125 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.125 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44201 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14249984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.125 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:12.125 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.453253795999998, 2024-09-17T07:30:12.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046303182999999998, 2024-09-17T07:30:12.125 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.125 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.125 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.126 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.126 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.126 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.126 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44201 2024-09-17T07:30:12.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.126 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:12.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:12.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641120+0000", 2024-09-17T07:30:12.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3933000000000001e-05 2024-09-17T07:30:12.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641123+0000", 2024-09-17T07:30:12.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.191e-06 2024-09-17T07:30:12.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641133+0000", 2024-09-17T07:30:12.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4020000000000008e-06 2024-09-17T07:30:12.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641138+0000", 2024-09-17T07:30:12.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6559999999999997e-06 2024-09-17T07:30:12.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687279+0000", 2024-09-17T07:30:12.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046140278999999999 2024-09-17T07:30:12.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687338+0000", 2024-09-17T07:30:12.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9268999999999998e-05 2024-09-17T07:30:12.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687379+0000", 2024-09-17T07:30:12.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1452999999999998e-05 2024-09-17T07:30:12.129 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.129 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.129 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.129 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44202 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14266368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.129 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.453183029999998, 2024-09-17T07:30:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046484299, 2024-09-17T07:30:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44202 2024-09-17T07:30:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:12.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642810+0000", 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016627980000000001 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642815+0000", 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.091e-06 2024-09-17T07:30:12.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642829+0000", 2024-09-17T07:30:12.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4047e-05 2024-09-17T07:30:12.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642844+0000", 2024-09-17T07:30:12.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4515000000000001e-05 2024-09-17T07:30:12.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687426+0000", 2024-09-17T07:30:12.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044582201000000002 2024-09-17T07:30:12.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687529+0000", 2024-09-17T07:30:12.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103078 2024-09-17T07:30:12.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687631+0000", 2024-09-17T07:30:12.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102569 2024-09-17T07:30:12.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.133 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.134 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44209 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14381056~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.134 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:12.134 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.451284878000003, 2024-09-17T07:30:12.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045944444000000001, 2024-09-17T07:30:12.134 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.134 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.134 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.134 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44209 2024-09-17T07:30:12.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.134 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:12.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:12.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643046+0000", 2024-09-17T07:30:12.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:30:12.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643048+0000", 2024-09-17T07:30:12.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:30:12.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643054+0000", 2024-09-17T07:30:12.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.032e-06 2024-09-17T07:30:12.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643065+0000", 2024-09-17T07:30:12.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1861999999999999e-05 2024-09-17T07:30:12.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688806+0000", 2024-09-17T07:30:12.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045740617999999997 2024-09-17T07:30:12.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688927+0000", 2024-09-17T07:30:12.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000121327 2024-09-17T07:30:12.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688990+0000", 2024-09-17T07:30:12.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2293999999999997e-05 2024-09-17T07:30:12.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.138 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.138 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44210 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14397440~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.138 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:12.138 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.451257951999999, 2024-09-17T07:30:12.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046104635999999997, 2024-09-17T07:30:12.138 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.138 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.138 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.139 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44210 2024-09-17T07:30:12.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.139 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:12.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:12.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643073+0000", 2024-09-17T07:30:12.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.77e-07 2024-09-17T07:30:12.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643075+0000", 2024-09-17T07:30:12.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-09-17T07:30:12.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643082+0000", 2024-09-17T07:30:12.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7789999999999996e-06 2024-09-17T07:30:12.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643087+0000", 2024-09-17T07:30:12.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.967e-06 2024-09-17T07:30:12.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689022+0000", 2024-09-17T07:30:12.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045934349999999999 2024-09-17T07:30:12.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689112+0000", 2024-09-17T07:30:12.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0039999999999999e-05 2024-09-17T07:30:12.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689177+0000", 2024-09-17T07:30:12.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5041000000000003e-05 2024-09-17T07:30:12.142 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.142 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.142 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.142 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44211 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14413824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.142 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:12.143 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.451236819000002, 2024-09-17T07:30:12.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046309003000000001, 2024-09-17T07:30:12.143 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.143 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.143 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.143 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44211 2024-09-17T07:30:12.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.143 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:12.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:12.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643094+0000", 2024-09-17T07:30:12.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:30:12.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643095+0000", 2024-09-17T07:30:12.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-17T07:30:12.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643101+0000", 2024-09-17T07:30:12.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8730000000000004e-06 2024-09-17T07:30:12.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643106+0000", 2024-09-17T07:30:12.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6859999999999997e-06 2024-09-17T07:30:12.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689220+0000", 2024-09-17T07:30:12.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046113945000000003 2024-09-17T07:30:12.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689309+0000", 2024-09-17T07:30:12.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9560000000000003e-05 2024-09-17T07:30:12.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689402+0000", 2024-09-17T07:30:12.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2786000000000002e-05 2024-09-17T07:30:12.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.147 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.147 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.147 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44212 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14430208~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.147 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:12.147 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.451192257000002, 2024-09-17T07:30:12.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046448912000000002, 2024-09-17T07:30:12.147 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.147 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.147 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.148 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.148 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44212 2024-09-17T07:30:12.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.148 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:12.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:12.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:12.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2200000000000004e-07 2024-09-17T07:30:12.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643141+0000", 2024-09-17T07:30:12.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5210000000000001e-06 2024-09-17T07:30:12.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643149+0000", 2024-09-17T07:30:12.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6490000000000008e-06 2024-09-17T07:30:12.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643161+0000", 2024-09-17T07:30:12.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2241e-05 2024-09-17T07:30:12.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689423+0000", 2024-09-17T07:30:12.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046262323000000001 2024-09-17T07:30:12.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689505+0000", 2024-09-17T07:30:12.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2103999999999999e-05 2024-09-17T07:30:12.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689587+0000", 2024-09-17T07:30:12.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1452000000000006e-05 2024-09-17T07:30:12.152 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.152 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.152 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.152 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44213 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14446592~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.152 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:12.152 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.451160758, 2024-09-17T07:30:12.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046601780000000002, 2024-09-17T07:30:12.152 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.152 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.153 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.153 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.153 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.153 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.153 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44213 2024-09-17T07:30:12.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.153 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:12.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:12.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643170+0000", 2024-09-17T07:30:12.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-17T07:30:12.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643172+0000", 2024-09-17T07:30:12.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1660000000000001e-06 2024-09-17T07:30:12.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643180+0000", 2024-09-17T07:30:12.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-09-17T07:30:12.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643185+0000", 2024-09-17T07:30:12.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-09-17T07:30:12.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689622+0000", 2024-09-17T07:30:12.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046436742000000003 2024-09-17T07:30:12.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689697+0000", 2024-09-17T07:30:12.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5179999999999995e-05 2024-09-17T07:30:12.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689771+0000", 2024-09-17T07:30:12.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3928000000000001e-05 2024-09-17T07:30:12.156 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.156 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.156 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.157 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44214 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14462976~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.157 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:12.157 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.451137697999997, 2024-09-17T07:30:12.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046736038000000001, 2024-09-17T07:30:12.157 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.157 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.157 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.157 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44214 2024-09-17T07:30:12.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.157 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:12.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:12.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643193+0000", 2024-09-17T07:30:12.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:30:12.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643195+0000", 2024-09-17T07:30:12.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0640000000000001e-06 2024-09-17T07:30:12.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643201+0000", 2024-09-17T07:30:12.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2859999999999996e-06 2024-09-17T07:30:12.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643206+0000", 2024-09-17T07:30:12.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-17T07:30:12.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689814+0000", 2024-09-17T07:30:12.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046607742000000001 2024-09-17T07:30:12.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689886+0000", 2024-09-17T07:30:12.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2020999999999994e-05 2024-09-17T07:30:12.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689928+0000", 2024-09-17T07:30:12.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2771000000000001e-05 2024-09-17T07:30:12.161 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.161 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.161 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.161 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44215 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14479360~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.161 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:12.161 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.451093800999999, 2024-09-17T07:30:12.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046868543999999998, 2024-09-17T07:30:12.161 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.161 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.162 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.162 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44215 2024-09-17T07:30:12.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.162 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:12.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:12.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643237+0000", 2024-09-17T07:30:12.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:30:12.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643245+0000", 2024-09-17T07:30:12.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3270000000000001e-06 2024-09-17T07:30:12.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643254+0000", 2024-09-17T07:30:12.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6049999999999998e-06 2024-09-17T07:30:12.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643260+0000", 2024-09-17T07:30:12.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7849999999999997e-06 2024-09-17T07:30:12.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689961+0000", 2024-09-17T07:30:12.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046701196 2024-09-17T07:30:12.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690036+0000", 2024-09-17T07:30:12.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5153000000000004e-05 2024-09-17T07:30:12.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690105+0000", 2024-09-17T07:30:12.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8874000000000005e-05 2024-09-17T07:30:12.165 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.165 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.165 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.166 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44216 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14495744~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.166 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:12.166 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.451062088, 2024-09-17T07:30:12.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047003047999999999, 2024-09-17T07:30:12.166 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.166 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.166 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.166 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.166 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.166 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.166 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44216 2024-09-17T07:30:12.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.166 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:12.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:12.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643269+0000", 2024-09-17T07:30:12.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:30:12.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643271+0000", 2024-09-17T07:30:12.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-09-17T07:30:12.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643278+0000", 2024-09-17T07:30:12.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8850000000000001e-06 2024-09-17T07:30:12.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643282+0000", 2024-09-17T07:30:12.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5979999999999999e-06 2024-09-17T07:30:12.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690133+0000", 2024-09-17T07:30:12.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850470999999998 2024-09-17T07:30:12.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690205+0000", 2024-09-17T07:30:12.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2442999999999997e-05 2024-09-17T07:30:12.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690271+0000", 2024-09-17T07:30:12.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5874e-05 2024-09-17T07:30:12.170 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.170 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.170 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.170 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44217 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14512128~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.170 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:12.170 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.451041076000003, 2024-09-17T07:30:12.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047166485000000001, 2024-09-17T07:30:12.170 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.170 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.170 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44217 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:12.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-17T07:30:12.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643291+0000", 2024-09-17T07:30:12.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.962e-06 2024-09-17T07:30:12.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643297+0000", 2024-09-17T07:30:12.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0700000000000003e-06 2024-09-17T07:30:12.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643302+0000", 2024-09-17T07:30:12.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:30:12.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690308+0000", 2024-09-17T07:30:12.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047005354999999999 2024-09-17T07:30:12.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690385+0000", 2024-09-17T07:30:12.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7541999999999997e-05 2024-09-17T07:30:12.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690455+0000", 2024-09-17T07:30:12.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0328999999999997e-05 2024-09-17T07:30:12.174 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.174 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.174 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.174 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44218 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14528512~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.174 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:12.174 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.451021398999998, 2024-09-17T07:30:12.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047321015000000001, 2024-09-17T07:30:12.174 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.175 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.175 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.175 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44218 2024-09-17T07:30:12.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.175 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:12.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:12.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:12.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-17T07:30:12.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643311+0000", 2024-09-17T07:30:12.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.933e-06 2024-09-17T07:30:12.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643318+0000", 2024-09-17T07:30:12.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3980000000000004e-06 2024-09-17T07:30:12.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643322+0000", 2024-09-17T07:30:12.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7609999999999998e-06 2024-09-17T07:30:12.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690484+0000", 2024-09-17T07:30:12.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047161428999999998 2024-09-17T07:30:12.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690564+0000", 2024-09-17T07:30:12.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9845000000000005e-05 2024-09-17T07:30:12.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690630+0000", 2024-09-17T07:30:12.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6021999999999999e-05 2024-09-17T07:30:12.178 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.178 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.178 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.179 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44219 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14544896~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.179 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:12.179 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.451001032000001, 2024-09-17T07:30:12.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048128708999999999, 2024-09-17T07:30:12.179 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.179 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.179 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.179 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.179 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.179 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.179 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44219 2024-09-17T07:30:12.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:12.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:30:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643337+0000", 2024-09-17T07:30:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.109e-06 2024-09-17T07:30:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643350+0000", 2024-09-17T07:30:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3046e-05 2024-09-17T07:30:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643356+0000", 2024-09-17T07:30:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8930000000000002e-06 2024-09-17T07:30:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690660+0000", 2024-09-17T07:30:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047304061000000001 2024-09-17T07:30:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691372+0000", 2024-09-17T07:30:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00071281299999999997 2024-09-17T07:30:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691458+0000", 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5352000000000005e-05 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44220 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14561280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.450966983999997, 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048309849000000002, 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44220 2024-09-17T07:30:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646869+0000", 2024-09-17T07:30:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003506267 2024-09-17T07:30:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646874+0000", 2024-09-17T07:30:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7500000000000003e-06 2024-09-17T07:30:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646887+0000", 2024-09-17T07:30:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3115999999999999e-05 2024-09-17T07:30:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646894+0000", 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5289999999999997e-06 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691547+0000", 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044652743000000002 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691622+0000", 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5037999999999995e-05 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691673+0000", 2024-09-17T07:30:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-17T07:30:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.187 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.187 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:12.188 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:12.189 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:12.186+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:12.196 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T07:30:12.289 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:12.288+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:12.294 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:12.294 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:12.294 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:12.295 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:12.303 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:12.390 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:12.389+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:12.405 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:12.405 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:12.405 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:12.405 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:12.405 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:12.413 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:12.491 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:12.490+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:12.517 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19985 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5275648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.263193830999995, 2024-09-17T07:30:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071819998999999995, 2024-09-17T07:30:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19985 2024-09-17T07:30:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252493+0000", 2024-09-17T07:30:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6300000000000004e-07 2024-09-17T07:30:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252496+0000", 2024-09-17T07:30:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-17T07:30:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252499+0000", 2024-09-17T07:30:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:30:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324036+0000", 2024-09-17T07:30:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071536587999999998 2024-09-17T07:30:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324118+0000", 2024-09-17T07:30:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2336999999999995e-05 2024-09-17T07:30:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324312+0000", 2024-09-17T07:30:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000193709 2024-09-17T07:30:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19986 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5505024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.263182678999996, 2024-09-17T07:30:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072083761999999996, 2024-09-17T07:30:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19986 2024-09-17T07:30:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:30:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252508+0000", 2024-09-17T07:30:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:30:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252511+0000", 2024-09-17T07:30:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.145e-06 2024-09-17T07:30:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252514+0000", 2024-09-17T07:30:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5569999999999998e-06 2024-09-17T07:30:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324326+0000", 2024-09-17T07:30:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071812681000000003 2024-09-17T07:30:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324372+0000", 2024-09-17T07:30:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5893000000000003e-05 2024-09-17T07:30:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324587+0000", 2024-09-17T07:30:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214608 2024-09-17T07:30:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19987 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5734400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.263169060999999, 2024-09-17T07:30:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072276676999999998, 2024-09-17T07:30:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19987 2024-09-17T07:30:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252518+0000", 2024-09-17T07:30:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-17T07:30:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252522+0000", 2024-09-17T07:30:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2440000000000004e-06 2024-09-17T07:30:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252525+0000", 2024-09-17T07:30:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5919999999999999e-06 2024-09-17T07:30:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324621+0000", 2024-09-17T07:30:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072096442999999996 2024-09-17T07:30:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324699+0000", 2024-09-17T07:30:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8061999999999999e-05 2024-09-17T07:30:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324793+0000", 2024-09-17T07:30:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4207000000000001e-05 2024-09-17T07:30:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19988 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.263158090000005, 2024-09-17T07:30:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072585798000000007, 2024-09-17T07:30:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19988 2024-09-17T07:30:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252529+0000", 2024-09-17T07:30:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:30:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252532+0000", 2024-09-17T07:30:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9579999999999999e-06 2024-09-17T07:30:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252534+0000", 2024-09-17T07:30:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5730000000000002e-06 2024-09-17T07:30:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324825+0000", 2024-09-17T07:30:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072290757999999997 2024-09-17T07:30:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324874+0000", 2024-09-17T07:30:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9360000000000002e-05 2024-09-17T07:30:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325113+0000", 2024-09-17T07:30:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000239025 2024-09-17T07:30:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19989 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.263148469000001, 2024-09-17T07:30:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072963494000000004, 2024-09-17T07:30:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19989 2024-09-17T07:30:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:30:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:30:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:30:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2900000000000002e-07 2024-09-17T07:30:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252542+0000", 2024-09-17T07:30:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8600000000000003e-06 2024-09-17T07:30:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252545+0000", 2024-09-17T07:30:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.605e-06 2024-09-17T07:30:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325144+0000", 2024-09-17T07:30:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072599460000000005 2024-09-17T07:30:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325223+0000", 2024-09-17T07:30:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8473999999999994e-05 2024-09-17T07:30:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325501+0000", 2024-09-17T07:30:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000278063 2024-09-17T07:30:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19990 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6422528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.263137998000005, 2024-09-17T07:30:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073241127000000003, 2024-09-17T07:30:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19990 2024-09-17T07:30:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252552+0000", 2024-09-17T07:30:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4719999999999997e-06 2024-09-17T07:30:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252557+0000", 2024-09-17T07:30:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3039999999999998e-06 2024-09-17T07:30:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252560+0000", 2024-09-17T07:30:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8830000000000002e-06 2024-09-17T07:30:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325529+0000", 2024-09-17T07:30:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072968880999999999 2024-09-17T07:30:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325574+0000", 2024-09-17T07:30:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466000000000001e-05 2024-09-17T07:30:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325789+0000", 2024-09-17T07:30:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021490999999999999 2024-09-17T07:30:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19991 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6651904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.263118696000006, 2024-09-17T07:30:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073720552999999994, 2024-09-17T07:30:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19991 2024-09-17T07:30:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4699999999999998e-07 2024-09-17T07:30:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252568+0000", 2024-09-17T07:30:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5300000000000002e-07 2024-09-17T07:30:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252572+0000", 2024-09-17T07:30:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9160000000000003e-06 2024-09-17T07:30:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252575+0000", 2024-09-17T07:30:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:30:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325820+0000", 2024-09-17T07:30:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073245597999999995 2024-09-17T07:30:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325901+0000", 2024-09-17T07:30:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0913000000000003e-05 2024-09-17T07:30:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326288+0000", 2024-09-17T07:30:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038646900000000002 2024-09-17T07:30:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19992 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6881280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.263108342999999, 2024-09-17T07:30:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074054683999999996, 2024-09-17T07:30:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19992 2024-09-17T07:30:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252578+0000", 2024-09-17T07:30:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:30:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252579+0000", 2024-09-17T07:30:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2500000000000004e-07 2024-09-17T07:30:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252582+0000", 2024-09-17T07:30:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-17T07:30:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252585+0000", 2024-09-17T07:30:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.542e-06 2024-09-17T07:30:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326315+0000", 2024-09-17T07:30:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073729921000000004 2024-09-17T07:30:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326359+0000", 2024-09-17T07:30:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4095999999999999e-05 2024-09-17T07:30:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326632+0000", 2024-09-17T07:30:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027314499999999997 2024-09-17T07:30:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19993 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7110656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.263098038999999, 2024-09-17T07:30:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074425142999999999, 2024-09-17T07:30:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19993 2024-09-17T07:30:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:30:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252589+0000", 2024-09-17T07:30:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.33e-07 2024-09-17T07:30:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252593+0000", 2024-09-17T07:30:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:30:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252595+0000", 2024-09-17T07:30:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.621e-06 2024-09-17T07:30:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326666+0000", 2024-09-17T07:30:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074070787999999999 2024-09-17T07:30:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326752+0000", 2024-09-17T07:30:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5748000000000006e-05 2024-09-17T07:30:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327013+0000", 2024-09-17T07:30:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026098400000000002 2024-09-17T07:30:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19994 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7340032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.263087580000004, 2024-09-17T07:30:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074765920999999999, 2024-09-17T07:30:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19994 2024-09-17T07:30:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:30:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252599+0000", 2024-09-17T07:30:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2099999999999995e-07 2024-09-17T07:30:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252603+0000", 2024-09-17T07:30:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:30:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252606+0000", 2024-09-17T07:30:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-09-17T07:30:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327047+0000", 2024-09-17T07:30:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074441050999999994 2024-09-17T07:30:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327121+0000", 2024-09-17T07:30:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4283999999999996e-05 2024-09-17T07:30:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327364+0000", 2024-09-17T07:30:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000243155 2024-09-17T07:30:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19995 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7569408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.263076882999997, 2024-09-17T07:30:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075172694999999998, 2024-09-17T07:30:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19995 2024-09-17T07:30:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:30:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252610+0000", 2024-09-17T07:30:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6799999999999999e-07 2024-09-17T07:30:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252614+0000", 2024-09-17T07:30:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000002e-06 2024-09-17T07:30:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252616+0000", 2024-09-17T07:30:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.419e-06 2024-09-17T07:30:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327407+0000", 2024-09-17T07:30:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074790617000000004 2024-09-17T07:30:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327498+0000", 2024-09-17T07:30:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0758000000000001e-05 2024-09-17T07:30:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327782+0000", 2024-09-17T07:30:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028395599999999998 2024-09-17T07:30:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7798784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.263066566999996, 2024-09-17T07:30:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075605832999999997, 2024-09-17T07:30:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:30:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:30:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:30:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:30:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:30:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:30:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:30:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:30:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327844+0000", 2024-09-17T07:30:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075217709999999993 2024-09-17T07:30:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327927+0000", 2024-09-17T07:30:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2628999999999998e-05 2024-09-17T07:30:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328225+0000", 2024-09-17T07:30:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029800600000000002 2024-09-17T07:30:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8028160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.263056214000002, 2024-09-17T07:30:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076061237000000004, 2024-09-17T07:30:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:30:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:30:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:30:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:30:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:30:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:30:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:30:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328268+0000", 2024-09-17T07:30:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075626323999999995 2024-09-17T07:30:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328352+0000", 2024-09-17T07:30:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3578999999999994e-05 2024-09-17T07:30:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328691+0000", 2024-09-17T07:30:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033915099999999999 2024-09-17T07:30:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8257536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.263040763000006, 2024-09-17T07:30:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076441903000000005, 2024-09-17T07:30:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:30:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:30:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:30:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:30:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:30:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:30:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:30:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:30:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328747+0000", 2024-09-17T07:30:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076094192000000005 2024-09-17T07:30:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328824+0000", 2024-09-17T07:30:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7096999999999997e-05 2024-09-17T07:30:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.329087+0000", 2024-09-17T07:30:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026249999999999998 2024-09-17T07:30:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20029 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:30:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.252955775000004, 2024-09-17T07:30:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071783696999999994, 2024-09-17T07:30:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20029 2024-09-17T07:30:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:30:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:30:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262735+0000", 2024-09-17T07:30:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.955e-06 2024-09-17T07:30:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262737+0000", 2024-09-17T07:30:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0090000000000002e-06 2024-09-17T07:30:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262743+0000", 2024-09-17T07:30:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2890000000000003e-06 2024-09-17T07:30:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262748+0000", 2024-09-17T07:30:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:30:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334163+0000", 2024-09-17T07:30:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071415336999999995 2024-09-17T07:30:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334243+0000", 2024-09-17T07:30:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9548000000000005e-05 2024-09-17T07:30:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334514+0000", 2024-09-17T07:30:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027086800000000001 2024-09-17T07:30:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7356416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.252924604, 2024-09-17T07:30:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072172616999999994, 2024-09-17T07:30:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:30:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:30:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:30:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:30:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:30:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:30:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:30:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:30:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:30:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334558+0000", 2024-09-17T07:30:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071780790999999997 2024-09-17T07:30:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334647+0000", 2024-09-17T07:30:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9448000000000001e-05 2024-09-17T07:30:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334934+0000", 2024-09-17T07:30:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028654599999999999 2024-09-17T07:30:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7585792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.252902891999994, 2024-09-17T07:30:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072606135000000002, 2024-09-17T07:30:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:30:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:30:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:30:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:30:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:30:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:30:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:30:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:30:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335116+0000", 2024-09-17T07:30:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072323897999999998 2024-09-17T07:30:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335166+0000", 2024-09-17T07:30:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9882000000000001e-05 2024-09-17T07:30:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335389+0000", 2024-09-17T07:30:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022334300000000001 2024-09-17T07:30:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7815168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.252890692999998, 2024-09-17T07:30:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073051230999999994, 2024-09-17T07:30:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:30:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:30:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:30:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:30:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:30:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:30:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:30:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335431+0000", 2024-09-17T07:30:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072627852000000007 2024-09-17T07:30:12.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335525+0000", 2024-09-17T07:30:12.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3708000000000005e-05 2024-09-17T07:30:12.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335846+0000", 2024-09-17T07:30:12.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032182000000000001 2024-09-17T07:30:12.602 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.602 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.602 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.603 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.603 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:12.603 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.252879835000002, 2024-09-17T07:30:12.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073284199999999994, 2024-09-17T07:30:12.603 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.603 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.603 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.603 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:30:12.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.603 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:12.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:12.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:12.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:30:12.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:30:12.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:30:12.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:30:12.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:30:12.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:30:12.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:30:12.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335863+0000", 2024-09-17T07:30:12.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073048921000000003 2024-09-17T07:30:12.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335904+0000", 2024-09-17T07:30:12.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1545000000000003e-05 2024-09-17T07:30:12.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336090+0000", 2024-09-17T07:30:12.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185787 2024-09-17T07:30:12.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.607 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.607 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8273920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:12.607 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:12.607 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.252865575000001, 2024-09-17T07:30:12.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073670478999999997, 2024-09-17T07:30:12.607 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:12.607 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:12.607 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:12.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:12.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:12.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:12.608 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:30:12.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.608 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:12.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:12.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:12.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:12.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:12.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:12.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:12.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:12.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:30:12.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:12.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:30:12.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:30:12.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:12.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:30:12.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:30:12.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:12.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:30:12.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:30:12.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:12.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336126+0000", 2024-09-17T07:30:12.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073297254000000006 2024-09-17T07:30:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336211+0000", 2024-09-17T07:30:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4728000000000006e-05 2024-09-17T07:30:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336491+0000", 2024-09-17T07:30:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027979399999999998 2024-09-17T07:30:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:12.612 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:12.612 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:12.613 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:12.614 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:12.612+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:12.714 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:12.713+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:12.757 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:12.757 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:12.757 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:12.757 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:12.765 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:12.815 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:12.813+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:12.868 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:12.876 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:12.916 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:12.915+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:13.017 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:13.015+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:13.020 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:13.020 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:13.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:13.020 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:13.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.020 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:13.020 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:13.020 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.113198319000006, 2024-09-17T07:30:13.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:30:13.020 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.021 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.021 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.021 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.021 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.021 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.021 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:30:13.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.021 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:13.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:13.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:13.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:30:13.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:30:13.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:30:13.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:30:13.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:30:13.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:30:13.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:30:13.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:30:13.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:30:13.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:30:13.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:30:13.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:30:13.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:30:13.024 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.024 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.024 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.025 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:13.025 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:13.025 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.113179642000006, 2024-09-17T07:30:13.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:30:13.025 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.025 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.025 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.025 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.025 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.025 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.025 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:30:13.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.025 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:13.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:13.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:30:13.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:13.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:30:13.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:30:13.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:30:13.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:30:13.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:30:13.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:30:13.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:30:13.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:30:13.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:30:13.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:30:13.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:30:13.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:30:13.029 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.029 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.029 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.029 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:13.029 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:13.029 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.113161289000004, 2024-09-17T07:30:13.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:30:13.029 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.029 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.029 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.030 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.030 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.030 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.030 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:30:13.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.030 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:13.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:30:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:30:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:30:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:30:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:30:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:30:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:30:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:30:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:30:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:30:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:30:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:30:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:30:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.113141690999996, 2024-09-17T07:30:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:30:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:30:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:30:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:30:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:30:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:30:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:30:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:30:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:30:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:30:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:30:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:30:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:30:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:30:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:30:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:30:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.113121735999997, 2024-09-17T07:30:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:30:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:30:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:30:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:30:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:30:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:30:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:30:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:30:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:30:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:30:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:30:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:30:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:30:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:30:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:30:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.113103081000006, 2024-09-17T07:30:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:30:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:30:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:30:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:30:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:30:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:30:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:30:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:30:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:30:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:30:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:30:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:30:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:30:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:30:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:30:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.113084604999997, 2024-09-17T07:30:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:30:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:30:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:30:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:30:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:30:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:30:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:30:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:30:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:30:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:30:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:30:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:30:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:30:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:30:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:30:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:30:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.113065005999999, 2024-09-17T07:30:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:30:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:30:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:30:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:30:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:30:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:30:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:30:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:30:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:30:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:30:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:30:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:30:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:30:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:30:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:30:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:13.056 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.113046431000001, 2024-09-17T07:30:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:30:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:30:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:30:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:30:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:30:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:30:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:30:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:30:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:30:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:30:13.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:30:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:30:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:30:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:30:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:30:13.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:30:13.061 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.061 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.061 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.061 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:13.061 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:13.061 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.113024003999996, 2024-09-17T07:30:13.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:30:13.061 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.061 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.061 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.062 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.062 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:30:13.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.062 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:13.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:13.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:13.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:13.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:30:13.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:30:13.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:30:13.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:30:13.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:30:13.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:30:13.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:30:13.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:30:13.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:30:13.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:30:13.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:30:13.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:30:13.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.065 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.066 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:13.066 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:13.066 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.112996186999993, 2024-09-17T07:30:13.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:30:13.066 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.066 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.066 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.066 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:30:13.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.066 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:13.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:13.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:13.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:30:13.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:30:13.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:30:13.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:30:13.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:30:13.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:30:13.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:30:13.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:30:13.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:30:13.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:30:13.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:30:13.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:30:13.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:30:13.070 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.070 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.070 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.071 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:13.071 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:13.071 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.112974242999996, 2024-09-17T07:30:13.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:30:13.071 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.071 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.071 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.072 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:30:13.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.072 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:13.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:13.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:13.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:30:13.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:30:13.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:30:13.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:30:13.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:30:13.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:30:13.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:30:13.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:30:13.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:30:13.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:30:13.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:30:13.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:30:13.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:30:13.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.075 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.076 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:13.076 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:13.076 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.112953138999998, 2024-09-17T07:30:13.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:30:13.076 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.076 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.076 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.077 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:30:13.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.077 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:13.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:13.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:13.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:30:13.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:30:13.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:30:13.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:30:13.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:13.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:30:13.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:30:13.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:30:13.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:30:13.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:30:13.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:30:13.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:30:13.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:30:13.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.080 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.080 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:13.080 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:13.081 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.112933536, 2024-09-17T07:30:13.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:30:13.081 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.081 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.081 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.081 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:30:13.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.081 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:13.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:13.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:30:13.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:30:13.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:30:13.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:30:13.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:30:13.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:30:13.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:30:13.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:30:13.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:30:13.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:30:13.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:30:13.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:30:13.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:30:13.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:30:13.084 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.085 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.085 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.085 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:13.085 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:13.085 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.159992422999998, 2024-09-17T07:30:13.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:30:13.085 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.086 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.086 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.086 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:30:13.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.086 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:13.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:13.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:30:13.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:30:13.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:30:13.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:13.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:30:13.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:30:13.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:30:13.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:30:13.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:30:13.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:30:13.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:30:13.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:30:13.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:30:13.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:30:13.090 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.090 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.090 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.090 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:13.090 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:13.090 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.159978762999998, 2024-09-17T07:30:13.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:30:13.090 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.090 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.091 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.091 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.091 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.091 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.091 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:30:13.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.091 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:13.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:13.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:13.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:30:13.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:30:13.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:30:13.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:30:13.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:30:13.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:30:13.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:30:13.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:30:13.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:30:13.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:30:13.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:30:13.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:30:13.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:30:13.095 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.095 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.095 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.095 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:13.095 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:13.095 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.159963980999997, 2024-09-17T07:30:13.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:30:13.095 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.095 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.095 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.096 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:30:13.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.096 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:13.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:13.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:13.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:30:13.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:30:13.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:30:13.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:30:13.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:13.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:30:13.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:30:13.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:30:13.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:30:13.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:30:13.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:30:13.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:30:13.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:30:13.099 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.099 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.099 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.100 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:13.100 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:13.100 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.159953090000002, 2024-09-17T07:30:13.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:30:13.100 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.100 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.100 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.100 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:30:13.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.100 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:13.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:13.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:13.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:13.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:30:13.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:30:13.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:30:13.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:30:13.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:30:13.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:30:13.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:30:13.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:30:13.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:30:13.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:30:13.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:30:13.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:30:13.104 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.104 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.104 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.104 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:13.104 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:13.104 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.15994353, 2024-09-17T07:30:13.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:30:13.104 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.104 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.104 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.105 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.105 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.105 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.105 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:30:13.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.105 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:13.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:13.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:30:13.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:13.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:30:13.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:30:13.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:30:13.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:30:13.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:30:13.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:30:13.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:30:13.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:30:13.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:30:13.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:30:13.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:30:13.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:30:13.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.110 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.111 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:13.111 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:13.111 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.15993297, 2024-09-17T07:30:13.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:30:13.111 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.111 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.111 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.111 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:30:13.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.111 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:13.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:13.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:30:13.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:30:13.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:30:13.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:30:13.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:30:13.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:30:13.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:30:13.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:30:13.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:30:13.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:30:13.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:30:13.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:30:13.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:30:13.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:30:13.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.115 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.115 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.115 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:13.116 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:13.118 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:13.117+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:13.190 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-17T07:30:13.218 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:13.217+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:13.271 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:13.271 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:13.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.272 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:115409 3.11 3:88e01b46:::rbd_header.118341193cb9:head [call lock.lock in=76b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104)", 2024-09-17T07:30:13.272 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:13.270653+0000", 2024-09-17T07:30:13.272 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.00061180999999999996, 2024-09-17T07:30:13.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00063222799999999998, 2024-09-17T07:30:13.272 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.272 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.272 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "waiting for sub ops", 2024-09-17T07:30:13.272 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.272 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.272 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.272 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115409 2024-09-17T07:30:13.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.273 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:13.270653+0000", 2024-09-17T07:30:13.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:13.270653+0000", 2024-09-17T07:30:13.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:13.270653+0000", 2024-09-17T07:30:13.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-17T07:30:13.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:13.270655+0000", 2024-09-17T07:30:13.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4380000000000002e-06 2024-09-17T07:30:13.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:13.270661+0000", 2024-09-17T07:30:13.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8610000000000003e-06 2024-09-17T07:30:13.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:13.270665+0000", 2024-09-17T07:30:13.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-17T07:30:13.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:13.270694+0000", 2024-09-17T07:30:13.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9008000000000001e-05 2024-09-17T07:30:13.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:13.270745+0000", 2024-09-17T07:30:13.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0831999999999997e-05 2024-09-17T07:30:13.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:30:13.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:13.270836+0000", 2024-09-17T07:30:13.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0861999999999998e-05 2024-09-17T07:30:13.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:13.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:13.271276+0000", 2024-09-17T07:30:13.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000440312 2024-09-17T07:30:13.276 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.276 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.276 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.276 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.276 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-09-17T07:30:13.277 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 1 2024-09-17T07:30:13.277 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:13.282 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:13.299 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-17T07:30:13.299 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:30:13.300 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-17T07:30:13.300 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-17T07:30:13.345 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:13.344+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:13.420 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:13.418+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:13.426 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:13.426 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:13.435 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:13.520 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:13.519+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:13.538 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:30:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 107.202822485, 2024-09-17T07:30:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:30:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:30:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:30:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:30:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:30:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:30:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:30:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:30:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:30:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:30:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:30:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:30:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:30:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:30:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:30:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:30:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:30:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:30:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:30:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:30:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:30:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:30:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:30:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:30:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:30:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:30:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:30:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:30:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:30:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:30:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7378 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7176192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:30:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.613780450999997, 2024-09-17T07:30:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073003980999999996, 2024-09-17T07:30:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7378 2024-09-17T07:30:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:30:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:30:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:30:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1799999999999999e-07 2024-09-17T07:30:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923713+0000", 2024-09-17T07:30:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.057e-06 2024-09-17T07:30:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923716+0000", 2024-09-17T07:30:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0599999999999999e-06 2024-09-17T07:30:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923719+0000", 2024-09-17T07:30:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8219999999999998e-06 2024-09-17T07:30:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996401+0000", 2024-09-17T07:30:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072681467999999999 2024-09-17T07:30:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996452+0000", 2024-09-17T07:30:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.134e-05 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996716+0000", 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000264016 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.613770334999998, 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:30:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:30:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:30:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:30:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:30:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:30:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:30:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:30:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:30:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:30:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:30:13.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:30:13.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:30:13.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:30:13.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.554 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.554 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7398 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6389760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.554 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:30:13.554 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.608501801000003, 2024-09-17T07:30:13.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072891580999999997, 2024-09-17T07:30:13.555 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.555 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.555 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.555 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7398 2024-09-17T07:30:13.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.555 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:30:13.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:30:13.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:30:13.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:30:13.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928995+0000", 2024-09-17T07:30:13.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0759999999999996e-06 2024-09-17T07:30:13.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928999+0000", 2024-09-17T07:30:13.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1609999999999996e-06 2024-09-17T07:30:13.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929002+0000", 2024-09-17T07:30:13.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8449999999999999e-06 2024-09-17T07:30:13.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001598+0000", 2024-09-17T07:30:13.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072596318000000007 2024-09-17T07:30:13.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001663+0000", 2024-09-17T07:30:13.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4995999999999999e-05 2024-09-17T07:30:13.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001882+0000", 2024-09-17T07:30:13.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021897299999999999 2024-09-17T07:30:13.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.558 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.559 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.559 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:13.559 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.608487218999997, 2024-09-17T07:30:13.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:30:13.559 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.559 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.559 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.559 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:30:13.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:30:13.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:30:13.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:30:13.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:30:13.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:30:13.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:30:13.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:30:13.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.563 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.563 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.563 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:13.563 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.608456239000006, 2024-09-17T07:30:13.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:30:13.563 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.563 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.563 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:30:13.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:30:13.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:30:13.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:30:13.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:30:13.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:30:13.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:30:13.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:30:13.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:30:13.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:30:13.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:30:13.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:30:13.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:30:13.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:30:13.567 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.567 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.567 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.567 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.567 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:13.567 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.608426190000003, 2024-09-17T07:30:13.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:30:13.567 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.567 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.567 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.568 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:30:13.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.568 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:13.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:13.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:30:13.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:30:13.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:30:13.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:30:13.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:30:13.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:30:13.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:30:13.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:30:13.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:30:13.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:30:13.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:30:13.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:30:13.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:30:13.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:30:13.571 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.571 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.571 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.572 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.572 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:13.572 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.608397783000001, 2024-09-17T07:30:13.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:30:13.572 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.572 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.572 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.572 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.572 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.572 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.572 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:30:13.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.573 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:13.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:13.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:13.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:13.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:30:13.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:30:13.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:30:13.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:30:13.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:30:13.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:30:13.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:30:13.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:30:13.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:30:13.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:30:13.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:30:13.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:30:13.576 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.576 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.576 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.576 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.576 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:13.576 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.608377962000006, 2024-09-17T07:30:13.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:30:13.576 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.577 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.577 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.577 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:30:13.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.577 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:13.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:13.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:30:13.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:30:13.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:30:13.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:30:13.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:30:13.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:30:13.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:30:13.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:30:13.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:30:13.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:30:13.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:30:13.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:30:13.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:30:13.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:30:13.580 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.580 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.580 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.581 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.581 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:13.581 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.608353710000003, 2024-09-17T07:30:13.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:30:13.581 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.581 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.581 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.581 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:30:13.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.581 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:30:13.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:30:13.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:30:13.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:30:13.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:30:13.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:30:13.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:30:13.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:30:13.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:30:13.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:30:13.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:30:13.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:30:13.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:30:13.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:30:13.585 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.585 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.585 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.585 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7418 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7700480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.585 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:30:13.585 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.602947686999997, 2024-09-17T07:30:13.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072752965000000003, 2024-09-17T07:30:13.585 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.585 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.585 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.586 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7418 2024-09-17T07:30:13.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.586 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:30:13.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:30:13.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:30:13.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3599999999999999e-07 2024-09-17T07:30:13.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934547+0000", 2024-09-17T07:30:13.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3350000000000001e-06 2024-09-17T07:30:13.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934554+0000", 2024-09-17T07:30:13.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9859999999999997e-06 2024-09-17T07:30:13.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934557+0000", 2024-09-17T07:30:13.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.818e-06 2024-09-17T07:30:13.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007104+0000", 2024-09-17T07:30:13.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072546779000000006 2024-09-17T07:30:13.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007147+0000", 2024-09-17T07:30:13.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2904000000000001e-05 2024-09-17T07:30:13.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007298+0000", 2024-09-17T07:30:13.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015080700000000001 2024-09-17T07:30:13.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.589 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.590 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.590 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:13.590 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.602931122000001, 2024-09-17T07:30:13.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:30:13.590 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.590 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.590 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.590 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:30:13.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.590 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:13.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:13.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:30:13.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:30:13.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:30:13.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:30:13.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:30:13.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:30:13.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:30:13.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:30:13.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:30:13.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:30:13.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:30:13.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:30:13.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:30:13.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:30:13.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.594 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.594 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.594 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:13.594 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.602884247000006, 2024-09-17T07:30:13.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:30:13.594 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.594 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.594 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.595 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:30:13.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.595 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:13.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:13.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:13.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:30:13.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:30:13.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:30:13.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:30:13.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:30:13.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:30:13.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:30:13.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:30:13.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:30:13.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:30:13.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:30:13.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:30:13.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:30:13.598 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.598 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.598 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.598 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.599 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:13.599 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.602869542999997, 2024-09-17T07:30:13.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:30:13.599 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.599 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.599 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.599 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:30:13.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.599 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:13.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:13.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:13.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:13.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:30:13.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:30:13.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:30:13.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:30:13.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:30:13.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:30:13.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:30:13.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:30:13.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:30:13.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:30:13.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:30:13.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:30:13.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.603 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.603 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.603 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:13.603 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.602846447999994, 2024-09-17T07:30:13.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:30:13.603 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.603 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.603 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.604 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:30:13.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.604 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:13.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:13.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:13.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:30:13.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:30:13.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:30:13.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:30:13.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:30:13.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:30:13.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:30:13.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:30:13.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:30:13.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:30:13.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:30:13.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:30:13.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:30:13.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.607 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.608 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.608 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:13.608 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.602815073000002, 2024-09-17T07:30:13.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:30:13.608 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.608 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.608 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.608 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:30:13.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.609 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:13.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:13.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:30:13.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:30:13.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:30:13.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:30:13.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:30:13.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:30:13.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:30:13.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:30:13.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:30:13.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:30:13.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:30:13.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:30:13.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:30:13.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:30:13.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.612 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.612 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.612 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:13.612 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.602799813000004, 2024-09-17T07:30:13.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:30:13.613 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.613 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.613 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.613 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:30:13.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.613 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:13.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:13.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:13.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:30:13.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:30:13.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:30:13.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:30:13.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:30:13.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:30:13.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:30:13.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:30:13.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:30:13.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:30:13.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:30:13.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:30:13.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:30:13.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.617 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.617 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.617 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:13.617 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.602779742999999, 2024-09-17T07:30:13.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:30:13.618 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.618 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.618 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.618 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:30:13.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.618 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:13.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:13.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:13.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:30:13.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:30:13.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:30:13.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:30:13.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:30:13.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:30:13.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:30:13.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:30:13.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:30:13.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:30:13.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:30:13.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:30:13.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:30:13.621 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.622 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.622 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.622 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:13.622 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.602767803000006, 2024-09-17T07:30:13.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:30:13.622 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.622 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.622 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.622 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.622 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.623 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:30:13.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.623 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:13.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:13.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:13.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:30:13.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:30:13.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:30:13.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:30:13.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:30:13.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:30:13.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:30:13.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:30:13.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:30:13.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:30:13.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:30:13.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:30:13.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:30:13.626 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.626 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.626 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.626 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:13.626 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:13.627 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.602744645000001, 2024-09-17T07:30:13.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:30:13.627 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:13.627 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:13.627 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:13.627 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:13.627 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:13.627 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:13.627 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:30:13.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.627 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:13.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:13.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:13.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:13.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:13.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:13.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:13.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:30:13.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:30:13.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:13.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:30:13.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:30:13.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:13.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:30:13.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:30:13.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:13.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:30:13.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:30:13.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:13.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:30:13.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:30:13.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:13.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:30:13.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:30:13.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:13.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:13.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:13.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:30:13.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:30:13.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.631 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:13.631 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:13.631 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:13.632 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-17T07:30:13.614+0000 7f6d53957640 -1 mon.a@0(leader).osd e104 definitely_dead 0 2024-09-17T07:30:13.633 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:13.633 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:13.631+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:13.733 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:13.731+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:13.741 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:13.741 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:13.741 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:13.741 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:13.750 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:13.834 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:13.833+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:13.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:30:13.896 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:13.896 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:13.896 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:13.897 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:13.897 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:13.905 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:13.935 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:13.934+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:14.036 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:14.035+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:14.037 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:14.037 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:14.054 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:14.054 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:14.067 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [] 2024-09-17T07:30:14.067 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:14.070 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:14.071 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:14.076 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:14.087 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:14.088 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:14.106 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:14.106 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:14.123 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:14.124 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:14.137 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:14.136+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:14.140 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:14.140 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:14.158 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:14.158 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:14.191 INFO:teuthology.orchestra.run.smithi045.stderr:marked in osd.3. 2024-09-17T07:30:14.207 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:14.207 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:14.217 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-17T07:30:14.217 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:14.259 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:14.257+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:14.340 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:14.339+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:14.364 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:14.364 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:14.373 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:14.441 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:14.439+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:14.480 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:14.481 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:14.489 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:14.541 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:14.541+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:14.637 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:14.637 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:14.642 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:14.641+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:14.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:14.745 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:14.744+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:14.794 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:14.794 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:14.803 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:14.847 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:14.845+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:14.949 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:14.948+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:14.950 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:14.951 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:14.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:15.050 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:15.049+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:15.108 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:15.116 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:15.151 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:15.149+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:15.209 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T07:30:15.253 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:15.252+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:15.269 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:15.270 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:15.279 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:15.353 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:15.352+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:15.429 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:15.429 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:15.438 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:15.482 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:15.481+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:15.554 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:15.554 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:15.554 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:15.553+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:15.563 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:15.655 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:15.654+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:15.707 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:15.708 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:15.716 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:15.759 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:15.758+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:15.818 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:15.819 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:15.827 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:15.869 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:15.867+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:15.957 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:15.956+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:15.971 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:15.971 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:15.980 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:16.063 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:16.062+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:16.122 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:16.122 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:16.131 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:16.170 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:30:16.172 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:16.171+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:16.197 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-17T07:30:16.264 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:16.263+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:16.278 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:16.278 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:16.287 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:16.329 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:16.329 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:16.351 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:16.361 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:16.360+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:16.372 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:16.372 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:16.403 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:16.403 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:16.413 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:16.462 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:16.461+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:16.557 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:16.557 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:16.563 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:16.562+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:16.565 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:16.596 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:16.617 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:16.618 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:16.639 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:16.639 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:16.657 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:16.657 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:16.664 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:16.663+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:16.678 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:16.678 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:16.711 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:16.712 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:16.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:16.712 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:16.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.712 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:16.712 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:16.712 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.804814845999999, 2024-09-17T07:30:16.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:30:16.712 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.712 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.712 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.713 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.713 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.713 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.713 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:30:16.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.713 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:16.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:16.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:16.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:30:16.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:30:16.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:30:16.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:30:16.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:30:16.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:30:16.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:30:16.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:30:16.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:30:16.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:30:16.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:30:16.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:30:16.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:30:16.716 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.717 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.717 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.717 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:16.717 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:16.717 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.804796168999999, 2024-09-17T07:30:16.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:30:16.717 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.717 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.717 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.718 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:30:16.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.718 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:16.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:16.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:30:16.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:16.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:30:16.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:30:16.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:30:16.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:30:16.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:30:16.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:30:16.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:30:16.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:30:16.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:30:16.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:30:16.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:30:16.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:30:16.721 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.721 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.721 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.721 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:16.721 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:16.721 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.804777815999998, 2024-09-17T07:30:16.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:30:16.722 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.722 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.722 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.722 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.722 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.722 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.722 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:30:16.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.722 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:16.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:16.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:16.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:30:16.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:30:16.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:30:16.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:30:16.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:30:16.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:30:16.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:30:16.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:30:16.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:30:16.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:30:16.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:30:16.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:30:16.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:30:16.726 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.726 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.726 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.726 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:16.726 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:16.726 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.804758218000003, 2024-09-17T07:30:16.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:30:16.726 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.726 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.727 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.727 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.727 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.727 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.727 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:30:16.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.727 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:16.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:16.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:30:16.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:30:16.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:30:16.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:30:16.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:30:16.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:30:16.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:30:16.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:30:16.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:30:16.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:30:16.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:30:16.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:30:16.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:30:16.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:30:16.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.730 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.731 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:16.731 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:16.731 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.804738263000004, 2024-09-17T07:30:16.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:30:16.731 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.731 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.731 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.731 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:30:16.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.732 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:16.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:16.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:16.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:30:16.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:30:16.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:30:16.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:30:16.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:30:16.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:30:16.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:30:16.735 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.735 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.735 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.735 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:16.735 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:16.735 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.804719607999999, 2024-09-17T07:30:16.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:30:16.736 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.736 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.736 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.736 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.736 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.736 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.736 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:30:16.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.736 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:16.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:16.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:16.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:30:16.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:30:16.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:30:16.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:30:16.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:30:16.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:30:16.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:30:16.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:30:16.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:30:16.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:30:16.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:30:16.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:30:16.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:30:16.739 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.739 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.739 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.740 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:16.740 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:16.740 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.804701132000005, 2024-09-17T07:30:16.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:30:16.740 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.740 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.740 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.740 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.740 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.740 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.740 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:30:16.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.741 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:16.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:16.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:30:16.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:30:16.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:30:16.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:30:16.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:30:16.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:30:16.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:30:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:30:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:30:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:30:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:30:16.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:30:16.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:30:16.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:30:16.744 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.744 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.744 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.744 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:16.744 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:16.744 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.804681532999993, 2024-09-17T07:30:16.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:30:16.745 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.745 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.745 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.745 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:30:16.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.745 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:16.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:16.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:16.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:30:16.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:30:16.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:30:16.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:30:16.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:30:16.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:30:16.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:30:16.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:30:16.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:30:16.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:30:16.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:30:16.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:30:16.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:30:16.749 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.749 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.749 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.749 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:16.749 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:16.749 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.804662957999994, 2024-09-17T07:30:16.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:30:16.749 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.749 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.749 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.750 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.750 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.750 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.750 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:30:16.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.750 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:16.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:16.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:30:16.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:30:16.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:30:16.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:30:16.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:30:16.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:30:16.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:30:16.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:30:16.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:30:16.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:30:16.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:30:16.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:30:16.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:30:16.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:30:16.754 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.754 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.754 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.754 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:16.754 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:16.754 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.804640531000004, 2024-09-17T07:30:16.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:30:16.754 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.754 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.754 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.754 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.755 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.755 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:30:16.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.755 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:16.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:16.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:16.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:16.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:30:16.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:30:16.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:30:16.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:30:16.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:30:16.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:30:16.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:30:16.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:30:16.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:30:16.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:30:16.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:30:16.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:30:16.758 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.758 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.758 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.758 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:16.758 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:16.759 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.804612714000001, 2024-09-17T07:30:16.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:30:16.759 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.759 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.759 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.759 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.759 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.759 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.759 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:30:16.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.759 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:16.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:16.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:16.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:30:16.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:30:16.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:30:16.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:30:16.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:30:16.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:30:16.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:30:16.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:30:16.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:30:16.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:30:16.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:30:16.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:30:16.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:30:16.762 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.762 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.762 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.763 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:16.763 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:16.763 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.804590770000004, 2024-09-17T07:30:16.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:30:16.763 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.763 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.763 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.763 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.763 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.763 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.763 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:30:16.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.764 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:16.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:16.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:16.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:30:16.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:30:16.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:30:16.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:30:16.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:30:16.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:30:16.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:30:16.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:30:16.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:30:16.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:30:16.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:30:16.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:30:16.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:30:16.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.767 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.767 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:16.767 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:16.767 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.804569666000006, 2024-09-17T07:30:16.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:30:16.767 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.767 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.767 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.767 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.768 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:30:16.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.768 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:16.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:16.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:16.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:30:16.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:30:16.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:30:16.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:30:16.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:16.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:30:16.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:30:16.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:30:16.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:30:16.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:30:16.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:30:16.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:30:16.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:30:16.771 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.771 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.771 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.771 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:16.771 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:16.771 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 72.804550062999994, 2024-09-17T07:30:16.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:30:16.772 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.772 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.772 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.772 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.772 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.772 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.772 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:30:16.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.772 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:16.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:16.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:30:16.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:30:16.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:30:16.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:30:16.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:30:16.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:30:16.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:30:16.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:30:16.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:30:16.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:30:16.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:30:16.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:30:16.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:30:16.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:30:16.775 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.776 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.776 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.776 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:16.776 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:16.776 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.851608949999999, 2024-09-17T07:30:16.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:30:16.776 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.776 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.776 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.776 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.776 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.777 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.777 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:30:16.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.777 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:16.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:16.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:30:16.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:30:16.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:30:16.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:16.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:30:16.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:30:16.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:30:16.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:30:16.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:30:16.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:30:16.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:30:16.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:30:16.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:30:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:30:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.851595289999999, 2024-09-17T07:30:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:30:16.780 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.781 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.781 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.781 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.781 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.781 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.781 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:30:16.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.781 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:16.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:16.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:16.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:30:16.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:30:16.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:30:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:30:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:30:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:30:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:30:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:30:16.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:30:16.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:30:16.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:30:16.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:30:16.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:30:16.784 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.784 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.784 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.785 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:16.785 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:16.785 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.851580507999998, 2024-09-17T07:30:16.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:30:16.785 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.785 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.785 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.785 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:30:16.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.786 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:16.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:16.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:16.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:30:16.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:30:16.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:30:16.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:30:16.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:16.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:30:16.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:30:16.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:30:16.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:30:16.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:30:16.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:30:16.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:30:16.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:30:16.789 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.789 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.789 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.789 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:16.790 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:16.790 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.851569617000003, 2024-09-17T07:30:16.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:30:16.790 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.790 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.790 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.790 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.790 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.790 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.790 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:30:16.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.790 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:16.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:16.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:16.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:16.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:30:16.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:30:16.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:30:16.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:30:16.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:30:16.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:30:16.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:30:16.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:30:16.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:30:16.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:30:16.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:30:16.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:30:16.793 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.793 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.794 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.794 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:16.794 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:16.794 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.851560057, 2024-09-17T07:30:16.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:30:16.794 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.794 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.794 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.794 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.794 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.794 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.794 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:30:16.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.795 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:16.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:16.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:30:16.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:16.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:30:16.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:30:16.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:30:16.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:30:16.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:30:16.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:30:16.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:30:16.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:30:16.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:30:16.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:30:16.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:30:16.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:30:16.798 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.798 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.798 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.798 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:16.798 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:16.798 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.851549497000001, 2024-09-17T07:30:16.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:30:16.798 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:16.798 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:16.799 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:16.799 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:16.799 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:16.799 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:16.799 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:30:16.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.799 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:16.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:16.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:16.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:16.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:16.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:16.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:16.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:30:16.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:30:16.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:16.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:30:16.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:30:16.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:16.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:30:16.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:30:16.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:16.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:30:16.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:30:16.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:16.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:30:16.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:30:16.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:16.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:30:16.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:30:16.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:16.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:16.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:16.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:30:16.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:30:16.803 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.804 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.804 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.804 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:16.804 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:16.804 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:16.805 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:16.806 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:16.805+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:16.907 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:16.906+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:16.952 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:16.952 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:16.952 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:16.952 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:16.961 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:17.008 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:17.007+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:17.064 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:17.064 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:17.064 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:17.064 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:17.064 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:17.072 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:17.109 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:17.108+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:17.190 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.190 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:30:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 110.854015391, 2024-09-17T07:30:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:30:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:30:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:30:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:30:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.191 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:30:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:30:17.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:30:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:30:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:30:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:30:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:30:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:30:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:30:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:30:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:30:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:30:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:30:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:30:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:30:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:30:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:30:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:30:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:30:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:30:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:30:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:30:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:30:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:30:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:30:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:30:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7378 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7176192~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:30:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.264973357000002, 2024-09-17T07:30:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073003980999999996, 2024-09-17T07:30:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.197 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7378 2024-09-17T07:30:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:30:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:30:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923712+0000", 2024-09-17T07:30:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1799999999999999e-07 2024-09-17T07:30:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923713+0000", 2024-09-17T07:30:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.057e-06 2024-09-17T07:30:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923716+0000", 2024-09-17T07:30:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0599999999999999e-06 2024-09-17T07:30:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923719+0000", 2024-09-17T07:30:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8219999999999998e-06 2024-09-17T07:30:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996401+0000", 2024-09-17T07:30:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072681467999999999 2024-09-17T07:30:17.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996452+0000", 2024-09-17T07:30:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.134e-05 2024-09-17T07:30:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996716+0000", 2024-09-17T07:30:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000264016 2024-09-17T07:30:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.264963241000004, 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:30:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:30:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:30:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:30:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:30:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:30:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:30:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:30:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:30:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:30:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:30:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:30:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:30:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:30:17.205 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7398 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6389760~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:30:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.259694706999994, 2024-09-17T07:30:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072891580999999997, 2024-09-17T07:30:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.206 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7398 2024-09-17T07:30:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:30:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:30:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928991+0000", 2024-09-17T07:30:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:30:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928995+0000", 2024-09-17T07:30:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0759999999999996e-06 2024-09-17T07:30:17.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928999+0000", 2024-09-17T07:30:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1609999999999996e-06 2024-09-17T07:30:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929002+0000", 2024-09-17T07:30:17.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8449999999999999e-06 2024-09-17T07:30:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001598+0000", 2024-09-17T07:30:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072596318000000007 2024-09-17T07:30:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001663+0000", 2024-09-17T07:30:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4995999999999999e-05 2024-09-17T07:30:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001882+0000", 2024-09-17T07:30:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021897299999999999 2024-09-17T07:30:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:17.214 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.259680125000003, 2024-09-17T07:30:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:30:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:30:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:30:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:30:17.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:30:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:30:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:30:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:30:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:30:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:30:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:30:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:30:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:30:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:30:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:30:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.259649144999997, 2024-09-17T07:30:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:30:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:30:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:17.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:30:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:30:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:30:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:30:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:30:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:30:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:30:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:30:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:30:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:30:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:30:17.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:30:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:30:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:30:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.259619095999994, 2024-09-17T07:30:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:30:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:30:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.225 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:30:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:30:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:30:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:30:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:30:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:30:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:30:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:30:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:30:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:30:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:30:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:30:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:30:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:30:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.259590689000007, 2024-09-17T07:30:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:30:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:30:17.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:17.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:30:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:30:17.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:30:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:30:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:30:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:30:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:30:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:30:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:30:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:30:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:30:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:30:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.234 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.259570867999997, 2024-09-17T07:30:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:30:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:30:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.235 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:30:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:30:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:30:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:30:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:30:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:30:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:30:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:30:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:30:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:30:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:30:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:30:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:30:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:30:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.259546615999994, 2024-09-17T07:30:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:30:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.239 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:30:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:30:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:30:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:30:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:30:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:30:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:30:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:30:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:30:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:30:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:30:17.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:30:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:30:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:30:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:30:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.243 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7418 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7700480~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:30:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.254140593000002, 2024-09-17T07:30:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072752965000000003, 2024-09-17T07:30:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7418 2024-09-17T07:30:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:30:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:30:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934545+0000", 2024-09-17T07:30:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3599999999999999e-07 2024-09-17T07:30:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934547+0000", 2024-09-17T07:30:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3350000000000001e-06 2024-09-17T07:30:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934554+0000", 2024-09-17T07:30:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9859999999999997e-06 2024-09-17T07:30:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934557+0000", 2024-09-17T07:30:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.818e-06 2024-09-17T07:30:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007104+0000", 2024-09-17T07:30:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072546779000000006 2024-09-17T07:30:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007147+0000", 2024-09-17T07:30:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2904000000000001e-05 2024-09-17T07:30:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007298+0000", 2024-09-17T07:30:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015080700000000001 2024-09-17T07:30:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.247 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.254124028000007, 2024-09-17T07:30:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:30:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.248 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:30:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:30:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:30:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:30:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:30:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:30:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:30:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:30:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:30:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:30:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:30:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:30:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:30:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:30:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:30:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.254077152999997, 2024-09-17T07:30:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:30:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.252 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:30:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:30:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:30:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:30:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:30:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:30:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:30:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:30:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:30:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:30:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:30:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:30:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:30:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:30:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.256 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.254062449000003, 2024-09-17T07:30:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:30:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:30:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:17.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:30:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:30:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:30:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:30:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:30:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:30:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:30:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:30:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:30:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:30:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:30:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:30:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.260 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.254039354, 2024-09-17T07:30:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:30:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.261 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:30:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:30:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:30:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:30:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:30:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:30:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:30:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:30:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:30:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:30:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:30:17.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:30:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:30:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:30:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:17.265 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.254007978999994, 2024-09-17T07:30:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:30:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:30:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:17.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:30:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:30:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:30:17.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:30:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:30:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:30:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:30:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:30:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:30:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:30:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:30:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:30:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:30:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:30:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.253992718999996, 2024-09-17T07:30:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:30:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:30:17.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:30:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:30:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:30:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:30:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:30:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:30:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:30:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:30:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:30:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:30:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:30:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:30:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:30:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.253972649000005, 2024-09-17T07:30:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:30:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.274 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:30:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:30:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:30:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:30:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:30:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:30:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:30:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:30:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:30:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:30:17.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:30:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:30:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:30:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:30:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.253960708999998, 2024-09-17T07:30:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:30:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:30:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:17.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:30:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:30:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:30:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:30:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:30:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:30:17.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:30:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:30:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:30:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:30:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:30:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:30:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:30:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.253937551000007, 2024-09-17T07:30:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:30:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:17.283 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:30:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:17.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:30:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:30:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:30:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:30:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:30:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:30:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:30:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:30:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:30:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:30:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:30:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:30:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:30:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:30:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:17.287 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:17.287 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:17.289 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:17.289 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:17.287+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:17.390 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:17.388+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:17.417 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:17.417 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:17.417 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:17.417 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:17.425 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:17.490 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:17.490+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:17.554 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:17.554 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:17.554 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:17.555 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:17.555 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:17.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:17.591 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:17.590+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:17.691 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:17.691 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:17.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:17.692 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:17.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.692 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6, version: 32'24, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6@32'24, size: 8388608, copy_subset: [(0, 8388608)], clone_subset: {}, snapset: 6=[]:{6: [6]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:17.692 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.795106+0000", 2024-09-17T07:30:17.692 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.89389597899999995, 2024-09-17T07:30:17.692 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017875429999999999, 2024-09-17T07:30:17.692 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.692 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.692 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.692 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.693 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.693 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.795106+0000", 2024-09-17T07:30:17.693 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.693 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.693 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.795106+0000", 2024-09-17T07:30:17.693 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.693 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.694 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.795107+0000", 2024-09-17T07:30:17.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0800000000000004e-07 2024-09-17T07:30:17.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.694 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.694 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.796120+0000", 2024-09-17T07:30:17.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001012712 2024-09-17T07:30:17.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.694 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.694 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.796138+0000", 2024-09-17T07:30:17.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7944999999999998e-05 2024-09-17T07:30:17.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.796148+0000", 2024-09-17T07:30:17.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0465000000000001e-05 2024-09-17T07:30:17.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.796216+0000", 2024-09-17T07:30:17.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7601000000000005e-05 2024-09-17T07:30:17.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.796224+0000", 2024-09-17T07:30:17.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3329999999999997e-06 2024-09-17T07:30:17.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.796894+0000", 2024-09-17T07:30:17.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00066987899999999996 2024-09-17T07:30:17.696 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.696 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.696 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.697 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6, version: 32'24, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6@32'24, size: 8388608, copy_subset: [(0, 8388608)], clone_subset: {}, snapset: 6=[]:{6: [6]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:17.697 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.798443+0000", 2024-09-17T07:30:17.697 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.89055880899999995, 2024-09-17T07:30:17.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020776530000000001, 2024-09-17T07:30:17.697 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.697 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.697 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.697 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.697 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.698 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.798443+0000", 2024-09-17T07:30:17.698 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.698 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.698 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.698 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.798443+0000", 2024-09-17T07:30:17.698 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.698 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.698 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.698 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.798444+0000", 2024-09-17T07:30:17.698 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4099999999999998e-07 2024-09-17T07:30:17.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.699 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.699 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.799902+0000", 2024-09-17T07:30:17.699 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014576979999999999 2024-09-17T07:30:17.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.699 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.699 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.799919+0000", 2024-09-17T07:30:17.699 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7007000000000002e-05 2024-09-17T07:30:17.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.700 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.700 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.799929+0000", 2024-09-17T07:30:17.700 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0276e-05 2024-09-17T07:30:17.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.700 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.700 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.700 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.799996+0000", 2024-09-17T07:30:17.700 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6957000000000003e-05 2024-09-17T07:30:17.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.700 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.700 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.701 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.800004+0000", 2024-09-17T07:30:17.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4460000000000001e-06 2024-09-17T07:30:17.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.701 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.701 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.800521+0000", 2024-09-17T07:30:17.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00051662799999999999 2024-09-17T07:30:17.701 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.701 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.701 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.702 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6, version: 32'24, data_included: [3276800~196608,3538944~393216,3934282~52307,3997696~196608,4227072~131072,4456448~78765], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6@32'24, size: 8388608, copy_subset: [(0, 8388608)], clone_subset: {}, snapset: 6=[]:{6: [6]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4535213, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3276800, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:17.702 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.804994+0000", 2024-09-17T07:30:17.702 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.88400779500000004, 2024-09-17T07:30:17.702 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001902064, 2024-09-17T07:30:17.702 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.702 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.702 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.702 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.702 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.703 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.804994+0000", 2024-09-17T07:30:17.703 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.703 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.703 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.804994+0000", 2024-09-17T07:30:17.703 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.703 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.703 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.804995+0000", 2024-09-17T07:30:17.704 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5000000000000003e-07 2024-09-17T07:30:17.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.704 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.704 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.704 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.805957+0000", 2024-09-17T07:30:17.704 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00096245300000000005 2024-09-17T07:30:17.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.704 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.704 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.704 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.805973+0000", 2024-09-17T07:30:17.704 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5815999999999999e-05 2024-09-17T07:30:17.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.705 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.705 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.805984+0000", 2024-09-17T07:30:17.705 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0896999999999999e-05 2024-09-17T07:30:17.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.705 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.705 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.806056+0000", 2024-09-17T07:30:17.705 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1854999999999996e-05 2024-09-17T07:30:17.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.706 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.706 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.806065+0000", 2024-09-17T07:30:17.706 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3840000000000003e-06 2024-09-17T07:30:17.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.706 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.707 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.806897+0000", 2024-09-17T07:30:17.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083110900000000001 2024-09-17T07:30:17.707 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.707 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.707 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.707 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6, version: 32'24, data_included: [4535213~52307,4685824~131072,4848394~42592,4915200~131072,5144576~131072,5373952~131072,5603328~131072,5767168~298317], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6@32'24, size: 8388608, copy_subset: [(0, 8388608)], clone_subset: {}, snapset: 6=[]:{6: [6]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 6065485, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4535213, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:17.707 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.808547+0000", 2024-09-17T07:30:17.707 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.88045487200000006, 2024-09-17T07:30:17.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019945240000000001, 2024-09-17T07:30:17.707 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.708 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.708 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.708 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.708 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.708 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.708 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.808547+0000", 2024-09-17T07:30:17.708 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.708 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.708 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.708 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.808547+0000", 2024-09-17T07:30:17.709 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.709 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.709 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.808549+0000", 2024-09-17T07:30:17.709 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6339999999999999e-06 2024-09-17T07:30:17.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.709 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.709 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.809490+0000", 2024-09-17T07:30:17.709 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00094110299999999999 2024-09-17T07:30:17.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.710 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.710 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.809508+0000", 2024-09-17T07:30:17.710 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7382000000000001e-05 2024-09-17T07:30:17.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.710 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.710 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.809520+0000", 2024-09-17T07:30:17.710 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2503999999999999e-05 2024-09-17T07:30:17.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.711 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.711 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.809592+0000", 2024-09-17T07:30:17.711 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1649000000000005e-05 2024-09-17T07:30:17.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.711 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.711 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.711 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.809601+0000", 2024-09-17T07:30:17.711 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0420000000000008e-06 2024-09-17T07:30:17.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.711 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.711 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.711 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.810542+0000", 2024-09-17T07:30:17.712 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00094121000000000005 2024-09-17T07:30:17.712 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.712 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.712 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.712 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.712 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6, version: 32'24, data_included: [6065485~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6@32'24, size: 8388608, copy_subset: [(0, 8388608)], clone_subset: {}, snapset: 6=[]:{6: [6]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7114061, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 6065485, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:17.712 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.811872+0000", 2024-09-17T07:30:17.712 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.877130309, 2024-09-17T07:30:17.712 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017456220000000001, 2024-09-17T07:30:17.712 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.713 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.713 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.713 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.713 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.713 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.811872+0000", 2024-09-17T07:30:17.713 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.713 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.713 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.811872+0000", 2024-09-17T07:30:17.713 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.714 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.714 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.811873+0000", 2024-09-17T07:30:17.714 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3600000000000004e-07 2024-09-17T07:30:17.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.714 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.714 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.812849+0000", 2024-09-17T07:30:17.714 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097618900000000003 2024-09-17T07:30:17.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.715 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.715 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.715 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.812863+0000", 2024-09-17T07:30:17.715 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4632e-05 2024-09-17T07:30:17.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.715 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.715 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.715 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.812874+0000", 2024-09-17T07:30:17.715 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0769e-05 2024-09-17T07:30:17.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.715 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.716 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.716 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.812934+0000", 2024-09-17T07:30:17.716 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9759999999999997e-05 2024-09-17T07:30:17.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.716 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.716 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.716 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.812942+0000", 2024-09-17T07:30:17.716 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3890000000000005e-06 2024-09-17T07:30:17.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.716 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.716 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.716 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.813618+0000", 2024-09-17T07:30:17.717 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00067534699999999997 2024-09-17T07:30:17.717 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.717 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.717 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.717 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.717 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ab68203a:::rbd_data.1183515b0721.000000000000000b:head, version: 39'59, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:ab68203a:::rbd_data.1183515b0721.000000000000000b:head@39'59, size: 9257781, copy_subset: [(0, 9257781)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:17.717 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.844924+0000", 2024-09-17T07:30:17.717 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.84407837900000005, 2024-09-17T07:30:17.717 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019831010000000001, 2024-09-17T07:30:17.717 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.718 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.718 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.718 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.718 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.718 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.718 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.844924+0000", 2024-09-17T07:30:17.718 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.718 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.718 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.718 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.844924+0000", 2024-09-17T07:30:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.844924+0000", 2024-09-17T07:30:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7899999999999998e-07 2024-09-17T07:30:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.845887+0000", 2024-09-17T07:30:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00096215099999999996 2024-09-17T07:30:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.719 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.720 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.720 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.845902+0000", 2024-09-17T07:30:17.720 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5466000000000001e-05 2024-09-17T07:30:17.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.720 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.720 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.720 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.845913+0000", 2024-09-17T07:30:17.720 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0873000000000001e-05 2024-09-17T07:30:17.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.720 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.720 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.721 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.845999+0000", 2024-09-17T07:30:17.721 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6205000000000006e-05 2024-09-17T07:30:17.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.721 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.721 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.721 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.846015+0000", 2024-09-17T07:30:17.721 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5381000000000001e-05 2024-09-17T07:30:17.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.721 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.721 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.721 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.846907+0000", 2024-09-17T07:30:17.721 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089244599999999997 2024-09-17T07:30:17.722 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.722 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.722 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.722 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.722 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ab68203a:::rbd_data.1183515b0721.000000000000000b:head, version: 39'59, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ab68203a:::rbd_data.1183515b0721.000000000000000b:head@39'59, size: 9257781, copy_subset: [(0, 9257781)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:17.722 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.848302+0000", 2024-09-17T07:30:17.723 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.84069994999999997, 2024-09-17T07:30:17.723 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001905503, 2024-09-17T07:30:17.723 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.723 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.723 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.723 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.723 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.723 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.848302+0000", 2024-09-17T07:30:17.723 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.724 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.724 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.848302+0000", 2024-09-17T07:30:17.724 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.724 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.724 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.848303+0000", 2024-09-17T07:30:17.724 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5300000000000004e-07 2024-09-17T07:30:17.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.724 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.724 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.849231+0000", 2024-09-17T07:30:17.725 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000928344 2024-09-17T07:30:17.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.725 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.725 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.725 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.849248+0000", 2024-09-17T07:30:17.725 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.632e-05 2024-09-17T07:30:17.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.725 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.725 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.725 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.849258+0000", 2024-09-17T07:30:17.725 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0342e-05 2024-09-17T07:30:17.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.726 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.726 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.849346+0000", 2024-09-17T07:30:17.726 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7767000000000002e-05 2024-09-17T07:30:17.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.726 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.726 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.849362+0000", 2024-09-17T07:30:17.726 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5772000000000001e-05 2024-09-17T07:30:17.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.727 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.727 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.727 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.850208+0000", 2024-09-17T07:30:17.727 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000846305 2024-09-17T07:30:17.727 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.727 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.727 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.727 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.727 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ae97839c:::rbd_data.11839a97b753.0000000000000004:head, version: 55'111, data_included: [0~16384,81920~16384,163840~16384,196608~16384,245760~16384,278528~16384,360448~16384,442368~16384,524288~16384,704512~16384,786432~16384,868352~16384,950272~16384,983040~16384,1032192~16384,1064960~16384,1146880~16384,1228800~16384,1310720~16384,1343488~16384,1392640~16384,1425408~32768,1523712~16384], data_size: 393216, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:ae97839c:::rbd_data.11839a97b753.0000000000000004:head@55'111, size: 1540096, copy_subset: [(0, 1540096)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1540096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:17.727 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.884937+0000", 2024-09-17T07:30:17.728 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.80406508899999996, 2024-09-17T07:30:17.728 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025497419999999998, 2024-09-17T07:30:17.728 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.728 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.728 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.728 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.728 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.728 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.728 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.884937+0000", 2024-09-17T07:30:17.728 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.729 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.729 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.884937+0000", 2024-09-17T07:30:17.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.729 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.729 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.884938+0000", 2024-09-17T07:30:17.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5499999999999998e-07 2024-09-17T07:30:17.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.729 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885283+0000", 2024-09-17T07:30:17.730 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00034585300000000002 2024-09-17T07:30:17.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.730 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885301+0000", 2024-09-17T07:30:17.730 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7207999999999999e-05 2024-09-17T07:30:17.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.730 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885316+0000", 2024-09-17T07:30:17.730 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4873e-05 2024-09-17T07:30:17.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.731 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885416+0000", 2024-09-17T07:30:17.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000100456 2024-09-17T07:30:17.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.731 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885432+0000", 2024-09-17T07:30:17.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6390000000000001e-05 2024-09-17T07:30:17.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.732 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.887487+0000", 2024-09-17T07:30:17.732 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020545070000000001 2024-09-17T07:30:17.732 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.732 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.732 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.732 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:aca46c33:::rbd_data.11835a3b5a27.0000000000000007:1c, version: 66'139, data_included: [208635~13631,524288~1034945], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:aca46c33:::rbd_data.11835a3b5a27.0000000000000007:1c@66'139, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 1c=[]:{1c: [1c]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1559233, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:17.732 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.903307+0000", 2024-09-17T07:30:17.733 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.78569503600000001, 2024-09-17T07:30:17.733 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020969980000000001, 2024-09-17T07:30:17.733 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.733 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.733 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.733 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.733 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.903307+0000", 2024-09-17T07:30:17.733 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.733 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.903307+0000", 2024-09-17T07:30:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.903308+0000", 2024-09-17T07:30:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2099999999999996e-07 2024-09-17T07:30:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.734 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.904250+0000", 2024-09-17T07:30:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00094257599999999998 2024-09-17T07:30:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.904267+0000", 2024-09-17T07:30:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6376999999999999e-05 2024-09-17T07:30:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.904277+0000", 2024-09-17T07:30:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0380000000000001e-05 2024-09-17T07:30:17.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.904370+0000", 2024-09-17T07:30:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2887999999999996e-05 2024-09-17T07:30:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.904389+0000", 2024-09-17T07:30:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9250999999999999e-05 2024-09-17T07:30:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.905404+0000", 2024-09-17T07:30:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001014905 2024-09-17T07:30:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:aca46c33:::rbd_data.11835a3b5a27.0000000000000007:1c, version: 66'139, data_included: [1559233~537919], data_size: 537919, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:aca46c33:::rbd_data.11835a3b5a27.0000000000000007:1c@66'139, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 1c=[]:{1c: [1c]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1559233, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.906563+0000", 2024-09-17T07:30:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.78243934299999995, 2024-09-17T07:30:17.737 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0018247649999999999, 2024-09-17T07:30:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.906563+0000", 2024-09-17T07:30:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.738 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.906563+0000", 2024-09-17T07:30:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.906564+0000", 2024-09-17T07:30:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3e-07 2024-09-17T07:30:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.907363+0000", 2024-09-17T07:30:17.739 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00079929800000000004 2024-09-17T07:30:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.907380+0000", 2024-09-17T07:30:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6711e-05 2024-09-17T07:30:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.907390+0000", 2024-09-17T07:30:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0186999999999999e-05 2024-09-17T07:30:17.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.907480+0000", 2024-09-17T07:30:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9920999999999995e-05 2024-09-17T07:30:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.907494+0000", 2024-09-17T07:30:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4262e-05 2024-09-17T07:30:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.908388+0000", 2024-09-17T07:30:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000893756 2024-09-17T07:30:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ab91cfbe:::rbd_data.118394b259f9.0000000000000000:1e, version: 68'159, data_included: [16384~81920,180224~81920,344064~92788,507904~81920,595536~49296,671744~81920,835584~81920,999424~49152,5169889~28787], data_size: 629623, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ab91cfbe:::rbd_data.118394b259f9.0000000000000000:1e@68'159, size: 5242880, copy_subset: [(0, 5242880)], clone_subset: {}, snapset: 1e=[]:{1e: [1e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.917685+0000", 2024-09-17T07:30:17.742 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.77131732600000003, 2024-09-17T07:30:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023928310000000002, 2024-09-17T07:30:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.917685+0000", 2024-09-17T07:30:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.917685+0000", 2024-09-17T07:30:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.917686+0000", 2024-09-17T07:30:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2399999999999998e-07 2024-09-17T07:30:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.918262+0000", 2024-09-17T07:30:17.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00057645199999999998 2024-09-17T07:30:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.918279+0000", 2024-09-17T07:30:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7277000000000001e-05 2024-09-17T07:30:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.918292+0000", 2024-09-17T07:30:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2303000000000001e-05 2024-09-17T07:30:17.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.918382+0000", 2024-09-17T07:30:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0469000000000004e-05 2024-09-17T07:30:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.918399+0000", 2024-09-17T07:30:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6565000000000001e-05 2024-09-17T07:30:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.746 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.920078+0000", 2024-09-17T07:30:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016791410000000001 2024-09-17T07:30:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:a80ac514:::rbd_data.11838d8fb7ca.000000000000000e:head, version: 80'212, data_included: [0~565248,589824~305553,917504~131072], data_size: 1001873, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a80ac514:::rbd_data.11838d8fb7ca.000000000000000e:head@80'212, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.945842+0000", 2024-09-17T07:30:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.74316032899999995, 2024-09-17T07:30:17.747 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020580469999999999, 2024-09-17T07:30:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.945842+0000", 2024-09-17T07:30:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.748 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.945842+0000", 2024-09-17T07:30:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.945842+0000", 2024-09-17T07:30:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:30:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.947034+0000", 2024-09-17T07:30:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011916229999999999 2024-09-17T07:30:17.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.947049+0000", 2024-09-17T07:30:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4999e-05 2024-09-17T07:30:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.947059+0000", 2024-09-17T07:30:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8300000000000008e-06 2024-09-17T07:30:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.947120+0000", 2024-09-17T07:30:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1424e-05 2024-09-17T07:30:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.947128+0000", 2024-09-17T07:30:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1159999999999999e-06 2024-09-17T07:30:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.947900+0000", 2024-09-17T07:30:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000771559 2024-09-17T07:30:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9, version: 36'125, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9@36'125, size: 10246384, copy_subset: [(0, 10246384)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.176147+0000", 2024-09-17T07:30:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.51285518100000005, 2024-09-17T07:30:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022794080000000001, 2024-09-17T07:30:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.752 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.176147+0000", 2024-09-17T07:30:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.176147+0000", 2024-09-17T07:30:17.753 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.176149+0000", 2024-09-17T07:30:17.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4699999999999999e-06 2024-09-17T07:30:17.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.177787+0000", 2024-09-17T07:30:17.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001638371 2024-09-17T07:30:17.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.755 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.755 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.177802+0000", 2024-09-17T07:30:17.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5336e-05 2024-09-17T07:30:17.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.755 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.755 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.177812+0000", 2024-09-17T07:30:17.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8079999999999999e-06 2024-09-17T07:30:17.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.755 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.177878+0000", 2024-09-17T07:30:17.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5926999999999994e-05 2024-09-17T07:30:17.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.177886+0000", 2024-09-17T07:30:17.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2630000000000003e-06 2024-09-17T07:30:17.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.178427+0000", 2024-09-17T07:30:17.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054023299999999999 2024-09-17T07:30:17.757 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.757 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.757 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.757 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9, version: 36'125, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9@36'125, size: 10246384, copy_subset: [(0, 10246384)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:17.757 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.182728+0000", 2024-09-17T07:30:17.757 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.50627409099999998, 2024-09-17T07:30:17.757 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020096250000000001, 2024-09-17T07:30:17.757 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.758 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.758 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.758 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.182728+0000", 2024-09-17T07:30:17.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.758 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.182728+0000", 2024-09-17T07:30:17.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.182729+0000", 2024-09-17T07:30:17.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.15e-07 2024-09-17T07:30:17.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.184065+0000", 2024-09-17T07:30:17.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001336512 2024-09-17T07:30:17.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.184081+0000", 2024-09-17T07:30:17.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5500000000000001e-05 2024-09-17T07:30:17.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.184090+0000", 2024-09-17T07:30:17.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1179999999999998e-06 2024-09-17T07:30:17.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.184171+0000", 2024-09-17T07:30:17.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1258e-05 2024-09-17T07:30:17.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.761 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.184179+0000", 2024-09-17T07:30:17.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2290000000000003e-06 2024-09-17T07:30:17.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.761 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.184738+0000", 2024-09-17T07:30:17.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00055859300000000002 2024-09-17T07:30:17.762 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.762 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.762 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.762 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9, version: 36'125, data_included: [4194304~524288,4854700~26316,4980736~497972], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9@36'125, size: 10246384, copy_subset: [(0, 10246384)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5478708, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:17.762 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.186273+0000", 2024-09-17T07:30:17.762 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.50272950500000002, 2024-09-17T07:30:17.762 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00179915, 2024-09-17T07:30:17.762 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.762 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.762 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.762 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.763 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.186273+0000", 2024-09-17T07:30:17.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.763 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.186273+0000", 2024-09-17T07:30:17.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.763 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.186288+0000", 2024-09-17T07:30:17.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.509e-05 2024-09-17T07:30:17.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.764 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.187166+0000", 2024-09-17T07:30:17.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087794299999999995 2024-09-17T07:30:17.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.764 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.187182+0000", 2024-09-17T07:30:17.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6626999999999999e-05 2024-09-17T07:30:17.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.187193+0000", 2024-09-17T07:30:17.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0847e-05 2024-09-17T07:30:17.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.187280+0000", 2024-09-17T07:30:17.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6679000000000001e-05 2024-09-17T07:30:17.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.187289+0000", 2024-09-17T07:30:17.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7839999999999992e-06 2024-09-17T07:30:17.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.188072+0000", 2024-09-17T07:30:17.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00078317999999999997 2024-09-17T07:30:17.766 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.766 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.766 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.767 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9, version: 36'125, data_included: [5478708~26316,5767168~524288,6553600~497972], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9@36'125, size: 10246384, copy_subset: [(0, 10246384)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7051572, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5478708, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:17.767 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:17.767 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.49959759599999998, 2024-09-17T07:30:17.767 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024847990000000002, 2024-09-17T07:30:17.767 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.767 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.767 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.767 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.767 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.768 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:17.768 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.768 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.768 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:17.768 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.768 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.768 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:17.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:30:17.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191002+0000", 2024-09-17T07:30:17.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015967100000000001 2024-09-17T07:30:17.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191017+0000", 2024-09-17T07:30:17.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4717e-05 2024-09-17T07:30:17.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.770 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191036+0000", 2024-09-17T07:30:17.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9882999999999999e-05 2024-09-17T07:30:17.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.770 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191120+0000", 2024-09-17T07:30:17.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3313999999999995e-05 2024-09-17T07:30:17.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.771 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191128+0000", 2024-09-17T07:30:17.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5499999999999995e-06 2024-09-17T07:30:17.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.771 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191889+0000", 2024-09-17T07:30:17.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076114100000000001 2024-09-17T07:30:17.771 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.771 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.771 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.772 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9, version: 36'125, data_included: [8100148~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9@36'125, size: 10246384, copy_subset: [(0, 10246384)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 9148724, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 8100148, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:17.772 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.196484+0000", 2024-09-17T07:30:17.772 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.49251852699999998, 2024-09-17T07:30:17.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017993060000000001, 2024-09-17T07:30:17.772 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.772 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.772 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.772 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.196484+0000", 2024-09-17T07:30:17.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.196484+0000", 2024-09-17T07:30:17.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.196484+0000", 2024-09-17T07:30:17.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4700000000000001e-07 2024-09-17T07:30:17.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.774 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.197487+0000", 2024-09-17T07:30:17.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001002515 2024-09-17T07:30:17.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.774 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.197506+0000", 2024-09-17T07:30:17.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8785999999999999e-05 2024-09-17T07:30:17.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.197517+0000", 2024-09-17T07:30:17.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1082e-05 2024-09-17T07:30:17.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.197601+0000", 2024-09-17T07:30:17.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3789999999999996e-05 2024-09-17T07:30:17.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.776 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.776 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.197609+0000", 2024-09-17T07:30:17.776 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4940000000000003e-06 2024-09-17T07:30:17.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.777 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.777 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.198283+0000", 2024-09-17T07:30:17.777 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00067399199999999999 2024-09-17T07:30:17.777 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.777 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.777 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.777 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head, version: 36'126, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head@36'126, size: 9431460, copy_subset: [], clone_subset: {3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9: [(0, 9431460)]}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:17.777 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:17.778 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.48521277699999998, 2024-09-17T07:30:17.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002812681, 2024-09-17T07:30:17.778 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.778 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.778 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.778 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.778 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:17.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:17.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:17.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9100000000000004e-07 2024-09-17T07:30:17.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203801+0000", 2024-09-17T07:30:17.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0696000000000001e-05 2024-09-17T07:30:17.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203817+0000", 2024-09-17T07:30:17.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6480000000000001e-05 2024-09-17T07:30:17.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203829+0000", 2024-09-17T07:30:17.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1935e-05 2024-09-17T07:30:17.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203891+0000", 2024-09-17T07:30:17.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2079e-05 2024-09-17T07:30:17.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203900+0000", 2024-09-17T07:30:17.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4999999999999999e-06 2024-09-17T07:30:17.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.782 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.206602+0000", 2024-09-17T07:30:17.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027024000000000002 2024-09-17T07:30:17.782 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.782 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.782 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.782 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a0532310:::rbd_data.1183515b0721.0000000000000004:head, version: 39'137, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a0532310:::rbd_data.1183515b0721.0000000000000004:head@39'137, size: 9437184, copy_subset: [(0, 9437184)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:17.782 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.210339+0000", 2024-09-17T07:30:17.782 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.47866294300000001, 2024-09-17T07:30:17.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001934247, 2024-09-17T07:30:17.783 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.783 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.783 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.783 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.210339+0000", 2024-09-17T07:30:17.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.210339+0000", 2024-09-17T07:30:17.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.784 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.784 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.210340+0000", 2024-09-17T07:30:17.784 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4300000000000001e-07 2024-09-17T07:30:17.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.784 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.784 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.211494+0000", 2024-09-17T07:30:17.784 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001153984 2024-09-17T07:30:17.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.211513+0000", 2024-09-17T07:30:17.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8270999999999998e-05 2024-09-17T07:30:17.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.211531+0000", 2024-09-17T07:30:17.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7977000000000001e-05 2024-09-17T07:30:17.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.211608+0000", 2024-09-17T07:30:17.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7690999999999999e-05 2024-09-17T07:30:17.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.211617+0000", 2024-09-17T07:30:17.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1700000000000003e-06 2024-09-17T07:30:17.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.212274+0000", 2024-09-17T07:30:17.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00065621099999999997 2024-09-17T07:30:17.786 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.787 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.787 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.787 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head, version: 82'426, data_included: [0~786432,981325~41682,1966080~6721], data_size: 834835, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head@82'426, size: 3276800, copy_subset: [(0, 3276800)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3276800, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:17.787 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.308228+0000", 2024-09-17T07:30:17.787 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.38077378899999997, 2024-09-17T07:30:17.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025329110000000001, 2024-09-17T07:30:17.787 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.787 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.787 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:17.788 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.788 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.788 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.308228+0000", 2024-09-17T07:30:17.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.788 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.788 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.308228+0000", 2024-09-17T07:30:17.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.308229+0000", 2024-09-17T07:30:17.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:30:17.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309432+0000", 2024-09-17T07:30:17.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001202704 2024-09-17T07:30:17.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309447+0000", 2024-09-17T07:30:17.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5184000000000001e-05 2024-09-17T07:30:17.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309457+0000", 2024-09-17T07:30:17.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9129999999999998e-06 2024-09-17T07:30:17.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309544+0000", 2024-09-17T07:30:17.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7249000000000004e-05 2024-09-17T07:30:17.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:17.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309558+0000", 2024-09-17T07:30:17.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3781000000000001e-05 2024-09-17T07:30:17.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:17.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.310761+0000", 2024-09-17T07:30:17.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012035839999999999 2024-09-17T07:30:17.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.791 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.792 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.792 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:17.793 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:17.792+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:17.794 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:17.894 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:17.893+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:17.954 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:17.954 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:17.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.954 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGRecoveryDeleteReply(3.b e108,106 3:d785206c:::rbd_data.1183177be714.000000000000011b:head,61'165)", 2024-09-17T07:30:17.954 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.953618+0000", 2024-09-17T07:30:17.955 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.00026106700000000003, 2024-09-17T07:30:17.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027383600000000001, 2024-09-17T07:30:17.955 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:17.955 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:17.955 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:17.955 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:17.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:17.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.953618+0000", 2024-09-17T07:30:17.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:17.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.953618+0000", 2024-09-17T07:30:17.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:17.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.956 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:17.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.953619+0000", 2024-09-17T07:30:17.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3900000000000005e-07 2024-09-17T07:30:17.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.956 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:17.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.953622+0000", 2024-09-17T07:30:17.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.23e-06 2024-09-17T07:30:17.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:17.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.953632+0000", 2024-09-17T07:30:17.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8009999999999998e-06 2024-09-17T07:30:17.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:17.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.953637+0000", 2024-09-17T07:30:17.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3009999999999997e-06 2024-09-17T07:30:17.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:17.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:17.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:17.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.953694+0000", 2024-09-17T07:30:17.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7222000000000002e-05 2024-09-17T07:30:17.958 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.958 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:17.958 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.958 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:17.958 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:30:17.959 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 1 2024-09-17T07:30:17.959 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:17.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:17.995 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:17.995+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:18.085 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:18.085 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:18.085 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:18.085 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:18.085 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:18.096 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:18.095+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:18.097 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:18.197 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:18.196+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:18.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T07:30:18.225 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:18.225 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:18.225 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:18.225 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:18.226 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.226 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.226 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:18.226 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.091361766000006, 2024-09-17T07:30:18.226 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:30:18.226 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.226 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.226 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.226 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.226 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.226 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.226 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:30:18.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.227 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.227 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.227 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:18.227 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.227 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.227 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:18.227 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.227 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.227 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.228 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:30:18.228 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:30:18.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.228 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.228 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:30:18.228 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:30:18.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.228 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.228 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:30:18.228 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:30:18.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.229 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.229 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:30:18.229 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:30:18.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.229 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.229 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:30:18.229 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:30:18.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.229 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:30:18.229 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:30:18.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:30:18.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.230 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.230 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:30:18.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:30:18.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.230 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.230 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:30:18.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:30:18.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:30:18.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:30:18.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:30:18.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:30:18.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:30:18.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:30:18.231 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.232 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.232 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.232 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.232 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:18.232 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.785395062000006, 2024-09-17T07:30:18.232 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:30:18.232 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.232 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.232 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.232 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.232 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.232 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.233 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:30:18.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.233 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.233 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.233 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:18.233 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.233 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.233 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.233 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:18.233 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:30:18.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:30:18.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:30:18.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:30:18.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:30:18.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:30:18.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.235 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:30:18.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:30:18.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.235 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:30:18.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:30:18.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.235 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:30:18.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:30:18.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.236 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:30:18.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:30:18.236 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.236 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.236 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.236 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.236 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:18.237 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.785372946999999, 2024-09-17T07:30:18.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:30:18.237 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.237 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.237 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.237 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.237 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.237 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:30:18.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.237 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.241 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.241 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:18.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:18.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:30:18.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:18.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:30:18.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:30:18.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:30:18.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:30:18.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:30:18.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:30:18.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.244 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:30:18.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:30:18.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.244 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:30:18.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:30:18.244 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.244 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.244 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.245 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.245 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:18.245 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.785348842999994, 2024-09-17T07:30:18.245 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:30:18.245 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.245 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.245 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.245 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.245 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.245 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.245 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:30:18.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.245 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:18.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:18.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:30:18.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:18.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:30:18.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:30:18.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:30:18.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:30:18.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:30:18.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:30:18.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.248 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.248 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:30:18.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:30:18.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.248 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.248 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:30:18.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:30:18.248 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.249 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.249 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.249 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.249 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:18.249 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.785324754000001, 2024-09-17T07:30:18.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:30:18.249 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.249 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.249 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.249 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.249 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.250 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:18.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:30:18.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:30:18.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:30:18.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:30:18.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:30:18.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:30:18.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:30:18.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:30:18.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:30:18.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:30:18.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:30:18.252 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.253 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.253 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.253 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.253 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:18.253 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.785300335000002, 2024-09-17T07:30:18.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:30:18.253 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.253 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.253 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.253 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.253 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.253 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.254 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:30:18.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.254 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:18.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:18.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:18.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:30:18.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:30:18.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:30:18.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.255 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:30:18.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:30:18.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:30:18.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:18.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:30:18.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:30:18.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:30:18.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:30:18.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.257 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.257 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.257 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.257 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:18.257 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.785278634999997, 2024-09-17T07:30:18.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:30:18.257 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.257 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.257 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.257 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.257 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.258 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.258 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:30:18.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.258 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.258 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.258 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:18.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.258 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.258 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:18.258 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:18.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:30:18.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:30:18.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:30:18.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.259 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.259 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:30:18.259 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:30:18.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:30:18.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:30:18.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:30:18.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:30:18.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:30:18.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:30:18.261 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.261 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.261 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.261 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.261 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:18.261 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.785233903000005, 2024-09-17T07:30:18.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:30:18.261 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.261 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:30:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:30:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:30:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:30:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:30:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:30:18.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:30:18.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:30:18.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:30:18.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:30:18.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:30:18.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:30:18.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:30:18.265 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.265 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.265 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.265 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.265 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:18.265 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.785205437000002, 2024-09-17T07:30:18.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:30:18.265 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:30:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:30:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:30:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:30:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:30:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:30:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:30:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:30:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:30:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:30:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:30:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:30:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:18.269 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.785088185000006, 2024-09-17T07:30:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:30:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:30:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:18.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:30:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:30:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:30:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:30:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:30:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:30:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:30:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:30:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:30:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:30:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:30:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:30:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.273 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.784662083000001, 2024-09-17T07:30:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:30:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:30:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:30:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:30:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:30:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:30:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:30:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:30:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:30:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:30:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:30:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:30:18.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:30:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.784642411999997, 2024-09-17T07:30:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:30:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.278 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:30:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:18.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:30:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:30:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:30:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:30:18.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:30:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:30:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:30:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:30:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:30:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:30:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:30:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.784619547999995, 2024-09-17T07:30:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:30:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.282 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:30:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:30:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:30:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:30:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:30:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:30:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:30:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:30:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:30:18.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:30:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:30:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:30:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:18.286 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.784598247000005, 2024-09-17T07:30:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:30:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:30:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:30:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:30:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:30:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:30:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:30:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:30:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:30:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:30:18.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:30:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:30:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:30:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:30:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.784550459000002, 2024-09-17T07:30:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:30:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:30:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:30:18.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:30:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:30:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:30:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:30:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:30:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:30:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:30:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:30:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:30:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:30:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:30:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.294 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.784523237000002, 2024-09-17T07:30:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:30:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.295 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:30:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:30:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:30:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:30:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:30:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:30:18.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:30:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:30:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:30:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:30:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:30:18.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:30:18.299 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.299 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.299 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.299 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.299 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:18.299 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.784502849999996, 2024-09-17T07:30:18.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:30:18.299 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.299 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.299 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.299 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.300 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.300 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.300 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:30:18.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.300 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:18.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:18.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:30:18.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:30:18.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:30:18.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:30:18.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:30:18.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:30:18.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:30:18.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:30:18.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:30:18.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:30:18.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:30:18.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:30:18.303 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.303 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.303 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.303 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.303 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:18.303 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.784483144999996, 2024-09-17T07:30:18.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:30:18.303 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.304 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.304 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.304 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.304 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:30:18.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.304 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:18.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:18.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:30:18.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:30:18.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:30:18.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:30:18.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:30:18.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:30:18.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:30:18.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:30:18.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:30:18.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:30:18.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:30:18.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:30:18.307 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.307 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.307 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.307 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.307 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:18.308 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.784460066999998, 2024-09-17T07:30:18.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:30:18.308 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.308 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.308 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.308 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.308 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.308 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:30:18.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.308 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:18.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:18.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:30:18.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:30:18.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:30:18.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:30:18.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:30:18.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:30:18.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:30:18.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:30:18.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:30:18.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:30:18.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:30:18.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:30:18.311 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.311 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.311 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.312 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:18.312 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:18.312 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.784439504999995, 2024-09-17T07:30:18.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:30:18.312 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.312 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:18.312 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.312 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.312 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.312 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:30:18.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.313 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:18.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:18.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:18.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:30:18.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:30:18.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:30:18.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:30:18.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:30:18.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:30:18.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:30:18.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:30:18.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:30:18.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:30:18.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:30:18.315 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.315 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.316 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.316 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.316 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.316 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:18.318 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:18.318 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:18.316+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:18.418 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:18.418+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:18.465 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:18.466 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:18.466 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:18.466 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:18.475 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:18.519 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:18.518+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:18.620 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:18.619+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:18.622 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:18.622 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:18.622 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:18.622 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:18.622 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:18.630 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:18.691 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:30:18.722 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:18.721+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:18.740 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:18.741 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:18.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:18.741 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:18.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.741 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.741 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:18.741 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605841456999997, 2024-09-17T07:30:18.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:30:18.741 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.741 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.741 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.741 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.742 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.742 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.742 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:30:18.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.742 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.742 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:18.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.742 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:18.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:30:18.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:30:18.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:30:18.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:30:18.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:30:18.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:30:18.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:30:18.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:30:18.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:30:18.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:30:18.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.745 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.745 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:30:18.745 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:30:18.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.745 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.745 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:30:18.745 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:30:18.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.746 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.746 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:30:18.746 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:30:18.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.746 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.746 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:30:18.746 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:30:18.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.747 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.747 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:30:18.747 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:30:18.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.747 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.747 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:30:18.747 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:30:18.747 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605804559999996, 2024-09-17T07:30:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:30:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.748 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:30:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.749 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:30:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:30:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:30:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:30:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:30:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:30:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:30:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:30:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:30:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:30:18.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:30:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:30:18.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:30:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:30:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:30:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:30:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:30:18.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:30:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:30:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:30:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:30:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605778592999997, 2024-09-17T07:30:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:30:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:30:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:30:18.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:30:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:30:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.757 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:30:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:30:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:30:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:30:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:30:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:30:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.758 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:30:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:30:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:30:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:30:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:30:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:30:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:30:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:30:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:30:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:30:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:30:18.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:30:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605753997999997, 2024-09-17T07:30:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:30:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.761 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:30:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:30:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:30:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:30:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:30:18.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:30:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:30:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:30:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:30:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:30:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:30:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:30:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:30:18.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:30:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:30:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:30:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:30:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:30:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.767 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.767 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:30:18.767 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:30:18.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.767 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.767 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:30:18.767 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:30:18.767 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.767 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.767 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.768 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.768 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:18.768 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605730511000004, 2024-09-17T07:30:18.768 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:30:18.768 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.769 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.769 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.769 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.769 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:30:18.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.770 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.770 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:18.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.770 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:18.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:30:18.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:18.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.771 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:30:18.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:30:18.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.771 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:30:18.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:30:18.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.772 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.772 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:30:18.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:30:18.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.772 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.772 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:30:18.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:30:18.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:18.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:30:18.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:30:18.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:30:18.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.774 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:30:18.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:30:18.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.774 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:30:18.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:30:18.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:30:18.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:30:18.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:30:18.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:30:18.775 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.775 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.775 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.776 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.776 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:18.776 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605716772999997, 2024-09-17T07:30:18.776 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:30:18.776 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.776 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.776 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.776 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.776 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.777 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.777 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:30:18.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.777 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.777 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.778 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:18.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.778 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.778 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:18.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.778 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:18.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:30:18.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:30:18.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:30:18.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:30:18.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:30:18.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:30:18.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:30:18.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:18.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:30:18.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:18.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:30:18.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:30:18.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:30:18.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.782 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:30:18.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:30:18.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.782 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:30:18.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:30:18.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.782 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:30:18.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:30:18.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:30:18.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:30:18.783 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.783 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.783 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.784 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.784 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:18.784 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605686583999997, 2024-09-17T07:30:18.784 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:30:18.784 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.784 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.784 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.784 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.785 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.785 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:30:18.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.785 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:18.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:18.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:30:18.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:30:18.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.787 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:30:18.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:30:18.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.788 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.788 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:18.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:30:18.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:30:18.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:30:18.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:30:18.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:30:18.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:30:18.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:30:18.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:30:18.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:30:18.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:30:18.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:18.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:30:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:30:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:30:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:30:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:30:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:30:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.791 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605650079, 2024-09-17T07:30:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:30:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.792 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:30:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:30:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:30:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:30:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:30:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:30:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:30:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:30:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:30:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:30:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:30:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:30:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:30:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:30:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:30:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:30:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:30:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:30:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:30:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:30:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:30:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.797 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:18.798 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605622319999995, 2024-09-17T07:30:18.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:30:18.798 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.798 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:30:18.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.798 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:18.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:18.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:30:18.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:30:18.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:30:18.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:30:18.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:30:18.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:30:18.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:30:18.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:30:18.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:18.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:30:18.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:30:18.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:30:18.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:30:18.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:30:18.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:30:18.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:18.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:30:18.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:30:18.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:30:18.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:30:18.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:30:18.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:30:18.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.803 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.803 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.803 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:18.803 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605600159999995, 2024-09-17T07:30:18.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:30:18.803 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.804 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.804 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.804 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.804 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.804 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:30:18.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.804 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:18.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:18.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:18.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:30:18.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:30:18.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:30:18.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:30:18.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:18.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:30:18.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:30:18.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:30:18.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:30:18.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:30:18.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:30:18.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:30:18.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:30:18.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:30:18.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:30:18.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:30:18.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:30:18.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:30:18.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:30:18.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:30:18.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:30:18.809 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.809 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.809 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.809 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.809 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:18.809 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605586998999996, 2024-09-17T07:30:18.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:30:18.810 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.810 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.810 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.810 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.810 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.810 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.810 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:30:18.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.810 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:18.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:18.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:18.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:30:18.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:30:18.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:30:18.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.812 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:30:18.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:30:18.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.812 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:30:18.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:30:18.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.812 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:30:18.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:30:18.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:18.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:30:18.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:30:18.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:30:18.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:30:18.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:30:18.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:30:18.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:30:18.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:30:18.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:30:18.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:30:18.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:30:18.815 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.815 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.815 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.815 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.815 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:18.815 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605568410999993, 2024-09-17T07:30:18.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:30:18.816 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.816 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.816 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.816 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.816 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.816 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:30:18.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.816 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:18.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:18.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:30:18.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:30:18.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:30:18.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:30:18.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:18.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:30:18.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:18.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:30:18.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:18.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:30:18.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:18.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:30:18.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:30:18.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:30:18.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:30:18.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:30:18.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:30:18.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:30:18.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:30:18.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:30:18.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:30:18.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:30:18.821 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.821 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.821 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.821 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.821 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:18.821 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605535599999996, 2024-09-17T07:30:18.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:30:18.822 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.822 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.822 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.822 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.822 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:30:18.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:30:18.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:30:18.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:30:18.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:30:18.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:30:18.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:30:18.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:30:18.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:30:18.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:30:18.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:18.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:18.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:30:18.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:30:18.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:30:18.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:18.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:30:18.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:30:18.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:30:18.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:30:18.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:30:18.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:30:18.827 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.827 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.827 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.827 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.827 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:18.827 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605516542999993, 2024-09-17T07:30:18.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:30:18.828 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.828 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.828 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.828 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.828 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.828 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:30:18.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.828 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:18.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:18.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:30:18.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:30:18.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:30:18.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:30:18.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:30:18.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:30:18.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:30:18.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:30:18.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:30:18.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:30:18.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:30:18.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:30:18.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:30:18.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:30:18.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:30:18.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:30:18.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:30:18.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:30:18.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:30:18.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:30:18.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:30:18.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:30:18.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.833 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.833 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.833 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.833 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:18.833 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605494433999993, 2024-09-17T07:30:18.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:30:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:30:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:18.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:30:18.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:30:18.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:18.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:30:18.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:30:18.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:30:18.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:30:18.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:18.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:30:18.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:30:18.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:30:18.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:30:18.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:30:18.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:30:18.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:30:18.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:30:18.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:30:18.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:30:18.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:30:18.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:30:18.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:30:18.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:30:18.839 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.839 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.839 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.839 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.839 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:18.839 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605468326999997, 2024-09-17T07:30:18.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:30:18.839 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.840 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.840 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.840 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.840 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:30:18.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.840 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:18.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:18.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:30:18.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:30:18.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:18.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:30:18.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:30:18.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:18.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:30:18.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:30:18.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:30:18.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:30:18.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:30:18.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:30:18.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:30:18.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:30:18.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:30:18.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:30:18.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:30:18.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:30:18.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:30:18.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:30:18.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:30:18.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:30:18.845 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.845 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.845 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.845 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.845 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:18.845 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605439181999998, 2024-09-17T07:30:18.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:30:18.845 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.845 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.846 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.846 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.846 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.846 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.846 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:30:18.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.846 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:18.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:18.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:18.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:30:18.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:30:18.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:18.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:30:18.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:30:18.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:30:18.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:30:18.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:30:18.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:30:18.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:30:18.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:30:18.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:30:18.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:30:18.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:30:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:30:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:30:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:30:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:30:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:30:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:30:18.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:30:18.851 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.851 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.851 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.851 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.851 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:18.851 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605407378999999, 2024-09-17T07:30:18.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:30:18.851 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.852 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.852 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.852 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.852 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.852 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:30:18.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.852 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:18.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:18.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:30:18.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:30:18.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:18.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:30:18.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:30:18.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:30:18.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:30:18.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:30:18.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:30:18.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:30:18.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:30:18.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:30:18.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:30:18.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:30:18.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:30:18.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:30:18.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:30:18.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:30:18.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:30:18.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:30:18.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:30:18.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:30:18.857 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.857 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.857 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.857 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.857 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:18.858 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605380432000004, 2024-09-17T07:30:18.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:30:18.858 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.858 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.858 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.858 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.858 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:30:18.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.858 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:18.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:18.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:30:18.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:30:18.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:30:18.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:30:18.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:18.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:30:18.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:18.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:30:18.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:30:18.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:30:18.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:30:18.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:30:18.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:30:18.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:30:18.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:30:18.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:30:18.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:30:18.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:30:18.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:30:18.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:30:18.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:30:18.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:30:18.863 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.863 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.863 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.863 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:18.863 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:18.863 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.605339482999995, 2024-09-17T07:30:18.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:30:18.864 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:18.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:18.864 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:18.864 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:18.864 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:18.864 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:18.864 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:30:18.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.864 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:18.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:18.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:18.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:18.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:18.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:18.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:18.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:30:18.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:30:18.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:18.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:30:18.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:30:18.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:18.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:30:18.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:30:18.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:18.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:30:18.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:30:18.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:30:18.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:30:18.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:18.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:30:18.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:30:18.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:30:18.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:30:18.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:30:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:30:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:30:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:30:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:30:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:30:18.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:30:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:30:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:18.869 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:18.869 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:18.871 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:18.871 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:18.871 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:18.872 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:18.872 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:18.872 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:18.872 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:18.873 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:18.873 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:18.873 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:18.870+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:18.883 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:18.883 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:18.900 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:18.900 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:18.917 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:18.918 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:18.934 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:18.935 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:18.971 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:18.969+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:18.992 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:18.992 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:19.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:19.072 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:19.070+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:19.083 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-17T07:30:19.122 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:19.123 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:19.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:19.173 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:19.172+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:19.218 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-17T07:30:19.218 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:30:19.218 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-17T07:30:19.218 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-09-17T07:30:19.274 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:19.273+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:19.282 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:19.282 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:19.292 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:19.376 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:19.374+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:19.401 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:19.401 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:19.409 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:19.493 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:19.492+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:19.527 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-17T07:30:19.527+0000 7f6d53957640 -1 mon.a@0(leader).osd e110 definitely_dead 0 2024-09-17T07:30:19.552 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:19.552 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:19.561 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:19.602 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:19.601+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:19.678 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:19.676+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:19.704 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:19.704 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:19.713 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:19.778 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:19.778+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:19.856 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:19.856 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:19.865 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:19.881 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:19.881+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:19.966 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:19.966 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:19.974 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:19.983 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:19.982+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:20.076 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:20.076 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:20.086 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:20.086 INFO:teuthology.orchestra.run.smithi045.stderr:marked out osd.4. 2024-09-17T07:30:20.088 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:20.087+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:20.187 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:20.186+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:20.191 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:20.192 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:20.200 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:20.290 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:20.288+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:20.303 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:20.303 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:20.312 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:20.390 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:20.390+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:20.455 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:20.455 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:20.463 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:20.506 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:20.506+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:20.565 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:20.565 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:20.574 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:20.592 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:20.592+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:20.693 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:20.693+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:20.717 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:20.717 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:20.725 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:20.794 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:20.793+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:20.827 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:20.828 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:20.836 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:20.895 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:20.895+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:20.946 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:30:20.956 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:20.957 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:20.966 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:21.008 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:21.007+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:21.074 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:21.074 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:21.095 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:21.095 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:21.096 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:21.096+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:21.112 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T07:30:21.113 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:21.113 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:21.120 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:21.120 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:21.128 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:21.128 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:21.129 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:21.145 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:21.146 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:21.163 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:21.163 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:21.181 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:21.181 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:21.197 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:21.197+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:21.198 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:21.198 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:21.260 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:21.260 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6, version: 32'24, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6@32'24, size: 8388608, copy_subset: [(0, 8388608)], clone_subset: {}, snapset: 6=[]:{6: [6]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.798443+0000", 2024-09-17T07:30:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.4601384060000004, 2024-09-17T07:30:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020776530000000001, 2024-09-17T07:30:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.798443+0000", 2024-09-17T07:30:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.798443+0000", 2024-09-17T07:30:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.798444+0000", 2024-09-17T07:30:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4099999999999998e-07 2024-09-17T07:30:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.799902+0000", 2024-09-17T07:30:21.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014576979999999999 2024-09-17T07:30:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.799919+0000", 2024-09-17T07:30:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7007000000000002e-05 2024-09-17T07:30:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.799929+0000", 2024-09-17T07:30:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0276e-05 2024-09-17T07:30:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.799996+0000", 2024-09-17T07:30:21.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6957000000000003e-05 2024-09-17T07:30:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.800004+0000", 2024-09-17T07:30:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4460000000000001e-06 2024-09-17T07:30:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.800521+0000", 2024-09-17T07:30:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00051662799999999999 2024-09-17T07:30:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6, version: 32'24, data_included: [3276800~196608,3538944~393216,3934282~52307,3997696~196608,4227072~131072,4456448~78765], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6@32'24, size: 8388608, copy_subset: [(0, 8388608)], clone_subset: {}, snapset: 6=[]:{6: [6]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4535213, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3276800, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.804994+0000", 2024-09-17T07:30:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.4535873920000002, 2024-09-17T07:30:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001902064, 2024-09-17T07:30:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.804994+0000", 2024-09-17T07:30:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.804994+0000", 2024-09-17T07:30:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.804995+0000", 2024-09-17T07:30:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5000000000000003e-07 2024-09-17T07:30:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.805957+0000", 2024-09-17T07:30:21.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00096245300000000005 2024-09-17T07:30:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.805973+0000", 2024-09-17T07:30:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5815999999999999e-05 2024-09-17T07:30:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.805984+0000", 2024-09-17T07:30:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0896999999999999e-05 2024-09-17T07:30:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.806056+0000", 2024-09-17T07:30:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1854999999999996e-05 2024-09-17T07:30:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.806065+0000", 2024-09-17T07:30:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3840000000000003e-06 2024-09-17T07:30:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.806897+0000", 2024-09-17T07:30:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00083110900000000001 2024-09-17T07:30:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.268 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6, version: 32'24, data_included: [4535213~52307,4685824~131072,4848394~42592,4915200~131072,5144576~131072,5373952~131072,5603328~131072,5767168~298317], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ad869e43:::rbd_data.118312a5715.0000000000000002:6@32'24, size: 8388608, copy_subset: [(0, 8388608)], clone_subset: {}, snapset: 6=[]:{6: [6]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 6065485, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4535213, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.808547+0000", 2024-09-17T07:30:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.4500344690000002, 2024-09-17T07:30:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019945240000000001, 2024-09-17T07:30:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.808547+0000", 2024-09-17T07:30:21.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.808547+0000", 2024-09-17T07:30:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.808549+0000", 2024-09-17T07:30:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6339999999999999e-06 2024-09-17T07:30:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.809490+0000", 2024-09-17T07:30:21.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00094110299999999999 2024-09-17T07:30:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.809508+0000", 2024-09-17T07:30:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7382000000000001e-05 2024-09-17T07:30:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.809520+0000", 2024-09-17T07:30:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2503999999999999e-05 2024-09-17T07:30:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.809592+0000", 2024-09-17T07:30:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1649000000000005e-05 2024-09-17T07:30:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.809601+0000", 2024-09-17T07:30:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0420000000000008e-06 2024-09-17T07:30:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.810542+0000", 2024-09-17T07:30:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00094121000000000005 2024-09-17T07:30:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.272 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ab68203a:::rbd_data.1183515b0721.000000000000000b:head, version: 39'59, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:ab68203a:::rbd_data.1183515b0721.000000000000000b:head@39'59, size: 9257781, copy_subset: [(0, 9257781)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.844924+0000", 2024-09-17T07:30:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.4136579759999996, 2024-09-17T07:30:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019831010000000001, 2024-09-17T07:30:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.844924+0000", 2024-09-17T07:30:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.844924+0000", 2024-09-17T07:30:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.844924+0000", 2024-09-17T07:30:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7899999999999998e-07 2024-09-17T07:30:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.274 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.845887+0000", 2024-09-17T07:30:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00096215099999999996 2024-09-17T07:30:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.845902+0000", 2024-09-17T07:30:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5466000000000001e-05 2024-09-17T07:30:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.845913+0000", 2024-09-17T07:30:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0873000000000001e-05 2024-09-17T07:30:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.845999+0000", 2024-09-17T07:30:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6205000000000006e-05 2024-09-17T07:30:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.846015+0000", 2024-09-17T07:30:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5381000000000001e-05 2024-09-17T07:30:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.846907+0000", 2024-09-17T07:30:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089244599999999997 2024-09-17T07:30:21.276 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ab68203a:::rbd_data.1183515b0721.000000000000000b:head, version: 39'59, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:ab68203a:::rbd_data.1183515b0721.000000000000000b:head@39'59, size: 9257781, copy_subset: [(0, 9257781)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.848302+0000", 2024-09-17T07:30:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.410279547, 2024-09-17T07:30:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001905503, 2024-09-17T07:30:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.848302+0000", 2024-09-17T07:30:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.848302+0000", 2024-09-17T07:30:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.848303+0000", 2024-09-17T07:30:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5300000000000004e-07 2024-09-17T07:30:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.849231+0000", 2024-09-17T07:30:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000928344 2024-09-17T07:30:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.849248+0000", 2024-09-17T07:30:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.632e-05 2024-09-17T07:30:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.849258+0000", 2024-09-17T07:30:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0342e-05 2024-09-17T07:30:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.849346+0000", 2024-09-17T07:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7767000000000002e-05 2024-09-17T07:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.849362+0000", 2024-09-17T07:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5772000000000001e-05 2024-09-17T07:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.850208+0000", 2024-09-17T07:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000846305 2024-09-17T07:30:21.280 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ae97839c:::rbd_data.11839a97b753.0000000000000004:head, version: 55'111, data_included: [0~16384,81920~16384,163840~16384,196608~16384,245760~16384,278528~16384,360448~16384,442368~16384,524288~16384,704512~16384,786432~16384,868352~16384,950272~16384,983040~16384,1032192~16384,1064960~16384,1146880~16384,1228800~16384,1310720~16384,1343488~16384,1392640~16384,1425408~32768,1523712~16384], data_size: 393216, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:ae97839c:::rbd_data.11839a97b753.0000000000000004:head@55'111, size: 1540096, copy_subset: [(0, 1540096)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1540096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.884937+0000", 2024-09-17T07:30:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.3736446860000004, 2024-09-17T07:30:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025497419999999998, 2024-09-17T07:30:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.884937+0000", 2024-09-17T07:30:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.884937+0000", 2024-09-17T07:30:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.884938+0000", 2024-09-17T07:30:21.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5499999999999998e-07 2024-09-17T07:30:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885283+0000", 2024-09-17T07:30:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00034585300000000002 2024-09-17T07:30:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885301+0000", 2024-09-17T07:30:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7207999999999999e-05 2024-09-17T07:30:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885316+0000", 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4873e-05 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885416+0000", 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000100456 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885432+0000", 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6390000000000001e-05 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.887487+0000", 2024-09-17T07:30:21.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020545070000000001 2024-09-17T07:30:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:aca46c33:::rbd_data.11835a3b5a27.0000000000000007:1c, version: 66'139, data_included: [208635~13631,524288~1034945], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:aca46c33:::rbd_data.11835a3b5a27.0000000000000007:1c@66'139, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 1c=[]:{1c: [1c]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1559233, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.903307+0000", 2024-09-17T07:30:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.3552746329999996, 2024-09-17T07:30:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020969980000000001, 2024-09-17T07:30:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.285 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.903307+0000", 2024-09-17T07:30:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.903307+0000", 2024-09-17T07:30:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.903308+0000", 2024-09-17T07:30:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2099999999999996e-07 2024-09-17T07:30:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.904250+0000", 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00094257599999999998 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.904267+0000", 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6376999999999999e-05 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.904277+0000", 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0380000000000001e-05 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.904370+0000", 2024-09-17T07:30:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2887999999999996e-05 2024-09-17T07:30:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.904389+0000", 2024-09-17T07:30:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9250999999999999e-05 2024-09-17T07:30:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.905404+0000", 2024-09-17T07:30:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001014905 2024-09-17T07:30:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.288 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ab91cfbe:::rbd_data.118394b259f9.0000000000000000:1e, version: 68'159, data_included: [16384~81920,180224~81920,344064~92788,507904~81920,595536~49296,671744~81920,835584~81920,999424~49152,5169889~28787], data_size: 629623, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ab91cfbe:::rbd_data.118394b259f9.0000000000000000:1e@68'159, size: 5242880, copy_subset: [(0, 5242880)], clone_subset: {}, snapset: 1e=[]:{1e: [1e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.917685+0000", 2024-09-17T07:30:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.3408969229999999, 2024-09-17T07:30:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023928310000000002, 2024-09-17T07:30:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.917685+0000", 2024-09-17T07:30:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.917685+0000", 2024-09-17T07:30:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.917686+0000", 2024-09-17T07:30:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2399999999999998e-07 2024-09-17T07:30:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.918262+0000", 2024-09-17T07:30:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00057645199999999998 2024-09-17T07:30:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.918279+0000", 2024-09-17T07:30:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7277000000000001e-05 2024-09-17T07:30:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.918292+0000", 2024-09-17T07:30:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2303000000000001e-05 2024-09-17T07:30:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.918382+0000", 2024-09-17T07:30:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0469000000000004e-05 2024-09-17T07:30:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.918399+0000", 2024-09-17T07:30:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6565000000000001e-05 2024-09-17T07:30:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.920078+0000", 2024-09-17T07:30:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016791410000000001 2024-09-17T07:30:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:a80ac514:::rbd_data.11838d8fb7ca.000000000000000e:head, version: 80'212, data_included: [0~565248,589824~305553,917504~131072], data_size: 1001873, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a80ac514:::rbd_data.11838d8fb7ca.000000000000000e:head@80'212, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.945842+0000", 2024-09-17T07:30:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.3127399259999999, 2024-09-17T07:30:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020580469999999999, 2024-09-17T07:30:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.945842+0000", 2024-09-17T07:30:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.945842+0000", 2024-09-17T07:30:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.945842+0000", 2024-09-17T07:30:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:30:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.947034+0000", 2024-09-17T07:30:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011916229999999999 2024-09-17T07:30:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.947049+0000", 2024-09-17T07:30:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4999e-05 2024-09-17T07:30:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.947059+0000", 2024-09-17T07:30:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8300000000000008e-06 2024-09-17T07:30:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.947120+0000", 2024-09-17T07:30:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1424e-05 2024-09-17T07:30:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.947128+0000", 2024-09-17T07:30:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1159999999999999e-06 2024-09-17T07:30:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.947900+0000", 2024-09-17T07:30:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000771559 2024-09-17T07:30:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9, version: 36'125, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9@36'125, size: 10246384, copy_subset: [(0, 10246384)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.176147+0000", 2024-09-17T07:30:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.0824347779999997, 2024-09-17T07:30:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022794080000000001, 2024-09-17T07:30:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.176147+0000", 2024-09-17T07:30:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.176147+0000", 2024-09-17T07:30:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.176149+0000", 2024-09-17T07:30:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4699999999999999e-06 2024-09-17T07:30:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.177787+0000", 2024-09-17T07:30:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001638371 2024-09-17T07:30:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.177802+0000", 2024-09-17T07:30:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5336e-05 2024-09-17T07:30:21.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.177812+0000", 2024-09-17T07:30:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8079999999999999e-06 2024-09-17T07:30:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.177878+0000", 2024-09-17T07:30:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5926999999999994e-05 2024-09-17T07:30:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.177886+0000", 2024-09-17T07:30:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2630000000000003e-06 2024-09-17T07:30:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.178427+0000", 2024-09-17T07:30:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054023299999999999 2024-09-17T07:30:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9, version: 36'125, data_included: [3145728~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9@36'125, size: 10246384, copy_subset: [(0, 10246384)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 4194304, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.182728+0000", 2024-09-17T07:30:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.0758536879999996, 2024-09-17T07:30:21.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020096250000000001, 2024-09-17T07:30:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.182728+0000", 2024-09-17T07:30:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.182728+0000", 2024-09-17T07:30:21.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.182729+0000", 2024-09-17T07:30:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.15e-07 2024-09-17T07:30:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.184065+0000", 2024-09-17T07:30:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001336512 2024-09-17T07:30:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.184081+0000", 2024-09-17T07:30:21.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5500000000000001e-05 2024-09-17T07:30:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.184090+0000", 2024-09-17T07:30:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1179999999999998e-06 2024-09-17T07:30:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.184171+0000", 2024-09-17T07:30:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1258e-05 2024-09-17T07:30:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.184179+0000", 2024-09-17T07:30:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2290000000000003e-06 2024-09-17T07:30:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.184738+0000", 2024-09-17T07:30:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00055859300000000002 2024-09-17T07:30:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9, version: 36'125, data_included: [5478708~26316,5767168~524288,6553600~497972], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9@36'125, size: 10246384, copy_subset: [(0, 10246384)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7051572, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5478708, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:21.305 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.0691771929999998, 2024-09-17T07:30:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024847990000000002, 2024-09-17T07:30:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:21.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:30:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191002+0000", 2024-09-17T07:30:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015967100000000001 2024-09-17T07:30:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191017+0000", 2024-09-17T07:30:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4717e-05 2024-09-17T07:30:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191036+0000", 2024-09-17T07:30:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9882999999999999e-05 2024-09-17T07:30:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191120+0000", 2024-09-17T07:30:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3313999999999995e-05 2024-09-17T07:30:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191128+0000", 2024-09-17T07:30:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5499999999999995e-06 2024-09-17T07:30:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191889+0000", 2024-09-17T07:30:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076114100000000001 2024-09-17T07:30:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head, version: 36'126, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head@36'126, size: 9431460, copy_subset: [], clone_subset: {3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9: [(0, 9431460)]}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:21.309 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.0547923739999998, 2024-09-17T07:30:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002812681, 2024-09-17T07:30:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9100000000000004e-07 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203801+0000", 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0696000000000001e-05 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203817+0000", 2024-09-17T07:30:21.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6480000000000001e-05 2024-09-17T07:30:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203829+0000", 2024-09-17T07:30:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1935e-05 2024-09-17T07:30:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203891+0000", 2024-09-17T07:30:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2079e-05 2024-09-17T07:30:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203900+0000", 2024-09-17T07:30:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4999999999999999e-06 2024-09-17T07:30:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.206602+0000", 2024-09-17T07:30:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027024000000000002 2024-09-17T07:30:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a0532310:::rbd_data.1183515b0721.0000000000000004:head, version: 39'137, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a0532310:::rbd_data.1183515b0721.0000000000000004:head@39'137, size: 9437184, copy_subset: [(0, 9437184)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.210339+0000", 2024-09-17T07:30:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.0482425400000004, 2024-09-17T07:30:21.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001934247, 2024-09-17T07:30:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.210339+0000", 2024-09-17T07:30:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.210339+0000", 2024-09-17T07:30:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.210340+0000", 2024-09-17T07:30:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4300000000000001e-07 2024-09-17T07:30:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.211494+0000", 2024-09-17T07:30:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001153984 2024-09-17T07:30:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.211513+0000", 2024-09-17T07:30:21.315 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8270999999999998e-05 2024-09-17T07:30:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.211531+0000", 2024-09-17T07:30:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7977000000000001e-05 2024-09-17T07:30:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.211608+0000", 2024-09-17T07:30:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7690999999999999e-05 2024-09-17T07:30:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.211617+0000", 2024-09-17T07:30:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1700000000000003e-06 2024-09-17T07:30:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.212274+0000", 2024-09-17T07:30:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00065621099999999997 2024-09-17T07:30:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.317 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head, version: 82'426, data_included: [0~786432,981325~41682,1966080~6721], data_size: 834835, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head@82'426, size: 3276800, copy_subset: [(0, 3276800)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3276800, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.308228+0000", 2024-09-17T07:30:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.9503533859999997, 2024-09-17T07:30:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025329110000000001, 2024-09-17T07:30:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.308228+0000", 2024-09-17T07:30:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.308228+0000", 2024-09-17T07:30:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.308229+0000", 2024-09-17T07:30:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:30:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309432+0000", 2024-09-17T07:30:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001202704 2024-09-17T07:30:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309447+0000", 2024-09-17T07:30:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5184000000000001e-05 2024-09-17T07:30:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309457+0000", 2024-09-17T07:30:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9129999999999998e-06 2024-09-17T07:30:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309544+0000", 2024-09-17T07:30:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7249000000000004e-05 2024-09-17T07:30:21.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309558+0000", 2024-09-17T07:30:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3781000000000001e-05 2024-09-17T07:30:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.310761+0000", 2024-09-17T07:30:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012035839999999999 2024-09-17T07:30:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.c 108/106 [PushOp(3:3034d26c:::rbd_data.11835a3b5a27.0000000000000039:1c, version: 66'255, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:3034d26c:::rbd_data.11835a3b5a27.0000000000000039:1c@66'255, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 1c=[]:{1c: [1c]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.781537+0000", 2024-09-17T07:30:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.477044389, 2024-09-17T07:30:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0026008479999999998, 2024-09-17T07:30:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.781537+0000", 2024-09-17T07:30:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.781537+0000", 2024-09-17T07:30:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.781538+0000", 2024-09-17T07:30:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0399999999999996e-07 2024-09-17T07:30:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.783137+0000", 2024-09-17T07:30:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015992739999999999 2024-09-17T07:30:21.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.783153+0000", 2024-09-17T07:30:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5608e-05 2024-09-17T07:30:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.783163+0000", 2024-09-17T07:30:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0028e-05 2024-09-17T07:30:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.783247+0000", 2024-09-17T07:30:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4099999999999998e-05 2024-09-17T07:30:21.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.783261+0000", 2024-09-17T07:30:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4191e-05 2024-09-17T07:30:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.784138+0000", 2024-09-17T07:30:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087714300000000004 2024-09-17T07:30:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.c 108/106 [PushOp(3:3034d26c:::rbd_data.11835a3b5a27.0000000000000039:1c, version: 66'255, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:3034d26c:::rbd_data.11835a3b5a27.0000000000000039:1c@66'255, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 1c=[]:{1c: [1c]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.785844+0000", 2024-09-17T07:30:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.4727378560000002, 2024-09-17T07:30:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0018265950000000001, 2024-09-17T07:30:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.785844+0000", 2024-09-17T07:30:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.785844+0000", 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.785846+0000", 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6330000000000001e-06 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.786776+0000", 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00092999900000000004 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.786792+0000", 2024-09-17T07:30:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6735000000000002e-05 2024-09-17T07:30:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.786803+0000", 2024-09-17T07:30:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.031e-05 2024-09-17T07:30:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.786867+0000", 2024-09-17T07:30:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4740999999999996e-05 2024-09-17T07:30:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.786876+0000", 2024-09-17T07:30:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3930000000000008e-06 2024-09-17T07:30:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.787671+0000", 2024-09-17T07:30:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00079478399999999999 2024-09-17T07:30:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.c 108/106 [PushOp(3:34edcbef:::rbd_data.11831719e038.0000000000000000:head, version: 68'273, data_included: [16384~81920,180224~81920,344064~92788,507904~81920,595536~49296,671744~81920,835584~81920,999424~81920,2605548~8985,2708290~8096,2995581~42933], data_size: 693618, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:34edcbef:::rbd_data.11831719e038.0000000000000000:head@68'273, size: 3038514, copy_subset: [(0, 3038514)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3038514, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:21.332 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.799290+0000", 2024-09-17T07:30:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.4592920249999999, 2024-09-17T07:30:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025209730000000001, 2024-09-17T07:30:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799290+0000", 2024-09-17T07:30:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799290+0000", 2024-09-17T07:30:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799290+0000", 2024-09-17T07:30:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:30:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799886+0000", 2024-09-17T07:30:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00059524899999999995 2024-09-17T07:30:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799903+0000", 2024-09-17T07:30:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7689999999999998e-05 2024-09-17T07:30:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799917+0000", 2024-09-17T07:30:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3189e-05 2024-09-17T07:30:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.800010+0000", 2024-09-17T07:30:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3690999999999995e-05 2024-09-17T07:30:21.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.800025+0000", 2024-09-17T07:30:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4453999999999999e-05 2024-09-17T07:30:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.801811+0000", 2024-09-17T07:30:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001786096 2024-09-17T07:30:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.336 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.c 108/106 [PushOp(3:36a642b6:::rbd_data.1183cb746bb8.0000000000000000:head, version: 72'340, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:36a642b6:::rbd_data.1183cb746bb8.0000000000000000:head@72'340, size: 1548288, copy_subset: [(0, 1548288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.811238+0000", 2024-09-17T07:30:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.4473440769999999, 2024-09-17T07:30:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019968210000000002, 2024-09-17T07:30:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.811238+0000", 2024-09-17T07:30:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.811238+0000", 2024-09-17T07:30:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.811238+0000", 2024-09-17T07:30:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:30:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.812184+0000", 2024-09-17T07:30:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00094582300000000004 2024-09-17T07:30:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.812200+0000", 2024-09-17T07:30:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5610000000000001e-05 2024-09-17T07:30:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.812210+0000", 2024-09-17T07:30:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0669999999999999e-05 2024-09-17T07:30:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.812296+0000", 2024-09-17T07:30:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5191000000000005e-05 2024-09-17T07:30:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.339 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.812310+0000", 2024-09-17T07:30:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4364e-05 2024-09-17T07:30:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.813235+0000", 2024-09-17T07:30:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000924667 2024-09-17T07:30:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.340 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.18 109/106 [PushOp(3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e, version: 44'134, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e@44'134, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:18.083461+0000", 2024-09-17T07:30:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.1751208989999999, 2024-09-17T07:30:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003057976, 2024-09-17T07:30:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083461+0000", 2024-09-17T07:30:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.341 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083461+0000", 2024-09-17T07:30:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083462+0000", 2024-09-17T07:30:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1500000000000004e-07 2024-09-17T07:30:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083603+0000", 2024-09-17T07:30:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140863 2024-09-17T07:30:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.342 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083623+0000", 2024-09-17T07:30:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.003e-05 2024-09-17T07:30:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083642+0000", 2024-09-17T07:30:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9734000000000001e-05 2024-09-17T07:30:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.084564+0000", 2024-09-17T07:30:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00092196200000000002 2024-09-17T07:30:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.343 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.084580+0000", 2024-09-17T07:30:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5464e-05 2024-09-17T07:30:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.086519+0000", 2024-09-17T07:30:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001939208 2024-09-17T07:30:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.344 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.344 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:21.346 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:21.347 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:21.345+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:21.447 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:21.447+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:21.454 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:21.454 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:21.454 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:21.454 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:21.463 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:21.548 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:21.548+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:21.610 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:21.610 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:21.610 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:21.610 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:21.610 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:21.619 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:21.663 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:21.663+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:21.728 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:21.728 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:21.728 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:21.728 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 83.594295747000004, 2024-09-17T07:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.729 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:21.730 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:30:21.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.733 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:30:21.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.735 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.288329043000005, 2024-09-17T07:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.736 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:30:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.737 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:30:21.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:30:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:30:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:30:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:30:21.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.740 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.740 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:30:21.740 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:30:21.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:30:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:30:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:30:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:30:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.741 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.288306927999997, 2024-09-17T07:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:30:21.742 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:21.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:30:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:30:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:30:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.745 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:30:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:30:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:30:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:30:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.746 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.747 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.288282824000007, 2024-09-17T07:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.748 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.750 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:30:21.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.752 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.288258734999999, 2024-09-17T07:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.753 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:30:21.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:30:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:30:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:30:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:30:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:30:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:30:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:21.757 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.288234316, 2024-09-17T07:30:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:30:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:30:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.758 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:21.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:30:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:30:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:30:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:30:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:30:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:30:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:30:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:30:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:30:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:30:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:21.762 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.288212615999996, 2024-09-17T07:30:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:30:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:30:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:30:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:30:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:30:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:30:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:30:21.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.767 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.288167884000003, 2024-09-17T07:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.768 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:30:21.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:30:21.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.288139418, 2024-09-17T07:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.772 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:21.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:30:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:30:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:30:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:30:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:30:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:30:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:30:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:30:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.776 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:30:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:30:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.288022166000005, 2024-09-17T07:30:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:30:21.777 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:30:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:30:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:30:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:30:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:30:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:30:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:30:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:30:21.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:30:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:30:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:30:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:30:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:30:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.781 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.287596063999999, 2024-09-17T07:30:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:30:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.782 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:30:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:30:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:30:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:30:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:30:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:30:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:30:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:30:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:30:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:30:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:30:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:30:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.786 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.287576392999995, 2024-09-17T07:30:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:30:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:30:21.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.788 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:30:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:30:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:30:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:30:21.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:30:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:30:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:30:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:30:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:30:21.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:30:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:30:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.791 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.287553528999993, 2024-09-17T07:30:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:30:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:30:21.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:30:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:30:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:30:21.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:30:21.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:30:21.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:30:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:30:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:30:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:30:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:30:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:30:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.287532228000003, 2024-09-17T07:30:21.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:30:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:30:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:21.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:30:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:30:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:30:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:30:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:30:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:30:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:30:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:30:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:30:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:30:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:30:21.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:30:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.28748444, 2024-09-17T07:30:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:30:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.802 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:30:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:30:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:30:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:30:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:30:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:30:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:30:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:30:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:30:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:30:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:30:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:30:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:30:21.806 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.287457218, 2024-09-17T07:30:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:30:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.807 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:30:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:30:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:30:21.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:30:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:30:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:30:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:30:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:30:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:30:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:30:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:30:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:30:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.287436830999994, 2024-09-17T07:30:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:30:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:30:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:30:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:30:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:30:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:30:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:30:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:30:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:30:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:30:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:30:21.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:30:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:30:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:30:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.816 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.287417125999994, 2024-09-17T07:30:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:30:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:30:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:30:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:30:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:30:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:30:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:30:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:30:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:30:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:30:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:30:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:30:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:30:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:30:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.287394047999996, 2024-09-17T07:30:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:30:21.821 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:30:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:21.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:30:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:30:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:30:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:30:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:30:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:30:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:30:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:30:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:30:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:30:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:30:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:30:21.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.287373486000007, 2024-09-17T07:30:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:30:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:21.826 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:30:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:30:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:21.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:30:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:30:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:30:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:30:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:30:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:30:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:30:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:30:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:30:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:30:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.830 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:21.831 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:21.831 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:21.832 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:21.833 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:21.831+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:21.936 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:21.932+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:21.982 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:21.982 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:21.982 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:21.982 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:21.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:22.034 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:22.033+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:22.084 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-17T07:30:22.134 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:22.134+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:22.172 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:22.173 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:22.173 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:22.173 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:22.173 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:22.186 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:22.235 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:22.235+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:22.336 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:22.335+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:22.366 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231605389999999, 2024-09-17T07:30:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:30:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:30:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:30:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:30:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:30:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:30:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:30:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:30:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:30:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:30:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:30:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:30:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:30:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:30:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:30:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:30:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:30:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:30:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:30:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:30:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:30:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:30:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:30:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:30:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231568492999997, 2024-09-17T07:30:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:30:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:30:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:30:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:30:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:30:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:30:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:30:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:30:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:30:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:30:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:30:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:30:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:30:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:30:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:30:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:30:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:30:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231542525999998, 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:30:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:30:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:30:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:30:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:30:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:30:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:30:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:30:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:30:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:30:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:30:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:30:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:30:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:30:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:30:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:30:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:30:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:30:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:30:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:30:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:30:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231517930999999, 2024-09-17T07:30:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:30:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:30:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:30:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:30:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:30:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:30:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:30:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:30:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:30:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:30:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:30:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:30:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:30:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:30:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:30:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:30:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:30:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:30:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:30:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:30:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:30:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:30:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:30:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231494444000006, 2024-09-17T07:30:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:30:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:30:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:30:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:30:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:30:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:30:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:30:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:30:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:30:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:30:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:30:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:30:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:30:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:30:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:30:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:30:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:30:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:30:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:30:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:30:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:30:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:30:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231480705999999, 2024-09-17T07:30:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:30:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:30:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:30:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:30:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:30:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:30:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:30:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:30:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:30:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:30:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:30:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:30:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:30:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:30:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:30:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:30:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:30:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:30:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:30:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:30:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:30:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231450516999999, 2024-09-17T07:30:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:30:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:30:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:30:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:30:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:30:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:30:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:30:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:30:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:30:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:30:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:30:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:30:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:30:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:30:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:30:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:30:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:30:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:30:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:30:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:30:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:30:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:30:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231414012000002, 2024-09-17T07:30:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:30:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:30:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:30:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:30:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:30:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:30:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:30:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:30:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:30:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:30:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:30:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:30:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:30:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:30:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:30:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:30:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:30:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:30:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:30:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:30:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:30:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:30:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231386252999997, 2024-09-17T07:30:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:30:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:30:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:30:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:30:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:30:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:30:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:30:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:30:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:30:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:30:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:30:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:30:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:30:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:30:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:30:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:30:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:30:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:30:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:30:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:30:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:30:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:30:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231364092999996, 2024-09-17T07:30:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:30:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:30:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:30:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:30:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:30:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:30:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:30:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:30:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:30:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:30:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:30:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:30:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:30:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:30:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:30:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:30:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:30:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:30:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:30:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:30:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:30:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:30:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231350931999998, 2024-09-17T07:30:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:30:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:30:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:30:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:30:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:30:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:30:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:30:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:30:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:30:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:30:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:30:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:30:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:30:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:30:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:30:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:30:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:30:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:30:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:30:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:30:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:30:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:30:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231332343999995, 2024-09-17T07:30:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:30:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:30:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:30:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:30:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:30:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:30:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:30:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:30:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:30:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:30:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:30:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:30:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:30:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:30:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:30:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:30:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:30:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:30:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:30:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:30:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231299532999998, 2024-09-17T07:30:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:30:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:30:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:30:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:30:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:30:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:30:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:30:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:30:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:30:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:30:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:30:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:30:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:30:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:30:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:30:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:30:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:30:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:30:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:30:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:30:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231280475999995, 2024-09-17T07:30:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:30:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:30:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:30:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:30:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:30:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:30:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:30:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:30:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:30:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:30:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:30:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:30:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:30:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:30:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:30:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:30:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:30:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:30:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:30:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:30:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:30:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:30:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:30:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:30:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231258366999995, 2024-09-17T07:30:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:30:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:30:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:30:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:30:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:30:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:30:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:30:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:30:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:30:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:30:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:30:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:30:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:30:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:30:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:30:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:30:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:30:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:30:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:30:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:30:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:30:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:30:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231232259999999, 2024-09-17T07:30:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:30:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:30:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:30:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:30:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:30:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:30:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:30:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:30:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:30:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:30:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:30:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:30:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:30:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:30:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:30:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:30:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:30:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:30:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:30:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:30:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:30:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:30:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231203115, 2024-09-17T07:30:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:30:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:30:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:30:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:30:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:30:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:30:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:30:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:30:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:30:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:30:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:30:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:30:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:30:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:30:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:30:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:30:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231171312000001, 2024-09-17T07:30:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:30:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:30:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:30:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:30:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:30:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:30:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:30:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:30:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:30:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:30:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:30:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:30:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:30:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:30:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:30:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:30:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:30:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:30:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:30:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:30:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:30:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:30:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:30:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231144365000006, 2024-09-17T07:30:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:30:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:30:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:30:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:30:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:30:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:30:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:30:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:30:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:30:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:30:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:30:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:30:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:30:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:30:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:30:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:30:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:30:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:30:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:30:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:30:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:30:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:30:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 84.231103415999996, 2024-09-17T07:30:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:30:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:30:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:30:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:30:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:30:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:30:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:30:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:30:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:30:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:30:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:30:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:30:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:30:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:30:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:30:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:30:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:30:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:30:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:30:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:30:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:30:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:30:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:30:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:30:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.488 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:22.489 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:22.490 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:22.489+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:22.590 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:22.590+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:22.643 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:22.643 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:22.651 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:22.691 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:22.690+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:22.763 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:22.763 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:22.773 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:22.792 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:22.792+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:22.884 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17069 3.0 e41/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 41'110, mlcod=40'108)", 2024-09-17T07:30:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 80.143441609999996, 2024-09-17T07:30:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011408908000000001, 2024-09-17T07:30:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739753+0000", 2024-09-17T07:30:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:30:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743532+0000", 2024-09-17T07:30:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037794510000000001 2024-09-17T07:30:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743557+0000", 2024-09-17T07:30:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5253e-05 2024-09-17T07:30:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743569+0000", 2024-09-17T07:30:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1161999999999999e-05 2024-09-17T07:30:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743650+0000", 2024-09-17T07:30:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1087999999999993e-05 2024-09-17T07:30:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743674+0000", 2024-09-17T07:30:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4562e-05 2024-09-17T07:30:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751095+0000", 2024-09-17T07:30:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074211140000000004 2024-09-17T07:30:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751161+0000", 2024-09-17T07:30:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5707e-05 2024-09-17T07:30:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17240 3.0 e42/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 42'113, mlcod=41'110)", 2024-09-17T07:30:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 79.657783125999998, 2024-09-17T07:30:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064392440000000002, 2024-09-17T07:30:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:30:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225418+0000", 2024-09-17T07:30:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8360000000000003e-06 2024-09-17T07:30:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225426+0000", 2024-09-17T07:30:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2670000000000006e-06 2024-09-17T07:30:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225434+0000", 2024-09-17T07:30:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0560000000000005e-06 2024-09-17T07:30:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225514+0000", 2024-09-17T07:30:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9979000000000001e-05 2024-09-17T07:30:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225535+0000", 2024-09-17T07:30:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0965e-05 2024-09-17T07:30:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231797+0000", 2024-09-17T07:30:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062621129999999997 2024-09-17T07:30:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231850+0000", 2024-09-17T07:30:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2611999999999997e-05 2024-09-17T07:30:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:32543 3.0 e55/23 3:03bfd97d:::rbd_data.11839a97b753.0000000000000007:head v 55'150, mlcod=54'148)", 2024-09-17T07:30:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.428556873000005, 2024-09-17T07:30:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031813480000000001, 2024-09-17T07:30:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5199999999999998e-07 2024-09-17T07:30:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455083+0000", 2024-09-17T07:30:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044629199999999999 2024-09-17T07:30:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455098+0000", 2024-09-17T07:30:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4183e-05 2024-09-17T07:30:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455106+0000", 2024-09-17T07:30:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:30:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455173+0000", 2024-09-17T07:30:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6346000000000005e-05 2024-09-17T07:30:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455200+0000", 2024-09-17T07:30:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7319000000000001e-05 2024-09-17T07:30:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457774+0000", 2024-09-17T07:30:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025742650000000001 2024-09-17T07:30:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457818+0000", 2024-09-17T07:30:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3891999999999999e-05 2024-09-17T07:30:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33735 3.1f e55/52 3:fea54097:::rbd_data.11839a97b753.0000000000000000:head v 55'260, mlcod=55'258)", 2024-09-17T07:30:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.058624245000004, 2024-09-17T07:30:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035215200000000002, 2024-09-17T07:30:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824571+0000", 2024-09-17T07:30:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.164e-06 2024-09-17T07:30:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825118+0000", 2024-09-17T07:30:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054696400000000004 2024-09-17T07:30:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825134+0000", 2024-09-17T07:30:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6342999999999999e-05 2024-09-17T07:30:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825142+0000", 2024-09-17T07:30:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5270000000000001e-06 2024-09-17T07:30:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825200+0000", 2024-09-17T07:30:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.893e-05 2024-09-17T07:30:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825218+0000", 2024-09-17T07:30:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7935999999999999e-05 2024-09-17T07:30:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828037+0000", 2024-09-17T07:30:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002818961 2024-09-17T07:30:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828091+0000", 2024-09-17T07:30:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3695e-05 2024-09-17T07:30:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33869 3.9 e55/23 3:907b81b5:::rbd_data.11839a97b753.0000000000000001:head v 55'171, mlcod=55'169)", 2024-09-17T07:30:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 65.041568421999997, 2024-09-17T07:30:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033990859999999999, 2024-09-17T07:30:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841626+0000", 2024-09-17T07:30:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.023e-06 2024-09-17T07:30:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842067+0000", 2024-09-17T07:30:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000440612 2024-09-17T07:30:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842080+0000", 2024-09-17T07:30:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.29e-05 2024-09-17T07:30:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842086+0000", 2024-09-17T07:30:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2090000000000002e-06 2024-09-17T07:30:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842144+0000", 2024-09-17T07:30:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7627999999999998e-05 2024-09-17T07:30:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842171+0000", 2024-09-17T07:30:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7214000000000001e-05 2024-09-17T07:30:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.844973+0000", 2024-09-17T07:30:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002802232 2024-09-17T07:30:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.845024+0000", 2024-09-17T07:30:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1267999999999998e-05 2024-09-17T07:30:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51432 3.0 e59/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 59'177, mlcod=58'174)", 2024-09-17T07:30:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 60.061971405000001, 2024-09-17T07:30:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010790086000000001, 2024-09-17T07:30:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821224+0000", 2024-09-17T07:30:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.511e-06 2024-09-17T07:30:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825174+0000", 2024-09-17T07:30:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0039497949999999999 2024-09-17T07:30:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825196+0000", 2024-09-17T07:30:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2643999999999999e-05 2024-09-17T07:30:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825206+0000", 2024-09-17T07:30:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4320000000000005e-06 2024-09-17T07:30:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825272+0000", 2024-09-17T07:30:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5829999999999998e-05 2024-09-17T07:30:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825299+0000", 2024-09-17T07:30:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7107e-05 2024-09-17T07:30:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.831939+0000", 2024-09-17T07:30:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0066398510000000004 2024-09-17T07:30:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.832012+0000", 2024-09-17T07:30:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3916000000000002e-05 2024-09-17T07:30:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51969 3.0 e59/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 59'179, mlcod=59'177)", 2024-09-17T07:30:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 59.953103026999997, 2024-09-17T07:30:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060297249999999997, 2024-09-17T07:30:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930093+0000", 2024-09-17T07:30:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.764e-06 2024-09-17T07:30:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931779+0000", 2024-09-17T07:30:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001686379 2024-09-17T07:30:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931802+0000", 2024-09-17T07:30:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2682000000000001e-05 2024-09-17T07:30:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931811+0000", 2024-09-17T07:30:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5640000000000002e-06 2024-09-17T07:30:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931877+0000", 2024-09-17T07:30:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6155999999999995e-05 2024-09-17T07:30:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931896+0000", 2024-09-17T07:30:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9111e-05 2024-09-17T07:30:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936063+0000", 2024-09-17T07:30:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004166711 2024-09-17T07:30:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936121+0000", 2024-09-17T07:30:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7358000000000002e-05 2024-09-17T07:30:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54417 3.0 e60/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 60'182, mlcod=59'179)", 2024-09-17T07:30:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 59.445586036000002, 2024-09-17T07:30:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0059885219999999996, 2024-09-17T07:30:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437609+0000", 2024-09-17T07:30:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5829999999999999e-06 2024-09-17T07:30:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437623+0000", 2024-09-17T07:30:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3348e-05 2024-09-17T07:30:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437643+0000", 2024-09-17T07:30:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0463999999999999e-05 2024-09-17T07:30:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437653+0000", 2024-09-17T07:30:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6290000000000005e-06 2024-09-17T07:30:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437719+0000", 2024-09-17T07:30:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6428999999999997e-05 2024-09-17T07:30:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437745+0000", 2024-09-17T07:30:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5690999999999998e-05 2024-09-17T07:30:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443549+0000", 2024-09-17T07:30:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0058038259999999998 2024-09-17T07:30:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443596+0000", 2024-09-17T07:30:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7552000000000001e-05 2024-09-17T07:30:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54419 3.0 e60/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 60'184, mlcod=59'179)", 2024-09-17T07:30:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 59.439233678999997, 2024-09-17T07:30:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004706844, 2024-09-17T07:30:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443961+0000", 2024-09-17T07:30:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:30:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443967+0000", 2024-09-17T07:30:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.268e-06 2024-09-17T07:30:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443983+0000", 2024-09-17T07:30:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5248999999999999e-05 2024-09-17T07:30:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443990+0000", 2024-09-17T07:30:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5120000000000002e-06 2024-09-17T07:30:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444056+0000", 2024-09-17T07:30:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6197000000000003e-05 2024-09-17T07:30:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444081+0000", 2024-09-17T07:30:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4791e-05 2024-09-17T07:30:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448616+0000", 2024-09-17T07:30:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045346129999999998 2024-09-17T07:30:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448667+0000", 2024-09-17T07:30:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-17T07:30:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:80794 3.9 e72/23 3:90311a61:::rbd_data.1183cb746bb8.0000000000000007:head v 72'222, mlcod=71'220)", 2024-09-17T07:30:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 45.451244795000001, 2024-09-17T07:30:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004371733, 2024-09-17T07:30:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0800000000000001e-07 2024-09-17T07:30:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434108+0000", 2024-09-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021587569999999999 2024-09-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434120+0000", 2024-09-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1821000000000001e-05 2024-09-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434128+0000", 2024-09-17T07:30:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7579999999999992e-06 2024-09-17T07:30:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434191+0000", 2024-09-17T07:30:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3202999999999998e-05 2024-09-17T07:30:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434218+0000", 2024-09-17T07:30:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6771000000000002e-05 2024-09-17T07:30:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436289+0000", 2024-09-17T07:30:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020717700000000001 2024-09-17T07:30:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436321+0000", 2024-09-17T07:30:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1445000000000001e-05 2024-09-17T07:30:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 75/74 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head, version: 36'126, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head@36'126, size: 9431460, copy_subset: [], clone_subset: {3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9: [(0, 9431460)]}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.142337202, 2024-09-17T07:30:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041288289999999997, 2024-09-17T07:30:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740858+0000", 2024-09-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1599999999999999e-06 2024-09-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740885+0000", 2024-09-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.707e-05 2024-09-17T07:30:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740925+0000", 2024-09-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0649000000000003e-05 2024-09-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740957+0000", 2024-09-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1585999999999999e-05 2024-09-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741185+0000", 2024-09-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022759700000000001 2024-09-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741228+0000", 2024-09-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3705999999999998e-05 2024-09-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.744985+0000", 2024-09-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037570609999999999 2024-09-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85061 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'257, mlcod=79'255)", 2024-09-17T07:30:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 37.734506246000002, 2024-09-17T07:30:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033850550000000001, 2024-09-17T07:30:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9699999999999995e-07 2024-09-17T07:30:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148835+0000", 2024-09-17T07:30:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000146852 2024-09-17T07:30:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148845+0000", 2024-09-17T07:30:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0232e-05 2024-09-17T07:30:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148852+0000", 2024-09-17T07:30:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6490000000000002e-06 2024-09-17T07:30:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148913+0000", 2024-09-17T07:30:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1099999999999994e-05 2024-09-17T07:30:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148931+0000", 2024-09-17T07:30:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7938e-05 2024-09-17T07:30:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152025+0000", 2024-09-17T07:30:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0030942729999999998 2024-09-17T07:30:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152073+0000", 2024-09-17T07:30:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7213999999999999e-05 2024-09-17T07:30:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85063 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'259, mlcod=79'255)", 2024-09-17T07:30:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 37.733359292000003, 2024-09-17T07:30:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037722659999999998, 2024-09-17T07:30:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5799999999999999e-07 2024-09-17T07:30:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149841+0000", 2024-09-17T07:30:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4840000000000003e-06 2024-09-17T07:30:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149849+0000", 2024-09-17T07:30:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4670000000000004e-06 2024-09-17T07:30:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149854+0000", 2024-09-17T07:30:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4779999999999999e-06 2024-09-17T07:30:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149920+0000", 2024-09-17T07:30:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5760999999999996e-05 2024-09-17T07:30:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149949+0000", 2024-09-17T07:30:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8838000000000001e-05 2024-09-17T07:30:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153558+0000", 2024-09-17T07:30:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036092939999999999 2024-09-17T07:30:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153607+0000", 2024-09-17T07:30:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8386000000000001e-05 2024-09-17T07:30:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:94996 3.0 e82/23 3:074b4f56:::rbd_data.118311e0d6f8.0000000000000001:head v 82'715, mlcod=80'713)", 2024-09-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 33.267453699000001, 2024-09-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031499179999999998, 2024-09-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615741+0000", 2024-09-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999996e-07 2024-09-17T07:30:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616917+0000", 2024-09-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011763310000000001 2024-09-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616936+0000", 2024-09-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8624e-05 2024-09-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616948+0000", 2024-09-17T07:30:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.238e-05 2024-09-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617054+0000", 2024-09-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000106556 2024-09-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617081+0000", 2024-09-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.622e-05 2024-09-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618836+0000", 2024-09-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001755412 2024-09-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618890+0000", 2024-09-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3915000000000001e-05 2024-09-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:95007 3.5 e82/76 3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head v 82'422, mlcod=82'420)", 2024-09-17T07:30:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 33.259530998000002, 2024-09-17T07:30:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003939901, 2024-09-17T07:30:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623664+0000", 2024-09-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3799999999999999e-06 2024-09-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626104+0000", 2024-09-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002440149 2024-09-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626120+0000", 2024-09-17T07:30:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5962000000000001e-05 2024-09-17T07:30:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626127+0000", 2024-09-17T07:30:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8340000000000001e-06 2024-09-17T07:30:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626186+0000", 2024-09-17T07:30:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9234999999999998e-05 2024-09-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626208+0000", 2024-09-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1719e-05 2024-09-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627567+0000", 2024-09-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013587919999999999 2024-09-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627603+0000", 2024-09-17T07:30:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5830000000000001e-05 2024-09-17T07:30:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:104928 3.18 e87/76 3:1ef990b1:::rbd_data.1183dd51323.0000000000000006:head v 87'562, mlcod=86'559)", 2024-09-17T07:30:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 27.340775287, 2024-09-17T07:30:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0061095869999999997, 2024-09-17T07:30:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542421+0000", 2024-09-17T07:30:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.069e-06 2024-09-17T07:30:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544190+0000", 2024-09-17T07:30:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017693539999999999 2024-09-17T07:30:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544215+0000", 2024-09-17T07:30:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5398999999999999e-05 2024-09-17T07:30:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544229+0000", 2024-09-17T07:30:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3865e-05 2024-09-17T07:30:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544322+0000", 2024-09-17T07:30:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2540000000000005e-05 2024-09-17T07:30:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544348+0000", 2024-09-17T07:30:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6197999999999999e-05 2024-09-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548475+0000", 2024-09-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041266000000000002 2024-09-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548528+0000", 2024-09-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3562e-05 2024-09-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115898 3.9 3:910101b0:::rbd_data.118340e4fbf.0000000000000000:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.69226118199999997, 2024-09-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0042466279999999997, 2024-09-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115898 2024-09-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0030000000000001e-06 2024-09-17T07:30:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190940+0000", 2024-09-17T07:30:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2349999999999999e-06 2024-09-17T07:30:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190953+0000", 2024-09-17T07:30:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3304e-05 2024-09-17T07:30:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190995+0000", 2024-09-17T07:30:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2317000000000002e-05 2024-09-17T07:30:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191592+0000", 2024-09-17T07:30:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00059607799999999999 2024-09-17T07:30:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191888+0000", 2024-09-17T07:30:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00029619999999999999 2024-09-17T07:30:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192069+0000", 2024-09-17T07:30:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00018125899999999999 2024-09-17T07:30:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193692+0000", 2024-09-17T07:30:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001623364 2024-09-17T07:30:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194968+0000", 2024-09-17T07:30:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012755189999999999 2024-09-17T07:30:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.195037+0000", 2024-09-17T07:30:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9450999999999996e-05 2024-09-17T07:30:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.195179+0000", 2024-09-17T07:30:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000141898 2024-09-17T07:30:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115936 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [watch watch cookie 140227662410368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.65934223000000003, 2024-09-17T07:30:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043930569999999997, 2024-09-17T07:30:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115936 2024-09-17T07:30:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7300000000000005e-07 2024-09-17T07:30:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223856+0000", 2024-09-17T07:30:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9169999999999999e-06 2024-09-17T07:30:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223870+0000", 2024-09-17T07:30:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3926e-05 2024-09-17T07:30:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223878+0000", 2024-09-17T07:30:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3699999999999997e-06 2024-09-17T07:30:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223934+0000", 2024-09-17T07:30:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6929000000000003e-05 2024-09-17T07:30:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224053+0000", 2024-09-17T07:30:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000118952 2024-09-17T07:30:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224271+0000", 2024-09-17T07:30:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021763300000000001 2024-09-17T07:30:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224992+0000", 2024-09-17T07:30:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072142099999999998 2024-09-17T07:30:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228099+0000", 2024-09-17T07:30:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031060850000000002 2024-09-17T07:30:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228143+0000", 2024-09-17T07:30:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4614999999999999e-05 2024-09-17T07:30:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228245+0000", 2024-09-17T07:30:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000101436 2024-09-17T07:30:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115955 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [call rbd.snapshot_trash_add in=21b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:30:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.64035547699999995, 2024-09-17T07:30:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004259054, 2024-09-17T07:30:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115955 2024-09-17T07:30:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:30:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:30:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242839+0000", 2024-09-17T07:30:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6899999999999998e-07 2024-09-17T07:30:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242842+0000", 2024-09-17T07:30:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1350000000000001e-06 2024-09-17T07:30:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242849+0000", 2024-09-17T07:30:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0400000000000004e-06 2024-09-17T07:30:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242854+0000", 2024-09-17T07:30:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4940000000000003e-06 2024-09-17T07:30:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243137+0000", 2024-09-17T07:30:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00028217100000000001 2024-09-17T07:30:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243196+0000", 2024-09-17T07:30:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9608999999999998e-05 2024-09-17T07:30:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243448+0000", 2024-09-17T07:30:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00025185000000000002 2024-09-17T07:30:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243990+0000", 2024-09-17T07:30:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054153199999999995 2024-09-17T07:30:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.246867+0000", 2024-09-17T07:30:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028771880000000001 2024-09-17T07:30:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.246943+0000", 2024-09-17T07:30:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5689e-05 2024-09-17T07:30:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247097+0000", 2024-09-17T07:30:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015487699999999999 2024-09-17T07:30:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115956 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:30:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.64005889299999996, 2024-09-17T07:30:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043741479999999996, 2024-09-17T07:30:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115956 2024-09-17T07:30:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:30:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:30:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243136+0000", 2024-09-17T07:30:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7100000000000001e-07 2024-09-17T07:30:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243141+0000", 2024-09-17T07:30:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.716e-06 2024-09-17T07:30:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243153+0000", 2024-09-17T07:30:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1198e-05 2024-09-17T07:30:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243160+0000", 2024-09-17T07:30:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6229999999999997e-06 2024-09-17T07:30:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243797+0000", 2024-09-17T07:30:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063716399999999996 2024-09-17T07:30:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for rw locks", 2024-09-17T07:30:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243852+0000", 2024-09-17T07:30:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4147999999999997e-05 2024-09-17T07:30:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247147+0000", 2024-09-17T07:30:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0032955940000000002 2024-09-17T07:30:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247271+0000", 2024-09-17T07:30:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012397800000000001 2024-09-17T07:30:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247509+0000", 2024-09-17T07:30:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000237956 2024-09-17T07:30:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:22.989 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:22.991 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:22.991 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:22.989+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:23.091 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:23.090+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:23.099 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:23.100 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:23.100 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:23.109 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:23.191 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:23.191+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:23.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:30:23.269 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:23.269 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:23.269 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:23.280 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:23.292 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:23.291+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:23.341 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:23.360 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:23.380 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:23.380 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:23.393 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:23.392+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:23.396 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:23.397 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:23.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:23.397 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:23.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.397 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:30525 3.1b 3:def7c8f0:::rbd_data.118381ffbbab.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T07:30:23.397 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:23.397 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.872627879999996, 2024-09-17T07:30:23.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55515850300000003, 2024-09-17T07:30:23.397 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.397 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.398 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:23.398 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.398 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.398 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.398 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30525 2024-09-17T07:30:23.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.398 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:23.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:23.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522300+0000", 2024-09-17T07:30:23.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:30:23.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522305+0000", 2024-09-17T07:30:23.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-17T07:30:23.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522318+0000", 2024-09-17T07:30:23.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3030999999999999e-05 2024-09-17T07:30:23.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522326+0000", 2024-09-17T07:30:23.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4200000000000001e-06 2024-09-17T07:30:23.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522385+0000", 2024-09-17T07:30:23.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9005000000000002e-05 2024-09-17T07:30:23.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:23.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522393+0000", 2024-09-17T07:30:23.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-06 2024-09-17T07:30:23.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075347+0000", 2024-09-17T07:30:23.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55295417800000002 2024-09-17T07:30:23.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075502+0000", 2024-09-17T07:30:23.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154272 2024-09-17T07:30:23.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:23.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075630+0000", 2024-09-17T07:30:23.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012816300000000001 2024-09-17T07:30:23.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:23.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077221+0000", 2024-09-17T07:30:23.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015912929999999999 2024-09-17T07:30:23.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:23.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077353+0000", 2024-09-17T07:30:23.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013215699999999999 2024-09-17T07:30:23.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:23.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077413+0000", 2024-09-17T07:30:23.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0034999999999997e-05 2024-09-17T07:30:23.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077457+0000", 2024-09-17T07:30:23.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4017999999999998e-05 2024-09-17T07:30:23.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.405 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44181 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13922304~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.757449231999999, 2024-09-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046010502000000002, 2024-09-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44181 2024-09-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:23.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:23.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637478+0000", 2024-09-17T07:30:23.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:30:23.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637480+0000", 2024-09-17T07:30:23.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.843e-06 2024-09-17T07:30:23.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637487+0000", 2024-09-17T07:30:23.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6159999999999996e-06 2024-09-17T07:30:23.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637492+0000", 2024-09-17T07:30:23.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5360000000000003e-06 2024-09-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683351+0000", 2024-09-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045858758999999999 2024-09-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683422+0000", 2024-09-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1705999999999994e-05 2024-09-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683488+0000", 2024-09-17T07:30:23.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5612999999999997e-05 2024-09-17T07:30:23.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.410 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.410 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44182 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13938688~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.410 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:23.410 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.757428560999998, 2024-09-17T07:30:23.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046153118999999999, 2024-09-17T07:30:23.410 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.410 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.410 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.410 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.411 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44182 2024-09-17T07:30:23.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.411 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:23.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:23.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:23.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:30:23.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637506+0000", 2024-09-17T07:30:23.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1520000000000003e-06 2024-09-17T07:30:23.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637512+0000", 2024-09-17T07:30:23.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-09-17T07:30:23.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637517+0000", 2024-09-17T07:30:23.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-09-17T07:30:23.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683514+0000", 2024-09-17T07:30:23.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045997616999999998 2024-09-17T07:30:23.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683584+0000", 2024-09-17T07:30:23.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9942000000000002e-05 2024-09-17T07:30:23.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683651+0000", 2024-09-17T07:30:23.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6951000000000003e-05 2024-09-17T07:30:23.414 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.414 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.414 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.414 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44183 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13955072~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.414 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:23.414 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.757403328999999, 2024-09-17T07:30:23.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046339793999999997, 2024-09-17T07:30:23.415 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.415 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.415 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.415 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.415 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44183 2024-09-17T07:30:23.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.415 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:23.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:23.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639583+0000", 2024-09-17T07:30:23.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002059412 2024-09-17T07:30:23.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639597+0000", 2024-09-17T07:30:23.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3832000000000001e-05 2024-09-17T07:30:23.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639607+0000", 2024-09-17T07:30:23.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485e-05 2024-09-17T07:30:23.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639614+0000", 2024-09-17T07:30:23.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0849999999999999e-06 2024-09-17T07:30:23.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683705+0000", 2024-09-17T07:30:23.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044091261 2024-09-17T07:30:23.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683792+0000", 2024-09-17T07:30:23.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6197000000000002e-05 2024-09-17T07:30:23.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683863+0000", 2024-09-17T07:30:23.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1521999999999997e-05 2024-09-17T07:30:23.418 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.418 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.419 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.419 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44199 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14217216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.419 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:23.419 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.753885777000001, 2024-09-17T07:30:23.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045995623999999999, 2024-09-17T07:30:23.419 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.419 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.419 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.419 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44199 2024-09-17T07:30:23.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.420 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:23.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:23.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:23.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:30:23.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641043+0000", 2024-09-17T07:30:23.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.624e-06 2024-09-17T07:30:23.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641048+0000", 2024-09-17T07:30:23.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4029999999999998e-06 2024-09-17T07:30:23.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641053+0000", 2024-09-17T07:30:23.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6670000000000004e-06 2024-09-17T07:30:23.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686918+0000", 2024-09-17T07:30:23.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045865131000000003 2024-09-17T07:30:23.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686986+0000", 2024-09-17T07:30:23.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8299000000000005e-05 2024-09-17T07:30:23.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687036+0000", 2024-09-17T07:30:23.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0164999999999998e-05 2024-09-17T07:30:23.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.423 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.423 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44200 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14233600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.423 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:23.423 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.753867948, 2024-09-17T07:30:23.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046183984999999997, 2024-09-17T07:30:23.423 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.423 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.424 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.424 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44200 2024-09-17T07:30:23.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.424 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:23.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:23.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:23.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:23.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641061+0000", 2024-09-17T07:30:23.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.767e-06 2024-09-17T07:30:23.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641067+0000", 2024-09-17T07:30:23.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7379999999999999e-06 2024-09-17T07:30:23.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641071+0000", 2024-09-17T07:30:23.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-09-17T07:30:23.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687078+0000", 2024-09-17T07:30:23.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046007095999999997 2024-09-17T07:30:23.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687158+0000", 2024-09-17T07:30:23.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0374e-05 2024-09-17T07:30:23.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687243+0000", 2024-09-17T07:30:23.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4221999999999994e-05 2024-09-17T07:30:23.427 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.427 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.427 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.428 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44201 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14249984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.428 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:23.428 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.753850302000004, 2024-09-17T07:30:23.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046303182999999998, 2024-09-17T07:30:23.428 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.428 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.428 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.428 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.428 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.428 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.428 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44201 2024-09-17T07:30:23.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.428 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641120+0000", 2024-09-17T07:30:23.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3933000000000001e-05 2024-09-17T07:30:23.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641123+0000", 2024-09-17T07:30:23.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.191e-06 2024-09-17T07:30:23.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641133+0000", 2024-09-17T07:30:23.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4020000000000008e-06 2024-09-17T07:30:23.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641138+0000", 2024-09-17T07:30:23.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6559999999999997e-06 2024-09-17T07:30:23.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687279+0000", 2024-09-17T07:30:23.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046140278999999999 2024-09-17T07:30:23.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687338+0000", 2024-09-17T07:30:23.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9268999999999998e-05 2024-09-17T07:30:23.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687379+0000", 2024-09-17T07:30:23.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1452999999999998e-05 2024-09-17T07:30:23.432 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.432 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.432 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.432 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44202 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14266368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.432 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:23.432 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.753779536000003, 2024-09-17T07:30:23.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046484299, 2024-09-17T07:30:23.432 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.432 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.433 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.433 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.433 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44202 2024-09-17T07:30:23.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.433 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:23.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:23.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642810+0000", 2024-09-17T07:30:23.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016627980000000001 2024-09-17T07:30:23.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642815+0000", 2024-09-17T07:30:23.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.091e-06 2024-09-17T07:30:23.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642829+0000", 2024-09-17T07:30:23.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4047e-05 2024-09-17T07:30:23.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642844+0000", 2024-09-17T07:30:23.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4515000000000001e-05 2024-09-17T07:30:23.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687426+0000", 2024-09-17T07:30:23.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044582201000000002 2024-09-17T07:30:23.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687529+0000", 2024-09-17T07:30:23.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103078 2024-09-17T07:30:23.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687631+0000", 2024-09-17T07:30:23.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102569 2024-09-17T07:30:23.436 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.436 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.436 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.437 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44209 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14381056~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.437 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:23.437 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.751881384000001, 2024-09-17T07:30:23.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045944444000000001, 2024-09-17T07:30:23.437 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.437 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.437 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.437 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44209 2024-09-17T07:30:23.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.437 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:23.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:23.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643046+0000", 2024-09-17T07:30:23.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:30:23.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643048+0000", 2024-09-17T07:30:23.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:30:23.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643054+0000", 2024-09-17T07:30:23.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.032e-06 2024-09-17T07:30:23.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643065+0000", 2024-09-17T07:30:23.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1861999999999999e-05 2024-09-17T07:30:23.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688806+0000", 2024-09-17T07:30:23.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045740617999999997 2024-09-17T07:30:23.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688927+0000", 2024-09-17T07:30:23.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000121327 2024-09-17T07:30:23.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688990+0000", 2024-09-17T07:30:23.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2293999999999997e-05 2024-09-17T07:30:23.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.441 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.441 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44210 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14397440~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.441 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:23.441 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.751854457999997, 2024-09-17T07:30:23.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046104635999999997, 2024-09-17T07:30:23.441 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.441 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.441 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.442 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44210 2024-09-17T07:30:23.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.442 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:23.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:23.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643073+0000", 2024-09-17T07:30:23.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.77e-07 2024-09-17T07:30:23.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643075+0000", 2024-09-17T07:30:23.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-09-17T07:30:23.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643082+0000", 2024-09-17T07:30:23.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7789999999999996e-06 2024-09-17T07:30:23.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643087+0000", 2024-09-17T07:30:23.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.967e-06 2024-09-17T07:30:23.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689022+0000", 2024-09-17T07:30:23.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045934349999999999 2024-09-17T07:30:23.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689112+0000", 2024-09-17T07:30:23.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0039999999999999e-05 2024-09-17T07:30:23.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689177+0000", 2024-09-17T07:30:23.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5041000000000003e-05 2024-09-17T07:30:23.445 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.445 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.446 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.446 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44211 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14413824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.446 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:23.446 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.751833325, 2024-09-17T07:30:23.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046309003000000001, 2024-09-17T07:30:23.446 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.446 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.446 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.446 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.446 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.446 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.446 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44211 2024-09-17T07:30:23.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.447 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:23.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:23.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643094+0000", 2024-09-17T07:30:23.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:30:23.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643095+0000", 2024-09-17T07:30:23.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-17T07:30:23.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643101+0000", 2024-09-17T07:30:23.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8730000000000004e-06 2024-09-17T07:30:23.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643106+0000", 2024-09-17T07:30:23.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6859999999999997e-06 2024-09-17T07:30:23.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689220+0000", 2024-09-17T07:30:23.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046113945000000003 2024-09-17T07:30:23.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689309+0000", 2024-09-17T07:30:23.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9560000000000003e-05 2024-09-17T07:30:23.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689402+0000", 2024-09-17T07:30:23.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2786000000000002e-05 2024-09-17T07:30:23.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.450 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.451 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44212 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14430208~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.451 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:23.451 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.751788763, 2024-09-17T07:30:23.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046448912000000002, 2024-09-17T07:30:23.451 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.451 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.451 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.451 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44212 2024-09-17T07:30:23.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.451 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:23.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:23.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:23.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2200000000000004e-07 2024-09-17T07:30:23.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643141+0000", 2024-09-17T07:30:23.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5210000000000001e-06 2024-09-17T07:30:23.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643149+0000", 2024-09-17T07:30:23.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6490000000000008e-06 2024-09-17T07:30:23.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643161+0000", 2024-09-17T07:30:23.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2241e-05 2024-09-17T07:30:23.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689423+0000", 2024-09-17T07:30:23.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046262323000000001 2024-09-17T07:30:23.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689505+0000", 2024-09-17T07:30:23.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2103999999999999e-05 2024-09-17T07:30:23.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689587+0000", 2024-09-17T07:30:23.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1452000000000006e-05 2024-09-17T07:30:23.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.455 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.455 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44213 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14446592~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.455 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:23.455 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.751757263999998, 2024-09-17T07:30:23.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046601780000000002, 2024-09-17T07:30:23.455 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.455 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.455 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.456 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44213 2024-09-17T07:30:23.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.456 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:23.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:23.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643170+0000", 2024-09-17T07:30:23.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-17T07:30:23.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643172+0000", 2024-09-17T07:30:23.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1660000000000001e-06 2024-09-17T07:30:23.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643180+0000", 2024-09-17T07:30:23.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-09-17T07:30:23.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643185+0000", 2024-09-17T07:30:23.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-09-17T07:30:23.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689622+0000", 2024-09-17T07:30:23.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046436742000000003 2024-09-17T07:30:23.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689697+0000", 2024-09-17T07:30:23.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5179999999999995e-05 2024-09-17T07:30:23.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689771+0000", 2024-09-17T07:30:23.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3928000000000001e-05 2024-09-17T07:30:23.459 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.459 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.459 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.460 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44214 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14462976~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.460 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:23.460 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.751734204000002, 2024-09-17T07:30:23.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046736038000000001, 2024-09-17T07:30:23.460 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.460 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.460 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.460 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.460 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.460 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.460 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44214 2024-09-17T07:30:23.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.461 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:23.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:23.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643193+0000", 2024-09-17T07:30:23.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:30:23.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643195+0000", 2024-09-17T07:30:23.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0640000000000001e-06 2024-09-17T07:30:23.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643201+0000", 2024-09-17T07:30:23.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2859999999999996e-06 2024-09-17T07:30:23.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643206+0000", 2024-09-17T07:30:23.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-17T07:30:23.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689814+0000", 2024-09-17T07:30:23.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046607742000000001 2024-09-17T07:30:23.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689886+0000", 2024-09-17T07:30:23.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2020999999999994e-05 2024-09-17T07:30:23.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689928+0000", 2024-09-17T07:30:23.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2771000000000001e-05 2024-09-17T07:30:23.464 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.464 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.464 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.464 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44215 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14479360~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.464 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:23.464 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.751690306999997, 2024-09-17T07:30:23.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046868543999999998, 2024-09-17T07:30:23.464 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.465 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.465 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.465 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44215 2024-09-17T07:30:23.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.465 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:23.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:23.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643237+0000", 2024-09-17T07:30:23.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:30:23.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643245+0000", 2024-09-17T07:30:23.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3270000000000001e-06 2024-09-17T07:30:23.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643254+0000", 2024-09-17T07:30:23.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6049999999999998e-06 2024-09-17T07:30:23.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643260+0000", 2024-09-17T07:30:23.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7849999999999997e-06 2024-09-17T07:30:23.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689961+0000", 2024-09-17T07:30:23.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046701196 2024-09-17T07:30:23.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690036+0000", 2024-09-17T07:30:23.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5153000000000004e-05 2024-09-17T07:30:23.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690105+0000", 2024-09-17T07:30:23.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8874000000000005e-05 2024-09-17T07:30:23.468 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.468 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.468 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.469 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44216 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14495744~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.469 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:23.469 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.751658593999998, 2024-09-17T07:30:23.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047003047999999999, 2024-09-17T07:30:23.469 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.469 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.469 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.469 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.469 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.469 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.469 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44216 2024-09-17T07:30:23.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.470 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:23.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:23.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643269+0000", 2024-09-17T07:30:23.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:30:23.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643271+0000", 2024-09-17T07:30:23.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-09-17T07:30:23.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643278+0000", 2024-09-17T07:30:23.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8850000000000001e-06 2024-09-17T07:30:23.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643282+0000", 2024-09-17T07:30:23.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5979999999999999e-06 2024-09-17T07:30:23.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690133+0000", 2024-09-17T07:30:23.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850470999999998 2024-09-17T07:30:23.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690205+0000", 2024-09-17T07:30:23.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2442999999999997e-05 2024-09-17T07:30:23.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690271+0000", 2024-09-17T07:30:23.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5874e-05 2024-09-17T07:30:23.473 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.473 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.473 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.473 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44217 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14512128~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.473 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:23.473 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.751637582000001, 2024-09-17T07:30:23.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047166485000000001, 2024-09-17T07:30:23.473 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.473 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.474 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.474 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44217 2024-09-17T07:30:23.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.474 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:23.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:23.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:23.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-17T07:30:23.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643291+0000", 2024-09-17T07:30:23.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.962e-06 2024-09-17T07:30:23.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643297+0000", 2024-09-17T07:30:23.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0700000000000003e-06 2024-09-17T07:30:23.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643302+0000", 2024-09-17T07:30:23.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:30:23.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690308+0000", 2024-09-17T07:30:23.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047005354999999999 2024-09-17T07:30:23.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690385+0000", 2024-09-17T07:30:23.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7541999999999997e-05 2024-09-17T07:30:23.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690455+0000", 2024-09-17T07:30:23.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0328999999999997e-05 2024-09-17T07:30:23.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.478 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.478 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44218 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14528512~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.478 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:23.478 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.751617905000003, 2024-09-17T07:30:23.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047321015000000001, 2024-09-17T07:30:23.478 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.478 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.478 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.479 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44218 2024-09-17T07:30:23.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.479 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:23.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:23.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:23.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-17T07:30:23.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643311+0000", 2024-09-17T07:30:23.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.933e-06 2024-09-17T07:30:23.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643318+0000", 2024-09-17T07:30:23.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3980000000000004e-06 2024-09-17T07:30:23.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643322+0000", 2024-09-17T07:30:23.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7609999999999998e-06 2024-09-17T07:30:23.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690484+0000", 2024-09-17T07:30:23.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047161428999999998 2024-09-17T07:30:23.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690564+0000", 2024-09-17T07:30:23.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9845000000000005e-05 2024-09-17T07:30:23.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690630+0000", 2024-09-17T07:30:23.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6021999999999999e-05 2024-09-17T07:30:23.482 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.482 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.482 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.483 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44219 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14544896~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.483 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:23.483 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.751597537999999, 2024-09-17T07:30:23.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048128708999999999, 2024-09-17T07:30:23.483 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.483 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.483 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.483 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.483 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.483 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.483 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44219 2024-09-17T07:30:23.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.484 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:23.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:23.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:23.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:30:23.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643337+0000", 2024-09-17T07:30:23.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.109e-06 2024-09-17T07:30:23.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643350+0000", 2024-09-17T07:30:23.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3046e-05 2024-09-17T07:30:23.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643356+0000", 2024-09-17T07:30:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8930000000000002e-06 2024-09-17T07:30:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690660+0000", 2024-09-17T07:30:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047304061000000001 2024-09-17T07:30:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691372+0000", 2024-09-17T07:30:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00071281299999999997 2024-09-17T07:30:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691458+0000", 2024-09-17T07:30:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5352000000000005e-05 2024-09-17T07:30:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44220 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14561280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.751563490000002, 2024-09-17T07:30:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048309849000000002, 2024-09-17T07:30:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:23.488 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:23.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:23.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:23.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:23.488 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44220 2024-09-17T07:30:23.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.488 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:23.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:23.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:23.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:23.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:23.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:23.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:23.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646869+0000", 2024-09-17T07:30:23.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003506267 2024-09-17T07:30:23.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:23.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646874+0000", 2024-09-17T07:30:23.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7500000000000003e-06 2024-09-17T07:30:23.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:23.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646887+0000", 2024-09-17T07:30:23.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3115999999999999e-05 2024-09-17T07:30:23.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:23.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646894+0000", 2024-09-17T07:30:23.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5289999999999997e-06 2024-09-17T07:30:23.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:23.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691547+0000", 2024-09-17T07:30:23.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044652743000000002 2024-09-17T07:30:23.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:23.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691622+0000", 2024-09-17T07:30:23.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5037999999999995e-05 2024-09-17T07:30:23.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:23.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:23.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:23.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691673+0000", 2024-09-17T07:30:23.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-17T07:30:23.491 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.491 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.491 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.491 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:23.491 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:23.492 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:23.492 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:23.493 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:23.493 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:23.494 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:23.494 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:23.494 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:23.495 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:23.495 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:23.496 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:23.497 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:23.497+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:23.598 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:23.597+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:23.600 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:23.600 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:23.609 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:23.698 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:23.698+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:23.751 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:23.751 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:23.760 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:23.799 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:23.799+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:23.900 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:23.900+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:23.902 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:23.903 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:23.911 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:24.001 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:24.001+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:24.015 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:24.015 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:24.024 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:24.102 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:24.102+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:24.107 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T07:30:24.168 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:24.168 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:24.177 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:24.220 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:24.220+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:24.298 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:24.298 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:24.304 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:24.303+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:24.308 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:24.406 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:24.405+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:24.452 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:24.452 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:24.460 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:24.544 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:24.544+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:24.602 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:24.603 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:24.607 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:24.607+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:24.611 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:24.708 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:24.707+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:24.754 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:24.754 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:24.762 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:24.808 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:24.808+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:24.875 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:24.875 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:24.883 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:24.909 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:24.908+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:25.010 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:25.010+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:25.031 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:25.031 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:25.040 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:25.108 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 5, 6, 7, 3] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-17T07:30:25.108 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:30:25.109 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-17T07:30:25.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-09-17T07:30:25.111 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:25.111+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:25.188 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:25.188 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:25.197 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:25.212 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:25.211+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:25.237 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-17T07:30:25.313 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:25.313+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:25.316 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:25.317 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:25.325 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:25.414 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:25.414+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:25.454 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-17T07:30:25.454+0000 7f6d53957640 -1 mon.a@0(leader).osd e116 definitely_dead 0 2024-09-17T07:30:25.473 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:25.473 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:25.482 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:25.497 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:30:25.523 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:25.522+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:25.617 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:25.616+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:25.648 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:25.648 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:25.657 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:25.664 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:25.664 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:25.681 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:25.682 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:25.698 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:25.698 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:25.716 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:25.716 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:25.717 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:25.716+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:25.734 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:25.751 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:25.751 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:25.766 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:25.767 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:25.767 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:25.768 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:25.768 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:25.775 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:25.785 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:25.785 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:25.818 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:25.817+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:25.892 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:25.892 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:25.900 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:25.918 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:25.918+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:26.010 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:26.010 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:26.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:26.010 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:26.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.010 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.010 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.875099779999999, 2024-09-17T07:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.011 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:30:26.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:30:26.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.016 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.875062882999998, 2024-09-17T07:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:30:26.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:30:26.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.875036915999999, 2024-09-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:30:26.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:30:26.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:30:26.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.028 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.875012321, 2024-09-17T07:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.029 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.031 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.031 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:26.031 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:30:26.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.031 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.031 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:30:26.031 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:30:26.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:30:26.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874988834000007, 2024-09-17T07:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.035 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:30:26.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874975096, 2024-09-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.046 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874944907, 2024-09-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:30:26.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.048 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.049 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.052 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874908402000003, 2024-09-17T07:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.053 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:30:26.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:30:26.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:26.058 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874880642999997, 2024-09-17T07:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:26.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.060 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:30:26.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874858482999997, 2024-09-17T07:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:30:26.064 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.066 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.067 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:30:26.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:30:26.069 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874845321999999, 2024-09-17T07:30:26.070 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:26.071 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.072 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:30:26.074 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:30:26.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874826733999996, 2024-09-17T07:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.077 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.078 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:30:26.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:30:26.080 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.081 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:26.082 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874793922999999, 2024-09-17T07:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.084 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.084 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:26.084 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.084 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.084 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:26.084 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:30:26.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.084 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.084 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:30:26.084 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.085 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:30:26.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:30:26.087 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:26.088 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874774865999996, 2024-09-17T07:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:26.089 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:30:26.090 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:30:26.091 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.092 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874752756999996, 2024-09-17T07:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.094 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:30:26.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:30:26.097 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.098 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.099 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874726649999999, 2024-09-17T07:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:26.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.103 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:30:26.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874697505, 2024-09-17T07:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:30:26.106 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.107 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.108 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:30:26.110 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:30:26.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.111 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.111 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.111 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:30:26.111 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:30:26.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.111 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.111 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.111 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:30:26.111 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:26.112 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874665702000001, 2024-09-17T07:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:26.113 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.114 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.115 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:30:26.116 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.117 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.118 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874638755000007, 2024-09-17T07:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.119 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:30:26.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:30:26.122 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.123 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:26.124 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.874597805999997, 2024-09-17T07:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.125 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.126 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.127 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:30:26.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.130 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.130 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:30:26.130 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:30:26.130 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.130 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.130 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.130 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.130 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.130 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:26.131 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:26.132 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:26.131+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:26.232 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:26.232+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:26.241 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:26.241 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:26.241 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:26.242 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:26.244 INFO:teuthology.orchestra.run.smithi045.stderr:marked in osd.4. 2024-09-17T07:30:26.258 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:26.265 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-17T07:30:26.334 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:26.333+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:26.412 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:26.413 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:26.421 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:26.465 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:26.465+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:26.530 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17069 3.0 e41/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 41'110, mlcod=40'108)", 2024-09-17T07:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 83.789106705999998, 2024-09-17T07:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011408908000000001, 2024-09-17T07:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739753+0000", 2024-09-17T07:30:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743532+0000", 2024-09-17T07:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037794510000000001 2024-09-17T07:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743557+0000", 2024-09-17T07:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5253e-05 2024-09-17T07:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743569+0000", 2024-09-17T07:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1161999999999999e-05 2024-09-17T07:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743650+0000", 2024-09-17T07:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1087999999999993e-05 2024-09-17T07:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743674+0000", 2024-09-17T07:30:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4562e-05 2024-09-17T07:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751095+0000", 2024-09-17T07:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074211140000000004 2024-09-17T07:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751161+0000", 2024-09-17T07:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5707e-05 2024-09-17T07:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17240 3.0 e42/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 42'113, mlcod=41'110)", 2024-09-17T07:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 83.303448222, 2024-09-17T07:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064392440000000002, 2024-09-17T07:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225418+0000", 2024-09-17T07:30:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8360000000000003e-06 2024-09-17T07:30:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225426+0000", 2024-09-17T07:30:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2670000000000006e-06 2024-09-17T07:30:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225434+0000", 2024-09-17T07:30:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0560000000000005e-06 2024-09-17T07:30:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225514+0000", 2024-09-17T07:30:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9979000000000001e-05 2024-09-17T07:30:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225535+0000", 2024-09-17T07:30:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0965e-05 2024-09-17T07:30:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231797+0000", 2024-09-17T07:30:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062621129999999997 2024-09-17T07:30:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231850+0000", 2024-09-17T07:30:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2611999999999997e-05 2024-09-17T07:30:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:32543 3.0 e55/23 3:03bfd97d:::rbd_data.11839a97b753.0000000000000007:head v 55'150, mlcod=54'148)", 2024-09-17T07:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 69.074221969000007, 2024-09-17T07:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031813480000000001, 2024-09-17T07:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5199999999999998e-07 2024-09-17T07:30:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455083+0000", 2024-09-17T07:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044629199999999999 2024-09-17T07:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455098+0000", 2024-09-17T07:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4183e-05 2024-09-17T07:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455106+0000", 2024-09-17T07:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455173+0000", 2024-09-17T07:30:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6346000000000005e-05 2024-09-17T07:30:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455200+0000", 2024-09-17T07:30:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7319000000000001e-05 2024-09-17T07:30:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457774+0000", 2024-09-17T07:30:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025742650000000001 2024-09-17T07:30:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457818+0000", 2024-09-17T07:30:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3891999999999999e-05 2024-09-17T07:30:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33735 3.1f e55/52 3:fea54097:::rbd_data.11839a97b753.0000000000000000:head v 55'260, mlcod=55'258)", 2024-09-17T07:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 68.704289341000006, 2024-09-17T07:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035215200000000002, 2024-09-17T07:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824571+0000", 2024-09-17T07:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.164e-06 2024-09-17T07:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825118+0000", 2024-09-17T07:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054696400000000004 2024-09-17T07:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825134+0000", 2024-09-17T07:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6342999999999999e-05 2024-09-17T07:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825142+0000", 2024-09-17T07:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5270000000000001e-06 2024-09-17T07:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825200+0000", 2024-09-17T07:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.893e-05 2024-09-17T07:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825218+0000", 2024-09-17T07:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7935999999999999e-05 2024-09-17T07:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828037+0000", 2024-09-17T07:30:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002818961 2024-09-17T07:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828091+0000", 2024-09-17T07:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3695e-05 2024-09-17T07:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33869 3.9 e55/23 3:907b81b5:::rbd_data.11839a97b753.0000000000000001:head v 55'171, mlcod=55'169)", 2024-09-17T07:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 68.687233517999999, 2024-09-17T07:30:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033990859999999999, 2024-09-17T07:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841626+0000", 2024-09-17T07:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.023e-06 2024-09-17T07:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842067+0000", 2024-09-17T07:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000440612 2024-09-17T07:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842080+0000", 2024-09-17T07:30:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.29e-05 2024-09-17T07:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842086+0000", 2024-09-17T07:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2090000000000002e-06 2024-09-17T07:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842144+0000", 2024-09-17T07:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7627999999999998e-05 2024-09-17T07:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842171+0000", 2024-09-17T07:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7214000000000001e-05 2024-09-17T07:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.844973+0000", 2024-09-17T07:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002802232 2024-09-17T07:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.845024+0000", 2024-09-17T07:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1267999999999998e-05 2024-09-17T07:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51432 3.0 e59/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 59'177, mlcod=58'174)", 2024-09-17T07:30:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 63.707636501000003, 2024-09-17T07:30:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010790086000000001, 2024-09-17T07:30:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821224+0000", 2024-09-17T07:30:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.511e-06 2024-09-17T07:30:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825174+0000", 2024-09-17T07:30:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0039497949999999999 2024-09-17T07:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825196+0000", 2024-09-17T07:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2643999999999999e-05 2024-09-17T07:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825206+0000", 2024-09-17T07:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4320000000000005e-06 2024-09-17T07:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825272+0000", 2024-09-17T07:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5829999999999998e-05 2024-09-17T07:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825299+0000", 2024-09-17T07:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7107e-05 2024-09-17T07:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.831939+0000", 2024-09-17T07:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0066398510000000004 2024-09-17T07:30:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.832012+0000", 2024-09-17T07:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3916000000000002e-05 2024-09-17T07:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51969 3.0 e59/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 59'179, mlcod=59'177)", 2024-09-17T07:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 63.598768122999999, 2024-09-17T07:30:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060297249999999997, 2024-09-17T07:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930093+0000", 2024-09-17T07:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.764e-06 2024-09-17T07:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931779+0000", 2024-09-17T07:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001686379 2024-09-17T07:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931802+0000", 2024-09-17T07:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2682000000000001e-05 2024-09-17T07:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931811+0000", 2024-09-17T07:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5640000000000002e-06 2024-09-17T07:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931877+0000", 2024-09-17T07:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6155999999999995e-05 2024-09-17T07:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931896+0000", 2024-09-17T07:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9111e-05 2024-09-17T07:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936063+0000", 2024-09-17T07:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004166711 2024-09-17T07:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936121+0000", 2024-09-17T07:30:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7358000000000002e-05 2024-09-17T07:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54417 3.0 e60/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 60'182, mlcod=59'179)", 2024-09-17T07:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 63.091251131999996, 2024-09-17T07:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0059885219999999996, 2024-09-17T07:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437609+0000", 2024-09-17T07:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5829999999999999e-06 2024-09-17T07:30:26.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437623+0000", 2024-09-17T07:30:26.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3348e-05 2024-09-17T07:30:26.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437643+0000", 2024-09-17T07:30:26.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0463999999999999e-05 2024-09-17T07:30:26.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437653+0000", 2024-09-17T07:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6290000000000005e-06 2024-09-17T07:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437719+0000", 2024-09-17T07:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6428999999999997e-05 2024-09-17T07:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437745+0000", 2024-09-17T07:30:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5690999999999998e-05 2024-09-17T07:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443549+0000", 2024-09-17T07:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0058038259999999998 2024-09-17T07:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443596+0000", 2024-09-17T07:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7552000000000001e-05 2024-09-17T07:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54419 3.0 e60/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 60'184, mlcod=59'179)", 2024-09-17T07:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 63.084898774999999, 2024-09-17T07:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004706844, 2024-09-17T07:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443961+0000", 2024-09-17T07:30:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:30:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443967+0000", 2024-09-17T07:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.268e-06 2024-09-17T07:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443983+0000", 2024-09-17T07:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5248999999999999e-05 2024-09-17T07:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443990+0000", 2024-09-17T07:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5120000000000002e-06 2024-09-17T07:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444056+0000", 2024-09-17T07:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6197000000000003e-05 2024-09-17T07:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444081+0000", 2024-09-17T07:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4791e-05 2024-09-17T07:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448616+0000", 2024-09-17T07:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045346129999999998 2024-09-17T07:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448667+0000", 2024-09-17T07:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-17T07:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:80794 3.9 e72/23 3:90311a61:::rbd_data.1183cb746bb8.0000000000000007:head v 72'222, mlcod=71'220)", 2024-09-17T07:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 49.096909891000003, 2024-09-17T07:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004371733, 2024-09-17T07:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0800000000000001e-07 2024-09-17T07:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434108+0000", 2024-09-17T07:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021587569999999999 2024-09-17T07:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434120+0000", 2024-09-17T07:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1821000000000001e-05 2024-09-17T07:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434128+0000", 2024-09-17T07:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7579999999999992e-06 2024-09-17T07:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434191+0000", 2024-09-17T07:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3202999999999998e-05 2024-09-17T07:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434218+0000", 2024-09-17T07:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6771000000000002e-05 2024-09-17T07:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436289+0000", 2024-09-17T07:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020717700000000001 2024-09-17T07:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436321+0000", 2024-09-17T07:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1445000000000001e-05 2024-09-17T07:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 75/74 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head, version: 36'126, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head@36'126, size: 9431460, copy_subset: [], clone_subset: {3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9: [(0, 9431460)]}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 44.788002298000002, 2024-09-17T07:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041288289999999997, 2024-09-17T07:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740858+0000", 2024-09-17T07:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1599999999999999e-06 2024-09-17T07:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740885+0000", 2024-09-17T07:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.707e-05 2024-09-17T07:30:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740925+0000", 2024-09-17T07:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0649000000000003e-05 2024-09-17T07:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740957+0000", 2024-09-17T07:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1585999999999999e-05 2024-09-17T07:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741185+0000", 2024-09-17T07:30:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022759700000000001 2024-09-17T07:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741228+0000", 2024-09-17T07:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3705999999999998e-05 2024-09-17T07:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.744985+0000", 2024-09-17T07:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037570609999999999 2024-09-17T07:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85061 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'257, mlcod=79'255)", 2024-09-17T07:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.380171341999997, 2024-09-17T07:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033850550000000001, 2024-09-17T07:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9699999999999995e-07 2024-09-17T07:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148835+0000", 2024-09-17T07:30:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000146852 2024-09-17T07:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148845+0000", 2024-09-17T07:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0232e-05 2024-09-17T07:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148852+0000", 2024-09-17T07:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6490000000000002e-06 2024-09-17T07:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148913+0000", 2024-09-17T07:30:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1099999999999994e-05 2024-09-17T07:30:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148931+0000", 2024-09-17T07:30:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7938e-05 2024-09-17T07:30:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152025+0000", 2024-09-17T07:30:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0030942729999999998 2024-09-17T07:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152073+0000", 2024-09-17T07:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7213999999999999e-05 2024-09-17T07:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85063 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'259, mlcod=79'255)", 2024-09-17T07:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.379024387999998, 2024-09-17T07:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037722659999999998, 2024-09-17T07:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5799999999999999e-07 2024-09-17T07:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149841+0000", 2024-09-17T07:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4840000000000003e-06 2024-09-17T07:30:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149849+0000", 2024-09-17T07:30:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4670000000000004e-06 2024-09-17T07:30:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149854+0000", 2024-09-17T07:30:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4779999999999999e-06 2024-09-17T07:30:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149920+0000", 2024-09-17T07:30:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5760999999999996e-05 2024-09-17T07:30:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149949+0000", 2024-09-17T07:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8838000000000001e-05 2024-09-17T07:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153558+0000", 2024-09-17T07:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036092939999999999 2024-09-17T07:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153607+0000", 2024-09-17T07:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8386000000000001e-05 2024-09-17T07:30:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:94996 3.0 e82/23 3:074b4f56:::rbd_data.118311e0d6f8.0000000000000001:head v 82'715, mlcod=80'713)", 2024-09-17T07:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 36.913118795000003, 2024-09-17T07:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031499179999999998, 2024-09-17T07:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615741+0000", 2024-09-17T07:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999996e-07 2024-09-17T07:30:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616917+0000", 2024-09-17T07:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011763310000000001 2024-09-17T07:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616936+0000", 2024-09-17T07:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8624e-05 2024-09-17T07:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616948+0000", 2024-09-17T07:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.238e-05 2024-09-17T07:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617054+0000", 2024-09-17T07:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000106556 2024-09-17T07:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617081+0000", 2024-09-17T07:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.622e-05 2024-09-17T07:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618836+0000", 2024-09-17T07:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001755412 2024-09-17T07:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618890+0000", 2024-09-17T07:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3915000000000001e-05 2024-09-17T07:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:95007 3.5 e82/76 3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head v 82'422, mlcod=82'420)", 2024-09-17T07:30:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 36.905196093999997, 2024-09-17T07:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003939901, 2024-09-17T07:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623664+0000", 2024-09-17T07:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3799999999999999e-06 2024-09-17T07:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626104+0000", 2024-09-17T07:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002440149 2024-09-17T07:30:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626120+0000", 2024-09-17T07:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5962000000000001e-05 2024-09-17T07:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626127+0000", 2024-09-17T07:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8340000000000001e-06 2024-09-17T07:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626186+0000", 2024-09-17T07:30:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9234999999999998e-05 2024-09-17T07:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626208+0000", 2024-09-17T07:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1719e-05 2024-09-17T07:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627567+0000", 2024-09-17T07:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013587919999999999 2024-09-17T07:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627603+0000", 2024-09-17T07:30:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5830000000000001e-05 2024-09-17T07:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:104928 3.18 e87/76 3:1ef990b1:::rbd_data.1183dd51323.0000000000000006:head v 87'562, mlcod=86'559)", 2024-09-17T07:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 30.986440383000001, 2024-09-17T07:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0061095869999999997, 2024-09-17T07:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542421+0000", 2024-09-17T07:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.069e-06 2024-09-17T07:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544190+0000", 2024-09-17T07:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017693539999999999 2024-09-17T07:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544215+0000", 2024-09-17T07:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5398999999999999e-05 2024-09-17T07:30:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544229+0000", 2024-09-17T07:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3865e-05 2024-09-17T07:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544322+0000", 2024-09-17T07:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2540000000000005e-05 2024-09-17T07:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544348+0000", 2024-09-17T07:30:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6197999999999999e-05 2024-09-17T07:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548475+0000", 2024-09-17T07:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041266000000000002 2024-09-17T07:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548528+0000", 2024-09-17T07:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3562e-05 2024-09-17T07:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115898 3.9 3:910101b0:::rbd_data.118340e4fbf.0000000000000000:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.3379262780000003, 2024-09-17T07:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0042466279999999997, 2024-09-17T07:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115898 2024-09-17T07:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0030000000000001e-06 2024-09-17T07:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190940+0000", 2024-09-17T07:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2349999999999999e-06 2024-09-17T07:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190953+0000", 2024-09-17T07:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3304e-05 2024-09-17T07:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190995+0000", 2024-09-17T07:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2317000000000002e-05 2024-09-17T07:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191592+0000", 2024-09-17T07:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00059607799999999999 2024-09-17T07:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191888+0000", 2024-09-17T07:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00029619999999999999 2024-09-17T07:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192069+0000", 2024-09-17T07:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00018125899999999999 2024-09-17T07:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193692+0000", 2024-09-17T07:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001623364 2024-09-17T07:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194968+0000", 2024-09-17T07:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012755189999999999 2024-09-17T07:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.195037+0000", 2024-09-17T07:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9450999999999996e-05 2024-09-17T07:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.195179+0000", 2024-09-17T07:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000141898 2024-09-17T07:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115936 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [watch watch cookie 140227662410368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.3050073260000001, 2024-09-17T07:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043930569999999997, 2024-09-17T07:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115936 2024-09-17T07:30:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7300000000000005e-07 2024-09-17T07:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223856+0000", 2024-09-17T07:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9169999999999999e-06 2024-09-17T07:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223870+0000", 2024-09-17T07:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3926e-05 2024-09-17T07:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223878+0000", 2024-09-17T07:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3699999999999997e-06 2024-09-17T07:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223934+0000", 2024-09-17T07:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6929000000000003e-05 2024-09-17T07:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224053+0000", 2024-09-17T07:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000118952 2024-09-17T07:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224271+0000", 2024-09-17T07:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021763300000000001 2024-09-17T07:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224992+0000", 2024-09-17T07:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072142099999999998 2024-09-17T07:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228099+0000", 2024-09-17T07:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031060850000000002 2024-09-17T07:30:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228143+0000", 2024-09-17T07:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4614999999999999e-05 2024-09-17T07:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228245+0000", 2024-09-17T07:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000101436 2024-09-17T07:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115955 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [call rbd.snapshot_trash_add in=21b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.2860205730000001, 2024-09-17T07:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004259054, 2024-09-17T07:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115955 2024-09-17T07:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:30:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242839+0000", 2024-09-17T07:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6899999999999998e-07 2024-09-17T07:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242842+0000", 2024-09-17T07:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1350000000000001e-06 2024-09-17T07:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242849+0000", 2024-09-17T07:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0400000000000004e-06 2024-09-17T07:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242854+0000", 2024-09-17T07:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4940000000000003e-06 2024-09-17T07:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243137+0000", 2024-09-17T07:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00028217100000000001 2024-09-17T07:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243196+0000", 2024-09-17T07:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9608999999999998e-05 2024-09-17T07:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243448+0000", 2024-09-17T07:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00025185000000000002 2024-09-17T07:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243990+0000", 2024-09-17T07:30:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054153199999999995 2024-09-17T07:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.246867+0000", 2024-09-17T07:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028771880000000001 2024-09-17T07:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.246943+0000", 2024-09-17T07:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5689e-05 2024-09-17T07:30:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247097+0000", 2024-09-17T07:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015487699999999999 2024-09-17T07:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115956 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.2857239890000001, 2024-09-17T07:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043741479999999996, 2024-09-17T07:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115956 2024-09-17T07:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243136+0000", 2024-09-17T07:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7100000000000001e-07 2024-09-17T07:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243141+0000", 2024-09-17T07:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.716e-06 2024-09-17T07:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243153+0000", 2024-09-17T07:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1198e-05 2024-09-17T07:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243160+0000", 2024-09-17T07:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6229999999999997e-06 2024-09-17T07:30:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243797+0000", 2024-09-17T07:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063716399999999996 2024-09-17T07:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for rw locks", 2024-09-17T07:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243852+0000", 2024-09-17T07:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4147999999999997e-05 2024-09-17T07:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247147+0000", 2024-09-17T07:30:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0032955940000000002 2024-09-17T07:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247271+0000", 2024-09-17T07:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012397800000000001 2024-09-17T07:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247509+0000", 2024-09-17T07:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000237956 2024-09-17T07:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:26.633 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:26.634 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:26.635 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:26.633+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:26.735 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:26.735+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:26.751 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:26.751 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:26.751 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:26.751 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:26.760 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:26.836 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:26.836+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:26.904 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:26.904 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:26.904 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:26.912 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:26.957 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:26.956+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:27.017 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:27.017 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:30525 3.1b 3:def7c8f0:::rbd_data.118381ffbbab.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T07:30:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.493588142999997, 2024-09-17T07:30:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55515850300000003, 2024-09-17T07:30:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.018 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30525 2024-09-17T07:30:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522300+0000", 2024-09-17T07:30:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:30:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522305+0000", 2024-09-17T07:30:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-17T07:30:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522318+0000", 2024-09-17T07:30:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3030999999999999e-05 2024-09-17T07:30:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522326+0000", 2024-09-17T07:30:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4200000000000001e-06 2024-09-17T07:30:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522385+0000", 2024-09-17T07:30:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9005000000000002e-05 2024-09-17T07:30:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522393+0000", 2024-09-17T07:30:27.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-06 2024-09-17T07:30:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075347+0000", 2024-09-17T07:30:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55295417800000002 2024-09-17T07:30:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075502+0000", 2024-09-17T07:30:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154272 2024-09-17T07:30:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075630+0000", 2024-09-17T07:30:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012816300000000001 2024-09-17T07:30:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077221+0000", 2024-09-17T07:30:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015912929999999999 2024-09-17T07:30:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077353+0000", 2024-09-17T07:30:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013215699999999999 2024-09-17T07:30:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077413+0000", 2024-09-17T07:30:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0034999999999997e-05 2024-09-17T07:30:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077457+0000", 2024-09-17T07:30:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4017999999999998e-05 2024-09-17T07:30:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44181 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13922304~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.378409495, 2024-09-17T07:30:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046010502000000002, 2024-09-17T07:30:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44181 2024-09-17T07:30:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.025 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637478+0000", 2024-09-17T07:30:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:30:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637480+0000", 2024-09-17T07:30:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.843e-06 2024-09-17T07:30:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637487+0000", 2024-09-17T07:30:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6159999999999996e-06 2024-09-17T07:30:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637492+0000", 2024-09-17T07:30:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5360000000000003e-06 2024-09-17T07:30:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683351+0000", 2024-09-17T07:30:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045858758999999999 2024-09-17T07:30:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683422+0000", 2024-09-17T07:30:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1705999999999994e-05 2024-09-17T07:30:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683488+0000", 2024-09-17T07:30:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5612999999999997e-05 2024-09-17T07:30:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44182 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13938688~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.378388823999998, 2024-09-17T07:30:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046153118999999999, 2024-09-17T07:30:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.029 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44182 2024-09-17T07:30:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:30:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637506+0000", 2024-09-17T07:30:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1520000000000003e-06 2024-09-17T07:30:27.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637512+0000", 2024-09-17T07:30:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-09-17T07:30:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637517+0000", 2024-09-17T07:30:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-09-17T07:30:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683514+0000", 2024-09-17T07:30:27.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045997616999999998 2024-09-17T07:30:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683584+0000", 2024-09-17T07:30:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9942000000000002e-05 2024-09-17T07:30:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683651+0000", 2024-09-17T07:30:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6951000000000003e-05 2024-09-17T07:30:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44183 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13955072~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.378363591999999, 2024-09-17T07:30:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046339793999999997, 2024-09-17T07:30:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44183 2024-09-17T07:30:27.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639583+0000", 2024-09-17T07:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002059412 2024-09-17T07:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639597+0000", 2024-09-17T07:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3832000000000001e-05 2024-09-17T07:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639607+0000", 2024-09-17T07:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485e-05 2024-09-17T07:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639614+0000", 2024-09-17T07:30:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0849999999999999e-06 2024-09-17T07:30:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683705+0000", 2024-09-17T07:30:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044091261 2024-09-17T07:30:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683792+0000", 2024-09-17T07:30:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6197000000000002e-05 2024-09-17T07:30:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683863+0000", 2024-09-17T07:30:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1521999999999997e-05 2024-09-17T07:30:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44199 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14217216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.374846039999994, 2024-09-17T07:30:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045995623999999999, 2024-09-17T07:30:27.038 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44199 2024-09-17T07:30:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:30:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641043+0000", 2024-09-17T07:30:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.624e-06 2024-09-17T07:30:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641048+0000", 2024-09-17T07:30:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4029999999999998e-06 2024-09-17T07:30:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641053+0000", 2024-09-17T07:30:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6670000000000004e-06 2024-09-17T07:30:27.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686918+0000", 2024-09-17T07:30:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045865131000000003 2024-09-17T07:30:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686986+0000", 2024-09-17T07:30:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8299000000000005e-05 2024-09-17T07:30:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687036+0000", 2024-09-17T07:30:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0164999999999998e-05 2024-09-17T07:30:27.042 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44200 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14233600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.374828210999993, 2024-09-17T07:30:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046183984999999997, 2024-09-17T07:30:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44200 2024-09-17T07:30:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641061+0000", 2024-09-17T07:30:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.767e-06 2024-09-17T07:30:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641067+0000", 2024-09-17T07:30:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7379999999999999e-06 2024-09-17T07:30:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641071+0000", 2024-09-17T07:30:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-09-17T07:30:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687078+0000", 2024-09-17T07:30:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046007095999999997 2024-09-17T07:30:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687158+0000", 2024-09-17T07:30:27.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0374e-05 2024-09-17T07:30:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687243+0000", 2024-09-17T07:30:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4221999999999994e-05 2024-09-17T07:30:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44201 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14249984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:27.047 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.374810565000004, 2024-09-17T07:30:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046303182999999998, 2024-09-17T07:30:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44201 2024-09-17T07:30:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:27.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641120+0000", 2024-09-17T07:30:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3933000000000001e-05 2024-09-17T07:30:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641123+0000", 2024-09-17T07:30:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.191e-06 2024-09-17T07:30:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641133+0000", 2024-09-17T07:30:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4020000000000008e-06 2024-09-17T07:30:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641138+0000", 2024-09-17T07:30:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6559999999999997e-06 2024-09-17T07:30:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687279+0000", 2024-09-17T07:30:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046140278999999999 2024-09-17T07:30:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687338+0000", 2024-09-17T07:30:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9268999999999998e-05 2024-09-17T07:30:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687379+0000", 2024-09-17T07:30:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1452999999999998e-05 2024-09-17T07:30:27.051 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44202 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14266368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.374739798999997, 2024-09-17T07:30:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046484299, 2024-09-17T07:30:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44202 2024-09-17T07:30:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642810+0000", 2024-09-17T07:30:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016627980000000001 2024-09-17T07:30:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642815+0000", 2024-09-17T07:30:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.091e-06 2024-09-17T07:30:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642829+0000", 2024-09-17T07:30:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4047e-05 2024-09-17T07:30:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642844+0000", 2024-09-17T07:30:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4515000000000001e-05 2024-09-17T07:30:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687426+0000", 2024-09-17T07:30:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044582201000000002 2024-09-17T07:30:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687529+0000", 2024-09-17T07:30:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103078 2024-09-17T07:30:27.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687631+0000", 2024-09-17T07:30:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102569 2024-09-17T07:30:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44209 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14381056~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.372841647000001, 2024-09-17T07:30:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045944444000000001, 2024-09-17T07:30:27.056 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44209 2024-09-17T07:30:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643046+0000", 2024-09-17T07:30:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:30:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643048+0000", 2024-09-17T07:30:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:30:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643054+0000", 2024-09-17T07:30:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.032e-06 2024-09-17T07:30:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643065+0000", 2024-09-17T07:30:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1861999999999999e-05 2024-09-17T07:30:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688806+0000", 2024-09-17T07:30:27.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045740617999999997 2024-09-17T07:30:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688927+0000", 2024-09-17T07:30:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000121327 2024-09-17T07:30:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688990+0000", 2024-09-17T07:30:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2293999999999997e-05 2024-09-17T07:30:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44210 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14397440~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.372814720999997, 2024-09-17T07:30:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046104635999999997, 2024-09-17T07:30:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44210 2024-09-17T07:30:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.061 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643073+0000", 2024-09-17T07:30:27.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.77e-07 2024-09-17T07:30:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643075+0000", 2024-09-17T07:30:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-09-17T07:30:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643082+0000", 2024-09-17T07:30:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7789999999999996e-06 2024-09-17T07:30:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643087+0000", 2024-09-17T07:30:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.967e-06 2024-09-17T07:30:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689022+0000", 2024-09-17T07:30:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045934349999999999 2024-09-17T07:30:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689112+0000", 2024-09-17T07:30:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0039999999999999e-05 2024-09-17T07:30:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689177+0000", 2024-09-17T07:30:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5041000000000003e-05 2024-09-17T07:30:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44211 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14413824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.372793587999993, 2024-09-17T07:30:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046309003000000001, 2024-09-17T07:30:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.065 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44211 2024-09-17T07:30:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:27.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643094+0000", 2024-09-17T07:30:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:30:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643095+0000", 2024-09-17T07:30:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-17T07:30:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643101+0000", 2024-09-17T07:30:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8730000000000004e-06 2024-09-17T07:30:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643106+0000", 2024-09-17T07:30:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6859999999999997e-06 2024-09-17T07:30:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689220+0000", 2024-09-17T07:30:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046113945000000003 2024-09-17T07:30:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689309+0000", 2024-09-17T07:30:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9560000000000003e-05 2024-09-17T07:30:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689402+0000", 2024-09-17T07:30:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2786000000000002e-05 2024-09-17T07:30:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44212 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14430208~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:27.069 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.372749025999994, 2024-09-17T07:30:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046448912000000002, 2024-09-17T07:30:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44212 2024-09-17T07:30:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2200000000000004e-07 2024-09-17T07:30:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643141+0000", 2024-09-17T07:30:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5210000000000001e-06 2024-09-17T07:30:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643149+0000", 2024-09-17T07:30:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6490000000000008e-06 2024-09-17T07:30:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643161+0000", 2024-09-17T07:30:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2241e-05 2024-09-17T07:30:27.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689423+0000", 2024-09-17T07:30:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046262323000000001 2024-09-17T07:30:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689505+0000", 2024-09-17T07:30:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2103999999999999e-05 2024-09-17T07:30:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689587+0000", 2024-09-17T07:30:27.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1452000000000006e-05 2024-09-17T07:30:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44213 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14446592~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.372717527000006, 2024-09-17T07:30:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046601780000000002, 2024-09-17T07:30:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.074 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44213 2024-09-17T07:30:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643170+0000", 2024-09-17T07:30:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-17T07:30:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643172+0000", 2024-09-17T07:30:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1660000000000001e-06 2024-09-17T07:30:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643180+0000", 2024-09-17T07:30:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-09-17T07:30:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643185+0000", 2024-09-17T07:30:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-09-17T07:30:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689622+0000", 2024-09-17T07:30:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046436742000000003 2024-09-17T07:30:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689697+0000", 2024-09-17T07:30:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5179999999999995e-05 2024-09-17T07:30:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689771+0000", 2024-09-17T07:30:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3928000000000001e-05 2024-09-17T07:30:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44214 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14462976~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.078 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.372694467000002, 2024-09-17T07:30:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046736038000000001, 2024-09-17T07:30:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44214 2024-09-17T07:30:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:27.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643193+0000", 2024-09-17T07:30:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:30:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643195+0000", 2024-09-17T07:30:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0640000000000001e-06 2024-09-17T07:30:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643201+0000", 2024-09-17T07:30:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2859999999999996e-06 2024-09-17T07:30:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643206+0000", 2024-09-17T07:30:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-17T07:30:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689814+0000", 2024-09-17T07:30:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046607742000000001 2024-09-17T07:30:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689886+0000", 2024-09-17T07:30:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2020999999999994e-05 2024-09-17T07:30:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689928+0000", 2024-09-17T07:30:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2771000000000001e-05 2024-09-17T07:30:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.082 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44215 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14479360~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.372650570000005, 2024-09-17T07:30:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046868543999999998, 2024-09-17T07:30:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.083 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44215 2024-09-17T07:30:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643237+0000", 2024-09-17T07:30:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:30:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643245+0000", 2024-09-17T07:30:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3270000000000001e-06 2024-09-17T07:30:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643254+0000", 2024-09-17T07:30:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6049999999999998e-06 2024-09-17T07:30:27.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643260+0000", 2024-09-17T07:30:27.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7849999999999997e-06 2024-09-17T07:30:27.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689961+0000", 2024-09-17T07:30:27.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046701196 2024-09-17T07:30:27.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690036+0000", 2024-09-17T07:30:27.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5153000000000004e-05 2024-09-17T07:30:27.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690105+0000", 2024-09-17T07:30:27.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8874000000000005e-05 2024-09-17T07:30:27.087 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.087 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.087 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.087 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44216 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14495744~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.087 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:27.087 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.372618857000006, 2024-09-17T07:30:27.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047003047999999999, 2024-09-17T07:30:27.087 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.088 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.088 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.088 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44216 2024-09-17T07:30:27.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.088 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:27.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:27.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643269+0000", 2024-09-17T07:30:27.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:30:27.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643271+0000", 2024-09-17T07:30:27.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-09-17T07:30:27.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643278+0000", 2024-09-17T07:30:27.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8850000000000001e-06 2024-09-17T07:30:27.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643282+0000", 2024-09-17T07:30:27.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5979999999999999e-06 2024-09-17T07:30:27.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690133+0000", 2024-09-17T07:30:27.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850470999999998 2024-09-17T07:30:27.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690205+0000", 2024-09-17T07:30:27.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2442999999999997e-05 2024-09-17T07:30:27.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690271+0000", 2024-09-17T07:30:27.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5874e-05 2024-09-17T07:30:27.091 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.091 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.091 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.092 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44217 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14512128~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.092 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:27.092 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.372597845000001, 2024-09-17T07:30:27.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047166485000000001, 2024-09-17T07:30:27.092 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.092 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.092 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.092 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44217 2024-09-17T07:30:27.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.093 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:27.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:27.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:27.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-17T07:30:27.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643291+0000", 2024-09-17T07:30:27.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.962e-06 2024-09-17T07:30:27.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643297+0000", 2024-09-17T07:30:27.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0700000000000003e-06 2024-09-17T07:30:27.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643302+0000", 2024-09-17T07:30:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:30:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690308+0000", 2024-09-17T07:30:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047005354999999999 2024-09-17T07:30:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690385+0000", 2024-09-17T07:30:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7541999999999997e-05 2024-09-17T07:30:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690455+0000", 2024-09-17T07:30:27.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0328999999999997e-05 2024-09-17T07:30:27.096 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.096 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.096 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.096 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44218 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14528512~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.096 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:27.096 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.372578168000004, 2024-09-17T07:30:27.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047321015000000001, 2024-09-17T07:30:27.096 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.096 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.096 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.097 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44218 2024-09-17T07:30:27.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.097 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:27.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:27.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:27.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-17T07:30:27.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643311+0000", 2024-09-17T07:30:27.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.933e-06 2024-09-17T07:30:27.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643318+0000", 2024-09-17T07:30:27.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3980000000000004e-06 2024-09-17T07:30:27.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643322+0000", 2024-09-17T07:30:27.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7609999999999998e-06 2024-09-17T07:30:27.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690484+0000", 2024-09-17T07:30:27.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047161428999999998 2024-09-17T07:30:27.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690564+0000", 2024-09-17T07:30:27.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9845000000000005e-05 2024-09-17T07:30:27.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690630+0000", 2024-09-17T07:30:27.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6021999999999999e-05 2024-09-17T07:30:27.100 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.100 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.100 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.101 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44219 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14544896~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.101 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:27.101 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.372557800999999, 2024-09-17T07:30:27.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048128708999999999, 2024-09-17T07:30:27.101 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.101 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.101 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.101 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44219 2024-09-17T07:30:27.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.101 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:27.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:27.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:27.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:30:27.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643337+0000", 2024-09-17T07:30:27.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.109e-06 2024-09-17T07:30:27.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643350+0000", 2024-09-17T07:30:27.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3046e-05 2024-09-17T07:30:27.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643356+0000", 2024-09-17T07:30:27.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8930000000000002e-06 2024-09-17T07:30:27.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690660+0000", 2024-09-17T07:30:27.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047304061000000001 2024-09-17T07:30:27.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691372+0000", 2024-09-17T07:30:27.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00071281299999999997 2024-09-17T07:30:27.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691458+0000", 2024-09-17T07:30:27.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5352000000000005e-05 2024-09-17T07:30:27.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.105 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.105 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44220 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14561280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:27.105 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:27.105 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.372523752999996, 2024-09-17T07:30:27.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048309849000000002, 2024-09-17T07:30:27.105 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.105 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.106 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.106 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44220 2024-09-17T07:30:27.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.106 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:27.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:27.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646869+0000", 2024-09-17T07:30:27.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003506267 2024-09-17T07:30:27.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646874+0000", 2024-09-17T07:30:27.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7500000000000003e-06 2024-09-17T07:30:27.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646887+0000", 2024-09-17T07:30:27.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3115999999999999e-05 2024-09-17T07:30:27.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646894+0000", 2024-09-17T07:30:27.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5289999999999997e-06 2024-09-17T07:30:27.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691547+0000", 2024-09-17T07:30:27.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044652743000000002 2024-09-17T07:30:27.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691622+0000", 2024-09-17T07:30:27.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5037999999999995e-05 2024-09-17T07:30:27.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691673+0000", 2024-09-17T07:30:27.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-17T07:30:27.109 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.109 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.109 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.109 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.110 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.110 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:27.111 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:27.112 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:27.110+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:27.212 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:27.211+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:27.256 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T07:30:27.260 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:27.260 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:27.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.260 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:116163 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-17T07:30:27.260 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:30:27.260 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.99347594800000005, 2024-09-17T07:30:27.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.99349648499999998, 2024-09-17T07:30:27.260 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.260 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.260 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "waiting for peered", 2024-09-17T07:30:27.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.261 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.261 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.261 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116163 2024-09-17T07:30:27.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.261 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:30:27.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:30:27.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265157+0000", 2024-09-17T07:30:27.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-06 2024-09-17T07:30:27.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265168+0000", 2024-09-17T07:30:27.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0827e-05 2024-09-17T07:30:27.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265198+0000", 2024-09-17T07:30:27.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0383e-05 2024-09-17T07:30:27.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265212+0000", 2024-09-17T07:30:27.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3799e-05 2024-09-17T07:30:27.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265338+0000", 2024-09-17T07:30:27.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012576800000000001 2024-09-17T07:30:27.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:27.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265357+0000", 2024-09-17T07:30:27.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9417000000000001e-05 2024-09-17T07:30:27.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:27.083102+0000", 2024-09-17T07:30:27.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.81774456299999998 2024-09-17T07:30:27.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:27.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:27.083113+0000", 2024-09-17T07:30:27.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1154e-05 2024-09-17T07:30:27.265 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.265 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.265 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.265 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.265 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-09-17T07:30:27.265 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 1 2024-09-17T07:30:27.265 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:27.267 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:27.312 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:27.312+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:27.370 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:27.370 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:27.370 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:27.370 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:27.370 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:27.378 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:27.413 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:27.413+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:27.514 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:27.513+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:27.522 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:27.522 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:27.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:27.522 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:27.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.523 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19985 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5275648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.523 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:27.523 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.268915391999997, 2024-09-17T07:30:27.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071819998999999995, 2024-09-17T07:30:27.523 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.523 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.523 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.524 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19985 2024-09-17T07:30:27.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.524 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:27.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:27.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:27.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:27.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252493+0000", 2024-09-17T07:30:27.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6300000000000004e-07 2024-09-17T07:30:27.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252496+0000", 2024-09-17T07:30:27.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-17T07:30:27.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252499+0000", 2024-09-17T07:30:27.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:30:27.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324036+0000", 2024-09-17T07:30:27.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071536587999999998 2024-09-17T07:30:27.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324118+0000", 2024-09-17T07:30:27.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2336999999999995e-05 2024-09-17T07:30:27.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324312+0000", 2024-09-17T07:30:27.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000193709 2024-09-17T07:30:27.527 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.528 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.528 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.528 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19986 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5505024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.528 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:27.528 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.268904239999998, 2024-09-17T07:30:27.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072083761999999996, 2024-09-17T07:30:27.528 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.528 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.528 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.529 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19986 2024-09-17T07:30:27.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.529 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:27.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:27.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:27.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:30:27.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252508+0000", 2024-09-17T07:30:27.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:30:27.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252511+0000", 2024-09-17T07:30:27.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.145e-06 2024-09-17T07:30:27.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252514+0000", 2024-09-17T07:30:27.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5569999999999998e-06 2024-09-17T07:30:27.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324326+0000", 2024-09-17T07:30:27.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071812681000000003 2024-09-17T07:30:27.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324372+0000", 2024-09-17T07:30:27.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5893000000000003e-05 2024-09-17T07:30:27.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324587+0000", 2024-09-17T07:30:27.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214608 2024-09-17T07:30:27.532 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.532 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.532 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.532 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19987 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5734400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.532 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:27.532 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.268890622000001, 2024-09-17T07:30:27.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072276676999999998, 2024-09-17T07:30:27.533 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.533 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.533 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.533 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19987 2024-09-17T07:30:27.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.533 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:27.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:27.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:27.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:27.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252518+0000", 2024-09-17T07:30:27.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-17T07:30:27.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252522+0000", 2024-09-17T07:30:27.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2440000000000004e-06 2024-09-17T07:30:27.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252525+0000", 2024-09-17T07:30:27.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5919999999999999e-06 2024-09-17T07:30:27.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324621+0000", 2024-09-17T07:30:27.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072096442999999996 2024-09-17T07:30:27.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324699+0000", 2024-09-17T07:30:27.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8061999999999999e-05 2024-09-17T07:30:27.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324793+0000", 2024-09-17T07:30:27.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4207000000000001e-05 2024-09-17T07:30:27.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.536 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.537 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19988 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.537 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:27.537 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.268879651000006, 2024-09-17T07:30:27.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072585798000000007, 2024-09-17T07:30:27.537 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.537 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.537 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.537 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19988 2024-09-17T07:30:27.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:27.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252529+0000", 2024-09-17T07:30:27.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:30:27.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252532+0000", 2024-09-17T07:30:27.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9579999999999999e-06 2024-09-17T07:30:27.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252534+0000", 2024-09-17T07:30:27.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5730000000000002e-06 2024-09-17T07:30:27.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324825+0000", 2024-09-17T07:30:27.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072290757999999997 2024-09-17T07:30:27.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324874+0000", 2024-09-17T07:30:27.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9360000000000002e-05 2024-09-17T07:30:27.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325113+0000", 2024-09-17T07:30:27.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000239025 2024-09-17T07:30:27.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.541 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.541 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19989 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.541 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:27.541 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.268870030000002, 2024-09-17T07:30:27.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072963494000000004, 2024-09-17T07:30:27.541 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.541 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.542 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.542 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19989 2024-09-17T07:30:27.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.542 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:27.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:27.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:30:27.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:30:27.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:30:27.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2900000000000002e-07 2024-09-17T07:30:27.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252542+0000", 2024-09-17T07:30:27.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8600000000000003e-06 2024-09-17T07:30:27.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252545+0000", 2024-09-17T07:30:27.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.605e-06 2024-09-17T07:30:27.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325144+0000", 2024-09-17T07:30:27.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072599460000000005 2024-09-17T07:30:27.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325223+0000", 2024-09-17T07:30:27.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8473999999999994e-05 2024-09-17T07:30:27.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325501+0000", 2024-09-17T07:30:27.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000278063 2024-09-17T07:30:27.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.545 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.546 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19990 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6422528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.546 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:27.546 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.268859559000006, 2024-09-17T07:30:27.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073241127000000003, 2024-09-17T07:30:27.546 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.546 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.546 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.546 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19990 2024-09-17T07:30:27.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.546 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:27.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:27.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:27.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:27.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252552+0000", 2024-09-17T07:30:27.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4719999999999997e-06 2024-09-17T07:30:27.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252557+0000", 2024-09-17T07:30:27.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3039999999999998e-06 2024-09-17T07:30:27.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252560+0000", 2024-09-17T07:30:27.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8830000000000002e-06 2024-09-17T07:30:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325529+0000", 2024-09-17T07:30:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072968880999999999 2024-09-17T07:30:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325574+0000", 2024-09-17T07:30:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466000000000001e-05 2024-09-17T07:30:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325789+0000", 2024-09-17T07:30:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021490999999999999 2024-09-17T07:30:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19991 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6651904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.268840256999994, 2024-09-17T07:30:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073720552999999994, 2024-09-17T07:30:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.550 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19991 2024-09-17T07:30:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4699999999999998e-07 2024-09-17T07:30:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252568+0000", 2024-09-17T07:30:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5300000000000002e-07 2024-09-17T07:30:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252572+0000", 2024-09-17T07:30:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9160000000000003e-06 2024-09-17T07:30:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252575+0000", 2024-09-17T07:30:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:30:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325820+0000", 2024-09-17T07:30:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073245597999999995 2024-09-17T07:30:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325901+0000", 2024-09-17T07:30:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0913000000000003e-05 2024-09-17T07:30:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326288+0000", 2024-09-17T07:30:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038646900000000002 2024-09-17T07:30:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19992 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6881280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.268829904, 2024-09-17T07:30:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074054683999999996, 2024-09-17T07:30:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19992 2024-09-17T07:30:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252578+0000", 2024-09-17T07:30:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:30:27.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252579+0000", 2024-09-17T07:30:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2500000000000004e-07 2024-09-17T07:30:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252582+0000", 2024-09-17T07:30:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-17T07:30:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252585+0000", 2024-09-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.542e-06 2024-09-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326315+0000", 2024-09-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073729921000000004 2024-09-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326359+0000", 2024-09-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4095999999999999e-05 2024-09-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326632+0000", 2024-09-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027314499999999997 2024-09-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19993 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7110656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.2688196, 2024-09-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074425142999999999, 2024-09-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19993 2024-09-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:30:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252589+0000", 2024-09-17T07:30:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.33e-07 2024-09-17T07:30:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252593+0000", 2024-09-17T07:30:27.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252595+0000", 2024-09-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.621e-06 2024-09-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326666+0000", 2024-09-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074070787999999999 2024-09-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326752+0000", 2024-09-17T07:30:27.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5748000000000006e-05 2024-09-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327013+0000", 2024-09-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026098400000000002 2024-09-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19994 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7340032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.268809141000006, 2024-09-17T07:30:27.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074765920999999999, 2024-09-17T07:30:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19994 2024-09-17T07:30:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252599+0000", 2024-09-17T07:30:27.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2099999999999995e-07 2024-09-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252603+0000", 2024-09-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252606+0000", 2024-09-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-09-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327047+0000", 2024-09-17T07:30:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074441050999999994 2024-09-17T07:30:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327121+0000", 2024-09-17T07:30:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4283999999999996e-05 2024-09-17T07:30:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327364+0000", 2024-09-17T07:30:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000243155 2024-09-17T07:30:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19995 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7569408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.268798443999998, 2024-09-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075172694999999998, 2024-09-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19995 2024-09-17T07:30:27.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252610+0000", 2024-09-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6799999999999999e-07 2024-09-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252614+0000", 2024-09-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000002e-06 2024-09-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252616+0000", 2024-09-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.419e-06 2024-09-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327407+0000", 2024-09-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074790617000000004 2024-09-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327498+0000", 2024-09-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0758000000000001e-05 2024-09-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327782+0000", 2024-09-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028395599999999998 2024-09-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7798784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.268788127999997, 2024-09-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075605832999999997, 2024-09-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.572 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:30:27.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327844+0000", 2024-09-17T07:30:27.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075217709999999993 2024-09-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327927+0000", 2024-09-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2628999999999998e-05 2024-09-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328225+0000", 2024-09-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029800600000000002 2024-09-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8028160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.268777775000004, 2024-09-17T07:30:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076061237000000004, 2024-09-17T07:30:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:30:27.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:30:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:30:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:30:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:30:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:30:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:30:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328268+0000", 2024-09-17T07:30:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075626323999999995 2024-09-17T07:30:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328352+0000", 2024-09-17T07:30:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3578999999999994e-05 2024-09-17T07:30:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328691+0000", 2024-09-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033915099999999999 2024-09-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8257536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.268762323999994, 2024-09-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076441903000000005, 2024-09-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.581 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:30:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328747+0000", 2024-09-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076094192000000005 2024-09-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328824+0000", 2024-09-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7096999999999997e-05 2024-09-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.329087+0000", 2024-09-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026249999999999998 2024-09-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.585 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20029 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7127040~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.258677336000005, 2024-09-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071783696999999994, 2024-09-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20029 2024-09-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262730+0000", 2024-09-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262735+0000", 2024-09-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.955e-06 2024-09-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262737+0000", 2024-09-17T07:30:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0090000000000002e-06 2024-09-17T07:30:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262743+0000", 2024-09-17T07:30:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2890000000000003e-06 2024-09-17T07:30:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262748+0000", 2024-09-17T07:30:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:30:27.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334163+0000", 2024-09-17T07:30:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071415336999999995 2024-09-17T07:30:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334243+0000", 2024-09-17T07:30:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9548000000000005e-05 2024-09-17T07:30:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334514+0000", 2024-09-17T07:30:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027086800000000001 2024-09-17T07:30:27.589 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7356416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.258646165000002, 2024-09-17T07:30:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072172616999999994, 2024-09-17T07:30:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.590 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:30:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:30:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:30:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:30:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:30:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:30:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:30:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:30:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:30:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334558+0000", 2024-09-17T07:30:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071780790999999997 2024-09-17T07:30:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334647+0000", 2024-09-17T07:30:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9448000000000001e-05 2024-09-17T07:30:27.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334934+0000", 2024-09-17T07:30:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028654599999999999 2024-09-17T07:30:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7585792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.258624452999996, 2024-09-17T07:30:27.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072606135000000002, 2024-09-17T07:30:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:30:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:30:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:30:27.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:30:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:30:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:30:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:30:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:30:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335116+0000", 2024-09-17T07:30:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072323897999999998 2024-09-17T07:30:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335166+0000", 2024-09-17T07:30:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9882000000000001e-05 2024-09-17T07:30:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335389+0000", 2024-09-17T07:30:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022334300000000001 2024-09-17T07:30:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7815168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.258612253999999, 2024-09-17T07:30:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073051230999999994, 2024-09-17T07:30:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:30:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:27.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:30:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:30:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:30:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:30:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:30:27.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:30:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335431+0000", 2024-09-17T07:30:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072627852000000007 2024-09-17T07:30:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335525+0000", 2024-09-17T07:30:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3708000000000005e-05 2024-09-17T07:30:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335846+0000", 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032182000000000001 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.258601396000003, 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073284199999999994, 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:30:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:30:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:30:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:30:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:30:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:30:27.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:30:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:30:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335863+0000", 2024-09-17T07:30:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073048921000000003 2024-09-17T07:30:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335904+0000", 2024-09-17T07:30:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1545000000000003e-05 2024-09-17T07:30:27.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336090+0000", 2024-09-17T07:30:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185787 2024-09-17T07:30:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8273920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.258587136000003, 2024-09-17T07:30:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073670478999999997, 2024-09-17T07:30:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:27.607 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:27.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:30:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:30:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:30:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:27.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:30:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:30:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:30:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:30:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336126+0000", 2024-09-17T07:30:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073297254000000006 2024-09-17T07:30:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:27.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336211+0000", 2024-09-17T07:30:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4728000000000006e-05 2024-09-17T07:30:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336491+0000", 2024-09-17T07:30:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027979399999999998 2024-09-17T07:30:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:27.611 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:27.611 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:27.612 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:27.615 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:27.614+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:27.716 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:27.716 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:27.716 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:27.716 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:27.716 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:27.715+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:27.724 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:27.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:30:27.817 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:27.816+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:27.868 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:27.869 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:27.869 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:27.869 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:27.869 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:27.877 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:27.918 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:27.918+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:27.923 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:27.923 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:27.940 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:27.940 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:27.956 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:27.956 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:27.972 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:27.972 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:28.019 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:28.018+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:28.023 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:28.023 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:28.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:28.023 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.116760587000002, 2024-09-17T07:30:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:30:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:30:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:30:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:30:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:30:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:30:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:30:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:30:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:30:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:30:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:30:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:30:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:30:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:30:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:30:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.116741910000002, 2024-09-17T07:30:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:30:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:30:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:30:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:30:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:30:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:30:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:30:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:30:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:30:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:30:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:30:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:30:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:30:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:30:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:30:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.116723557, 2024-09-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:30:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:30:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:30:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:30:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:30:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:30:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:30:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.116703959000006, 2024-09-17T07:30:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:30:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:30:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.116684004000007, 2024-09-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:30:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:30:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:30:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:30:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:30:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:30:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:30:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:30:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.116665349000002, 2024-09-17T07:30:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:30:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:30:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:30:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:30:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:30:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:30:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:30:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.116646872999993, 2024-09-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:30:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:30:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:30:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:30:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:30:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:30:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:30:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:30:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:30:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:30:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:30:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:30:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.116627273999995, 2024-09-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:30:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:30:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:30:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:30:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:30:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:30:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:30:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:30:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:30:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:30:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:30:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.116608698999997, 2024-09-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:30:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:30:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:30:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:30:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:30:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:30:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:30:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:30:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:30:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:30:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:30:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:30:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.116586272000006, 2024-09-17T07:30:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:30:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:30:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:30:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:30:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:30:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:30:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:30:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:30:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:30:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:30:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:30:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:30:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:30:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:30:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:28.070 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.116558455000003, 2024-09-17T07:30:28.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:30:28.070 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.070 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.070 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.070 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:30:28.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.070 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:28.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:28.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:28.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:30:28.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:30:28.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:30:28.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:30:28.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:30:28.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:30:28.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:30:28.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:30:28.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:30:28.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:30:28.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:30:28.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:30:28.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:30:28.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.074 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.074 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:28.074 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:28.074 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.116536511000007, 2024-09-17T07:30:28.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:30:28.074 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.074 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.074 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.075 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:30:28.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.075 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:28.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:28.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:28.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:30:28.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:30:28.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:30:28.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:30:28.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:30:28.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:30:28.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:30:28.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:30:28.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:30:28.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:30:28.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:30:28.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:30:28.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:30:28.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.080 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.081 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.081 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:28.081 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:28.081 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.116515406999994, 2024-09-17T07:30:28.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:30:28.081 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.081 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.081 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.082 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:30:28.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.082 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:28.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:28.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:28.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:30:28.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:30:28.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:30:28.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:30:28.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:28.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:30:28.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:30:28.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:30:28.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:30:28.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:30:28.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:30:28.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:30:28.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:30:28.085 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.085 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.085 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.086 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:28.086 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:28.086 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.116495803999996, 2024-09-17T07:30:28.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:30:28.086 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.086 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.086 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.086 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:30:28.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.086 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:28.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:28.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:30:28.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:30:28.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:30:28.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:30:28.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:30:28.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:30:28.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:30:28.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:30:28.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:30:28.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:30:28.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:30:28.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:30:28.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:30:28.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:30:28.090 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.090 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.090 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.090 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:28.090 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:28.090 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 66.163554691000002, 2024-09-17T07:30:28.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:30:28.090 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.090 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.090 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.091 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.091 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.091 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:30:28.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.091 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:28.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:28.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:30:28.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:30:28.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:30:28.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:28.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:30:28.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:30:28.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:30:28.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:30:28.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:30:28.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:30:28.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:30:28.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:30:28.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:30:28.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:30:28.094 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.094 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.094 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.095 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:28.095 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:28.095 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 66.163541030999994, 2024-09-17T07:30:28.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:30:28.095 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.095 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.095 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.095 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.095 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:30:28.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.096 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:28.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:28.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:28.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:30:28.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:30:28.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:30:28.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:30:28.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:30:28.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:30:28.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:30:28.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:30:28.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:30:28.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:30:28.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:30:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:30:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:30:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 66.163526249, 2024-09-17T07:30:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:30:28.099 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:30:28.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:30:28.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:30:28.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:30:28.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:30:28.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 66.163515357999998, 2024-09-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:28.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:28.108 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 66.163505798000003, 2024-09-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:28.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:30:28.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:30:28.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:30:28.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:30:28.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:30:28.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:30:28.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:30:28.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 66.163495237999996, 2024-09-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:28.113 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:28.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:28.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:30:28.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:30:28.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:28.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:30:28.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:30:28.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:28.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:28.117 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:28.119 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-17T07:30:28.119 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:28.161 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:28.160+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:28.194 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:28.194 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:28.212 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:28.221 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:28.221+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:28.231 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:28.231 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:28.248 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:28.249 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:28.286 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:28.287 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:28.297 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:28.338 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:28.338+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:28.423 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:28.423+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:28.450 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:28.450 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:28.459 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:28.543 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:28.543+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:28.611 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:28.612 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:28.621 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:28.662 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:28.662+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:28.726 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:28.725+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:28.733 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:28.733 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:28.742 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:28.827 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:28.827+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:28.855 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:28.855 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:28.865 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:28.928 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:28.927+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:28.981 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:28.981 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:28.990 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:29.034 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:29.034+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:29.108 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:29.108 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:29.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:29.159 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:29.158+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:29.230 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:29.230+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:29.275 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:29.276 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:29.285 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:29.331 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:29.330+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:29.403 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:29.403 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:29.412 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:29.456 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:29.456+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:29.519 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:29.520 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:29.528 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:29.570 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:29.569+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:29.633 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:29.633+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:29.675 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:29.675 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:29.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:29.734 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:29.733+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:29.832 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:29.832 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:29.835 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:29.835+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:29.840 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:29.936 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:29.936+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:29.990 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:29.990 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:29.999 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:30.037 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:30.037+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:30.116 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:30.117 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:30.121 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T07:30:30.127 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:30.138 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:30.138+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:30.239 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:30.239+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:30.260 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:30:30.281 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:30.281 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:30.291 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:30.340 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:30.339+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:30.388 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:30.388 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:30.402 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:30.403 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:30.405 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:30.405 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:30.413 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:30.421 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:30.437 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:30.440 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:30.440+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:30.454 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:30.472 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:30.472 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:30.489 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:30.489 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:30.506 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:30.506 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:30.524 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:30.524 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:30.524 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:30.524 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:30.524 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:30.533 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:30.559 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:30.558+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:30.638 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:30.638 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:30.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:30.639 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:30.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.639 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:30525 3.1b 3:def7c8f0:::rbd_data.118381ffbbab.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T07:30:30.639 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:30.639 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 77.114686929000001, 2024-09-17T07:30:30.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55515850300000003, 2024-09-17T07:30:30.639 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.639 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.639 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:30.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.640 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30525 2024-09-17T07:30:30.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.640 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:30.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:30.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522300+0000", 2024-09-17T07:30:30.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:30:30.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522305+0000", 2024-09-17T07:30:30.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-17T07:30:30.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522318+0000", 2024-09-17T07:30:30.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3030999999999999e-05 2024-09-17T07:30:30.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522326+0000", 2024-09-17T07:30:30.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4200000000000001e-06 2024-09-17T07:30:30.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522385+0000", 2024-09-17T07:30:30.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9005000000000002e-05 2024-09-17T07:30:30.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:30.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522393+0000", 2024-09-17T07:30:30.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-06 2024-09-17T07:30:30.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075347+0000", 2024-09-17T07:30:30.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55295417800000002 2024-09-17T07:30:30.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075502+0000", 2024-09-17T07:30:30.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154272 2024-09-17T07:30:30.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:30.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075630+0000", 2024-09-17T07:30:30.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012816300000000001 2024-09-17T07:30:30.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:30.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077221+0000", 2024-09-17T07:30:30.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015912929999999999 2024-09-17T07:30:30.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:30.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077353+0000", 2024-09-17T07:30:30.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013215699999999999 2024-09-17T07:30:30.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:30.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077413+0000", 2024-09-17T07:30:30.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0034999999999997e-05 2024-09-17T07:30:30.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077457+0000", 2024-09-17T07:30:30.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4017999999999998e-05 2024-09-17T07:30:30.645 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.645 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.645 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.646 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44181 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13922304~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.646 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:30.646 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.999508281000004, 2024-09-17T07:30:30.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046010502000000002, 2024-09-17T07:30:30.646 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.646 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.646 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.646 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.646 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.646 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.646 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44181 2024-09-17T07:30:30.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.647 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:30.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:30.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637478+0000", 2024-09-17T07:30:30.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:30:30.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637480+0000", 2024-09-17T07:30:30.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.843e-06 2024-09-17T07:30:30.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637487+0000", 2024-09-17T07:30:30.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6159999999999996e-06 2024-09-17T07:30:30.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637492+0000", 2024-09-17T07:30:30.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5360000000000003e-06 2024-09-17T07:30:30.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683351+0000", 2024-09-17T07:30:30.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045858758999999999 2024-09-17T07:30:30.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683422+0000", 2024-09-17T07:30:30.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1705999999999994e-05 2024-09-17T07:30:30.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683488+0000", 2024-09-17T07:30:30.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5612999999999997e-05 2024-09-17T07:30:30.650 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.650 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.650 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.650 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44182 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13938688~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.650 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:30.651 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.999487610000003, 2024-09-17T07:30:30.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046153118999999999, 2024-09-17T07:30:30.651 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.651 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.651 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.651 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.651 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.651 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.651 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44182 2024-09-17T07:30:30.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.651 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:30.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:30.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:30.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:30:30.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637506+0000", 2024-09-17T07:30:30.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1520000000000003e-06 2024-09-17T07:30:30.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637512+0000", 2024-09-17T07:30:30.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-09-17T07:30:30.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637517+0000", 2024-09-17T07:30:30.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-09-17T07:30:30.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683514+0000", 2024-09-17T07:30:30.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045997616999999998 2024-09-17T07:30:30.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683584+0000", 2024-09-17T07:30:30.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9942000000000002e-05 2024-09-17T07:30:30.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683651+0000", 2024-09-17T07:30:30.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6951000000000003e-05 2024-09-17T07:30:30.655 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.655 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.655 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.655 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44183 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13955072~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.655 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:30.655 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.999462378000004, 2024-09-17T07:30:30.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046339793999999997, 2024-09-17T07:30:30.655 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.656 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.656 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.656 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44183 2024-09-17T07:30:30.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.656 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:30.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:30.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639583+0000", 2024-09-17T07:30:30.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002059412 2024-09-17T07:30:30.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639597+0000", 2024-09-17T07:30:30.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3832000000000001e-05 2024-09-17T07:30:30.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639607+0000", 2024-09-17T07:30:30.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485e-05 2024-09-17T07:30:30.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639614+0000", 2024-09-17T07:30:30.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0849999999999999e-06 2024-09-17T07:30:30.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683705+0000", 2024-09-17T07:30:30.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044091261 2024-09-17T07:30:30.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683792+0000", 2024-09-17T07:30:30.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6197000000000002e-05 2024-09-17T07:30:30.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683863+0000", 2024-09-17T07:30:30.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1521999999999997e-05 2024-09-17T07:30:30.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.660 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.660 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44199 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14217216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.660 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:30.660 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.995944825999999, 2024-09-17T07:30:30.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045995623999999999, 2024-09-17T07:30:30.660 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.660 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.660 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.661 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44199 2024-09-17T07:30:30.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.661 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:30.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:30.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:30.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:30:30.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641043+0000", 2024-09-17T07:30:30.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.624e-06 2024-09-17T07:30:30.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641048+0000", 2024-09-17T07:30:30.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4029999999999998e-06 2024-09-17T07:30:30.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641053+0000", 2024-09-17T07:30:30.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6670000000000004e-06 2024-09-17T07:30:30.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686918+0000", 2024-09-17T07:30:30.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045865131000000003 2024-09-17T07:30:30.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686986+0000", 2024-09-17T07:30:30.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8299000000000005e-05 2024-09-17T07:30:30.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687036+0000", 2024-09-17T07:30:30.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0164999999999998e-05 2024-09-17T07:30:30.664 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.664 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.665 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.665 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44200 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14233600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.665 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:30.665 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.995926996999998, 2024-09-17T07:30:30.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046183984999999997, 2024-09-17T07:30:30.665 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.665 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.665 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.665 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.666 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44200 2024-09-17T07:30:30.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.666 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:30.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:30.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:30.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:30.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641061+0000", 2024-09-17T07:30:30.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.767e-06 2024-09-17T07:30:30.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641067+0000", 2024-09-17T07:30:30.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7379999999999999e-06 2024-09-17T07:30:30.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641071+0000", 2024-09-17T07:30:30.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-09-17T07:30:30.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687078+0000", 2024-09-17T07:30:30.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046007095999999997 2024-09-17T07:30:30.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687158+0000", 2024-09-17T07:30:30.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0374e-05 2024-09-17T07:30:30.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687243+0000", 2024-09-17T07:30:30.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4221999999999994e-05 2024-09-17T07:30:30.669 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.669 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.669 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.669 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44201 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14249984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.669 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:30.670 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.995909350999995, 2024-09-17T07:30:30.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046303182999999998, 2024-09-17T07:30:30.670 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.670 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.670 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.670 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44201 2024-09-17T07:30:30.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.670 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:30.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:30.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641120+0000", 2024-09-17T07:30:30.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3933000000000001e-05 2024-09-17T07:30:30.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641123+0000", 2024-09-17T07:30:30.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.191e-06 2024-09-17T07:30:30.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641133+0000", 2024-09-17T07:30:30.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4020000000000008e-06 2024-09-17T07:30:30.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641138+0000", 2024-09-17T07:30:30.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6559999999999997e-06 2024-09-17T07:30:30.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687279+0000", 2024-09-17T07:30:30.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046140278999999999 2024-09-17T07:30:30.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687338+0000", 2024-09-17T07:30:30.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9268999999999998e-05 2024-09-17T07:30:30.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687379+0000", 2024-09-17T07:30:30.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1452999999999998e-05 2024-09-17T07:30:30.674 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.674 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.674 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.674 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44202 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14266368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.674 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:30.674 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.995838585000001, 2024-09-17T07:30:30.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046484299, 2024-09-17T07:30:30.675 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.675 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.675 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.675 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.675 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.675 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.675 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44202 2024-09-17T07:30:30.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.675 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:30.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:30.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642810+0000", 2024-09-17T07:30:30.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016627980000000001 2024-09-17T07:30:30.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642815+0000", 2024-09-17T07:30:30.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.091e-06 2024-09-17T07:30:30.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642829+0000", 2024-09-17T07:30:30.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4047e-05 2024-09-17T07:30:30.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642844+0000", 2024-09-17T07:30:30.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4515000000000001e-05 2024-09-17T07:30:30.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687426+0000", 2024-09-17T07:30:30.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044582201000000002 2024-09-17T07:30:30.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687529+0000", 2024-09-17T07:30:30.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103078 2024-09-17T07:30:30.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687631+0000", 2024-09-17T07:30:30.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102569 2024-09-17T07:30:30.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.679 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.679 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44209 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14381056~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.679 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:30.679 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.993940433000006, 2024-09-17T07:30:30.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045944444000000001, 2024-09-17T07:30:30.679 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.679 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.679 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.679 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.679 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.680 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44209 2024-09-17T07:30:30.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.680 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:30.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:30.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643046+0000", 2024-09-17T07:30:30.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:30:30.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643048+0000", 2024-09-17T07:30:30.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:30:30.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643054+0000", 2024-09-17T07:30:30.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.032e-06 2024-09-17T07:30:30.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643065+0000", 2024-09-17T07:30:30.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1861999999999999e-05 2024-09-17T07:30:30.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688806+0000", 2024-09-17T07:30:30.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045740617999999997 2024-09-17T07:30:30.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688927+0000", 2024-09-17T07:30:30.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000121327 2024-09-17T07:30:30.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688990+0000", 2024-09-17T07:30:30.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2293999999999997e-05 2024-09-17T07:30:30.683 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.683 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.683 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.683 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44210 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14397440~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.683 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:30.683 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.993913507000002, 2024-09-17T07:30:30.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046104635999999997, 2024-09-17T07:30:30.684 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.684 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.684 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.684 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.684 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.684 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.684 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44210 2024-09-17T07:30:30.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.684 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:30.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:30.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643073+0000", 2024-09-17T07:30:30.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.77e-07 2024-09-17T07:30:30.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643075+0000", 2024-09-17T07:30:30.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-09-17T07:30:30.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643082+0000", 2024-09-17T07:30:30.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7789999999999996e-06 2024-09-17T07:30:30.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643087+0000", 2024-09-17T07:30:30.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.967e-06 2024-09-17T07:30:30.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689022+0000", 2024-09-17T07:30:30.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045934349999999999 2024-09-17T07:30:30.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689112+0000", 2024-09-17T07:30:30.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0039999999999999e-05 2024-09-17T07:30:30.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689177+0000", 2024-09-17T07:30:30.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5041000000000003e-05 2024-09-17T07:30:30.687 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.687 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.687 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.688 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44211 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14413824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.688 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:30.688 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.993892373999998, 2024-09-17T07:30:30.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046309003000000001, 2024-09-17T07:30:30.688 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.688 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.688 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.688 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.688 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.688 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.688 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44211 2024-09-17T07:30:30.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.689 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:30.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:30.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643094+0000", 2024-09-17T07:30:30.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:30:30.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643095+0000", 2024-09-17T07:30:30.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-17T07:30:30.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643101+0000", 2024-09-17T07:30:30.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8730000000000004e-06 2024-09-17T07:30:30.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643106+0000", 2024-09-17T07:30:30.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6859999999999997e-06 2024-09-17T07:30:30.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689220+0000", 2024-09-17T07:30:30.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046113945000000003 2024-09-17T07:30:30.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689309+0000", 2024-09-17T07:30:30.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9560000000000003e-05 2024-09-17T07:30:30.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689402+0000", 2024-09-17T07:30:30.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2786000000000002e-05 2024-09-17T07:30:30.692 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.692 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.692 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.692 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44212 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14430208~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.692 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:30.693 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.993847811999998, 2024-09-17T07:30:30.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046448912000000002, 2024-09-17T07:30:30.693 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.693 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.693 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.693 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.693 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.693 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.693 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44212 2024-09-17T07:30:30.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.693 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:30.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:30.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:30.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2200000000000004e-07 2024-09-17T07:30:30.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643141+0000", 2024-09-17T07:30:30.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5210000000000001e-06 2024-09-17T07:30:30.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643149+0000", 2024-09-17T07:30:30.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6490000000000008e-06 2024-09-17T07:30:30.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643161+0000", 2024-09-17T07:30:30.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2241e-05 2024-09-17T07:30:30.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689423+0000", 2024-09-17T07:30:30.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046262323000000001 2024-09-17T07:30:30.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689505+0000", 2024-09-17T07:30:30.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2103999999999999e-05 2024-09-17T07:30:30.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689587+0000", 2024-09-17T07:30:30.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1452000000000006e-05 2024-09-17T07:30:30.697 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.697 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.697 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.697 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44213 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14446592~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.697 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:30.697 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.993816312999996, 2024-09-17T07:30:30.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046601780000000002, 2024-09-17T07:30:30.697 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.698 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.698 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.698 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.698 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.698 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.698 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44213 2024-09-17T07:30:30.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.698 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:30.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:30.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643170+0000", 2024-09-17T07:30:30.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-17T07:30:30.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643172+0000", 2024-09-17T07:30:30.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1660000000000001e-06 2024-09-17T07:30:30.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643180+0000", 2024-09-17T07:30:30.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-09-17T07:30:30.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643185+0000", 2024-09-17T07:30:30.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-09-17T07:30:30.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689622+0000", 2024-09-17T07:30:30.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046436742000000003 2024-09-17T07:30:30.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689697+0000", 2024-09-17T07:30:30.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5179999999999995e-05 2024-09-17T07:30:30.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689771+0000", 2024-09-17T07:30:30.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3928000000000001e-05 2024-09-17T07:30:30.701 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.701 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.701 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.701 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44214 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14462976~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.701 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:30.701 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.993793253000007, 2024-09-17T07:30:30.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046736038000000001, 2024-09-17T07:30:30.702 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.702 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.702 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.702 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.702 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.702 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.702 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44214 2024-09-17T07:30:30.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.702 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:30.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:30.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643193+0000", 2024-09-17T07:30:30.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:30:30.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643195+0000", 2024-09-17T07:30:30.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0640000000000001e-06 2024-09-17T07:30:30.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643201+0000", 2024-09-17T07:30:30.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2859999999999996e-06 2024-09-17T07:30:30.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643206+0000", 2024-09-17T07:30:30.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-17T07:30:30.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689814+0000", 2024-09-17T07:30:30.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046607742000000001 2024-09-17T07:30:30.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689886+0000", 2024-09-17T07:30:30.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2020999999999994e-05 2024-09-17T07:30:30.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689928+0000", 2024-09-17T07:30:30.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2771000000000001e-05 2024-09-17T07:30:30.706 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.706 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.706 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.706 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44215 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14479360~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.706 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:30.706 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.993749355999995, 2024-09-17T07:30:30.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046868543999999998, 2024-09-17T07:30:30.706 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.706 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.706 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.706 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.706 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.707 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44215 2024-09-17T07:30:30.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.707 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:30.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:30.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643237+0000", 2024-09-17T07:30:30.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:30:30.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643245+0000", 2024-09-17T07:30:30.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3270000000000001e-06 2024-09-17T07:30:30.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643254+0000", 2024-09-17T07:30:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6049999999999998e-06 2024-09-17T07:30:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643260+0000", 2024-09-17T07:30:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7849999999999997e-06 2024-09-17T07:30:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689961+0000", 2024-09-17T07:30:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046701196 2024-09-17T07:30:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690036+0000", 2024-09-17T07:30:30.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5153000000000004e-05 2024-09-17T07:30:30.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690105+0000", 2024-09-17T07:30:30.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8874000000000005e-05 2024-09-17T07:30:30.710 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.710 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.710 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.710 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44216 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14495744~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.710 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:30.711 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.993717642999997, 2024-09-17T07:30:30.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047003047999999999, 2024-09-17T07:30:30.711 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.711 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.711 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.711 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.711 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.711 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.711 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44216 2024-09-17T07:30:30.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.711 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:30.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:30.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643269+0000", 2024-09-17T07:30:30.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:30:30.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643271+0000", 2024-09-17T07:30:30.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-09-17T07:30:30.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643278+0000", 2024-09-17T07:30:30.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8850000000000001e-06 2024-09-17T07:30:30.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643282+0000", 2024-09-17T07:30:30.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5979999999999999e-06 2024-09-17T07:30:30.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690133+0000", 2024-09-17T07:30:30.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850470999999998 2024-09-17T07:30:30.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690205+0000", 2024-09-17T07:30:30.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2442999999999997e-05 2024-09-17T07:30:30.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690271+0000", 2024-09-17T07:30:30.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5874e-05 2024-09-17T07:30:30.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.715 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.715 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44217 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14512128~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.715 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:30.715 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.993696631000006, 2024-09-17T07:30:30.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047166485000000001, 2024-09-17T07:30:30.715 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.715 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.716 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.716 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44217 2024-09-17T07:30:30.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.716 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:30.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:30.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:30.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-17T07:30:30.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643291+0000", 2024-09-17T07:30:30.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.962e-06 2024-09-17T07:30:30.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643297+0000", 2024-09-17T07:30:30.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0700000000000003e-06 2024-09-17T07:30:30.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643302+0000", 2024-09-17T07:30:30.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:30:30.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690308+0000", 2024-09-17T07:30:30.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047005354999999999 2024-09-17T07:30:30.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690385+0000", 2024-09-17T07:30:30.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7541999999999997e-05 2024-09-17T07:30:30.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690455+0000", 2024-09-17T07:30:30.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0328999999999997e-05 2024-09-17T07:30:30.719 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.719 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.719 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.720 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44218 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14528512~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.720 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:30.720 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.993676953999994, 2024-09-17T07:30:30.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047321015000000001, 2024-09-17T07:30:30.720 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.720 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.720 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.720 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44218 2024-09-17T07:30:30.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.721 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:30.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:30.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:30.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-17T07:30:30.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643311+0000", 2024-09-17T07:30:30.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.933e-06 2024-09-17T07:30:30.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643318+0000", 2024-09-17T07:30:30.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3980000000000004e-06 2024-09-17T07:30:30.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643322+0000", 2024-09-17T07:30:30.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7609999999999998e-06 2024-09-17T07:30:30.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690484+0000", 2024-09-17T07:30:30.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047161428999999998 2024-09-17T07:30:30.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690564+0000", 2024-09-17T07:30:30.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9845000000000005e-05 2024-09-17T07:30:30.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690630+0000", 2024-09-17T07:30:30.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6021999999999999e-05 2024-09-17T07:30:30.724 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.724 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.724 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.724 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44219 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14544896~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.724 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:30.724 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.993656587000004, 2024-09-17T07:30:30.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048128708999999999, 2024-09-17T07:30:30.724 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.724 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.725 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.725 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.725 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.725 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.725 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44219 2024-09-17T07:30:30.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.725 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:30.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:30.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:30.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:30:30.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643337+0000", 2024-09-17T07:30:30.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.109e-06 2024-09-17T07:30:30.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643350+0000", 2024-09-17T07:30:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3046e-05 2024-09-17T07:30:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643356+0000", 2024-09-17T07:30:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8930000000000002e-06 2024-09-17T07:30:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690660+0000", 2024-09-17T07:30:30.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047304061000000001 2024-09-17T07:30:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691372+0000", 2024-09-17T07:30:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00071281299999999997 2024-09-17T07:30:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691458+0000", 2024-09-17T07:30:30.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5352000000000005e-05 2024-09-17T07:30:30.729 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.729 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.729 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.729 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44220 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14561280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:30.729 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:30.729 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.993622539, 2024-09-17T07:30:30.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048309849000000002, 2024-09-17T07:30:30.729 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:30.729 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:30.729 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:30.730 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:30.730 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:30.730 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:30.730 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44220 2024-09-17T07:30:30.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.730 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:30.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:30.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:30.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:30.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:30.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:30.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:30.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646869+0000", 2024-09-17T07:30:30.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003506267 2024-09-17T07:30:30.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:30.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646874+0000", 2024-09-17T07:30:30.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7500000000000003e-06 2024-09-17T07:30:30.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:30.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646887+0000", 2024-09-17T07:30:30.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3115999999999999e-05 2024-09-17T07:30:30.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:30.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646894+0000", 2024-09-17T07:30:30.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5289999999999997e-06 2024-09-17T07:30:30.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:30.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691547+0000", 2024-09-17T07:30:30.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044652743000000002 2024-09-17T07:30:30.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:30.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691622+0000", 2024-09-17T07:30:30.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5037999999999995e-05 2024-09-17T07:30:30.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:30.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:30.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:30.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691673+0000", 2024-09-17T07:30:30.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-17T07:30:30.733 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.733 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.734 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.734 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:30.734 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:30.734 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:30.735 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:30.736 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:30.735+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:30.835 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:30.835+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:30.880 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:30.880 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:30.880 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:30.880 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:30.889 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:30.972 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:30.972+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:31.031 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:31.031 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:31.031 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:31.031 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:31.031 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:31.037 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:31.036+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:31.040 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:31.087 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-17T07:30:31.152 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:31.151+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:31.202 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:31.202 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:31.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:31.202 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:31.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.202 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19985 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5275648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.202 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:31.203 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.948289623999997, 2024-09-17T07:30:31.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071819998999999995, 2024-09-17T07:30:31.203 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.203 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.203 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.203 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.203 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.203 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.203 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19985 2024-09-17T07:30:31.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.203 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:31.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:31.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:31.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:31.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252493+0000", 2024-09-17T07:30:31.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6300000000000004e-07 2024-09-17T07:30:31.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252496+0000", 2024-09-17T07:30:31.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-17T07:30:31.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252499+0000", 2024-09-17T07:30:31.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:30:31.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324036+0000", 2024-09-17T07:30:31.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071536587999999998 2024-09-17T07:30:31.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324118+0000", 2024-09-17T07:30:31.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2336999999999995e-05 2024-09-17T07:30:31.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324312+0000", 2024-09-17T07:30:31.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000193709 2024-09-17T07:30:31.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.207 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.207 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.207 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19986 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5505024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.207 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:31.207 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.948278471999998, 2024-09-17T07:30:31.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072083761999999996, 2024-09-17T07:30:31.207 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.207 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.207 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.207 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.207 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.207 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.208 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19986 2024-09-17T07:30:31.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.208 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:31.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:31.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:31.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:30:31.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252508+0000", 2024-09-17T07:30:31.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:30:31.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252511+0000", 2024-09-17T07:30:31.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.145e-06 2024-09-17T07:30:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252514+0000", 2024-09-17T07:30:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5569999999999998e-06 2024-09-17T07:30:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324326+0000", 2024-09-17T07:30:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071812681000000003 2024-09-17T07:30:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324372+0000", 2024-09-17T07:30:31.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5893000000000003e-05 2024-09-17T07:30:31.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324587+0000", 2024-09-17T07:30:31.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214608 2024-09-17T07:30:31.211 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.211 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.211 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.211 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19987 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5734400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.211 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:31.212 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.948264854000001, 2024-09-17T07:30:31.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072276676999999998, 2024-09-17T07:30:31.212 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.212 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.212 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.212 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.212 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.212 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.212 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19987 2024-09-17T07:30:31.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.212 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:31.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:31.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:31.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:31.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252518+0000", 2024-09-17T07:30:31.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-17T07:30:31.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252522+0000", 2024-09-17T07:30:31.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2440000000000004e-06 2024-09-17T07:30:31.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252525+0000", 2024-09-17T07:30:31.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5919999999999999e-06 2024-09-17T07:30:31.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324621+0000", 2024-09-17T07:30:31.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072096442999999996 2024-09-17T07:30:31.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324699+0000", 2024-09-17T07:30:31.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8061999999999999e-05 2024-09-17T07:30:31.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324793+0000", 2024-09-17T07:30:31.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4207000000000001e-05 2024-09-17T07:30:31.216 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.216 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.216 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.216 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19988 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.216 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:31.216 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.948253883000007, 2024-09-17T07:30:31.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072585798000000007, 2024-09-17T07:30:31.216 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.216 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.216 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.216 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.217 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.217 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.217 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19988 2024-09-17T07:30:31.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.217 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:31.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:31.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:31.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:31.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252529+0000", 2024-09-17T07:30:31.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:30:31.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252532+0000", 2024-09-17T07:30:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9579999999999999e-06 2024-09-17T07:30:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252534+0000", 2024-09-17T07:30:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5730000000000002e-06 2024-09-17T07:30:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324825+0000", 2024-09-17T07:30:31.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072290757999999997 2024-09-17T07:30:31.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324874+0000", 2024-09-17T07:30:31.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9360000000000002e-05 2024-09-17T07:30:31.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325113+0000", 2024-09-17T07:30:31.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000239025 2024-09-17T07:30:31.220 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.220 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.221 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.222 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19989 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.222 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:31.222 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.948244262000003, 2024-09-17T07:30:31.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072963494000000004, 2024-09-17T07:30:31.222 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.222 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.222 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.222 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.222 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.222 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.222 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19989 2024-09-17T07:30:31.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.223 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:31.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:31.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:30:31.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:30:31.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:30:31.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2900000000000002e-07 2024-09-17T07:30:31.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252542+0000", 2024-09-17T07:30:31.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8600000000000003e-06 2024-09-17T07:30:31.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252545+0000", 2024-09-17T07:30:31.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.605e-06 2024-09-17T07:30:31.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325144+0000", 2024-09-17T07:30:31.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072599460000000005 2024-09-17T07:30:31.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325223+0000", 2024-09-17T07:30:31.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8473999999999994e-05 2024-09-17T07:30:31.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325501+0000", 2024-09-17T07:30:31.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000278063 2024-09-17T07:30:31.226 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.226 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.226 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.226 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19990 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6422528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.226 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:31.226 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.948233791000007, 2024-09-17T07:30:31.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073241127000000003, 2024-09-17T07:30:31.226 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.226 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.227 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.227 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.227 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.227 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.227 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19990 2024-09-17T07:30:31.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.227 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:31.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:31.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:31.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:31.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252552+0000", 2024-09-17T07:30:31.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4719999999999997e-06 2024-09-17T07:30:31.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252557+0000", 2024-09-17T07:30:31.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3039999999999998e-06 2024-09-17T07:30:31.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252560+0000", 2024-09-17T07:30:31.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8830000000000002e-06 2024-09-17T07:30:31.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325529+0000", 2024-09-17T07:30:31.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072968880999999999 2024-09-17T07:30:31.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325574+0000", 2024-09-17T07:30:31.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466000000000001e-05 2024-09-17T07:30:31.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325789+0000", 2024-09-17T07:30:31.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021490999999999999 2024-09-17T07:30:31.231 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.231 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.231 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.231 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19991 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6651904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.231 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:31.231 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.948214488999994, 2024-09-17T07:30:31.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073720552999999994, 2024-09-17T07:30:31.231 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.232 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.232 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.232 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19991 2024-09-17T07:30:31.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.232 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:31.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:31.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:31.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4699999999999998e-07 2024-09-17T07:30:31.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252568+0000", 2024-09-17T07:30:31.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5300000000000002e-07 2024-09-17T07:30:31.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252572+0000", 2024-09-17T07:30:31.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9160000000000003e-06 2024-09-17T07:30:31.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252575+0000", 2024-09-17T07:30:31.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:30:31.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325820+0000", 2024-09-17T07:30:31.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073245597999999995 2024-09-17T07:30:31.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325901+0000", 2024-09-17T07:30:31.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0913000000000003e-05 2024-09-17T07:30:31.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326288+0000", 2024-09-17T07:30:31.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038646900000000002 2024-09-17T07:30:31.235 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.235 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.236 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.236 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19992 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6881280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.236 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:31.236 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.948204136000001, 2024-09-17T07:30:31.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074054683999999996, 2024-09-17T07:30:31.236 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.236 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.236 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.237 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19992 2024-09-17T07:30:31.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.237 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:31.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:31.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252578+0000", 2024-09-17T07:30:31.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:30:31.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252579+0000", 2024-09-17T07:30:31.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2500000000000004e-07 2024-09-17T07:30:31.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252582+0000", 2024-09-17T07:30:31.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-17T07:30:31.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252585+0000", 2024-09-17T07:30:31.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.542e-06 2024-09-17T07:30:31.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326315+0000", 2024-09-17T07:30:31.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073729921000000004 2024-09-17T07:30:31.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326359+0000", 2024-09-17T07:30:31.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4095999999999999e-05 2024-09-17T07:30:31.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326632+0000", 2024-09-17T07:30:31.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027314499999999997 2024-09-17T07:30:31.240 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.240 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.241 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.241 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19993 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7110656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.241 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:31.241 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.948193832000001, 2024-09-17T07:30:31.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074425142999999999, 2024-09-17T07:30:31.241 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.241 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.241 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.241 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.241 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.241 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.242 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19993 2024-09-17T07:30:31.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.242 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:31.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:31.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:31.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:30:31.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252589+0000", 2024-09-17T07:30:31.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.33e-07 2024-09-17T07:30:31.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252593+0000", 2024-09-17T07:30:31.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:30:31.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252595+0000", 2024-09-17T07:30:31.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.621e-06 2024-09-17T07:30:31.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326666+0000", 2024-09-17T07:30:31.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074070787999999999 2024-09-17T07:30:31.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326752+0000", 2024-09-17T07:30:31.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5748000000000006e-05 2024-09-17T07:30:31.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327013+0000", 2024-09-17T07:30:31.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026098400000000002 2024-09-17T07:30:31.245 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.245 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.245 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.245 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19994 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7340032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.245 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:31.245 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.948183373000006, 2024-09-17T07:30:31.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074765920999999999, 2024-09-17T07:30:31.245 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.246 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.246 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.246 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.246 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.246 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.246 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19994 2024-09-17T07:30:31.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.246 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:31.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:31.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:31.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:30:31.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252599+0000", 2024-09-17T07:30:31.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2099999999999995e-07 2024-09-17T07:30:31.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252603+0000", 2024-09-17T07:30:31.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:30:31.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252606+0000", 2024-09-17T07:30:31.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-09-17T07:30:31.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327047+0000", 2024-09-17T07:30:31.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074441050999999994 2024-09-17T07:30:31.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327121+0000", 2024-09-17T07:30:31.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4283999999999996e-05 2024-09-17T07:30:31.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327364+0000", 2024-09-17T07:30:31.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000243155 2024-09-17T07:30:31.250 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.250 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.250 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.250 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19995 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7569408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.250 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:31.250 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.948172675999999, 2024-09-17T07:30:31.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075172694999999998, 2024-09-17T07:30:31.250 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.251 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.251 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.251 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.251 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.251 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.251 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19995 2024-09-17T07:30:31.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.251 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:31.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:31.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:31.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:30:31.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252610+0000", 2024-09-17T07:30:31.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6799999999999999e-07 2024-09-17T07:30:31.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252614+0000", 2024-09-17T07:30:31.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000002e-06 2024-09-17T07:30:31.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252616+0000", 2024-09-17T07:30:31.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.419e-06 2024-09-17T07:30:31.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327407+0000", 2024-09-17T07:30:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074790617000000004 2024-09-17T07:30:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327498+0000", 2024-09-17T07:30:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0758000000000001e-05 2024-09-17T07:30:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327782+0000", 2024-09-17T07:30:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028395599999999998 2024-09-17T07:30:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.254 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7798784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.948162359999998, 2024-09-17T07:30:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075605832999999997, 2024-09-17T07:30:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:30:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.255 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:31.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:31.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:31.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:30:31.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:30:31.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:30:31.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:30:31.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:30:31.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:30:31.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:30:31.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327844+0000", 2024-09-17T07:30:31.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075217709999999993 2024-09-17T07:30:31.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327927+0000", 2024-09-17T07:30:31.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2628999999999998e-05 2024-09-17T07:30:31.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328225+0000", 2024-09-17T07:30:31.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029800600000000002 2024-09-17T07:30:31.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.259 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.259 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8028160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.259 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:31.259 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.948152007000004, 2024-09-17T07:30:31.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076061237000000004, 2024-09-17T07:30:31.259 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.259 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.260 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.260 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:30:31.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.260 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:31.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:31.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:31.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:31.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:30:31.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:30:31.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:30:31.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:30:31.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:30:31.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:30:31.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328268+0000", 2024-09-17T07:30:31.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075626323999999995 2024-09-17T07:30:31.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328352+0000", 2024-09-17T07:30:31.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3578999999999994e-05 2024-09-17T07:30:31.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328691+0000", 2024-09-17T07:30:31.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033915099999999999 2024-09-17T07:30:31.263 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.263 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.263 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.264 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8257536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.264 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:31.264 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.948136555999994, 2024-09-17T07:30:31.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076441903000000005, 2024-09-17T07:30:31.264 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.264 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.264 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.264 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.264 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.264 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.264 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:30:31.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.265 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:31.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:31.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:31.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:30:31.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:30:31.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:30:31.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:30:31.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:30:31.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:30:31.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:30:31.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328747+0000", 2024-09-17T07:30:31.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076094192000000005 2024-09-17T07:30:31.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328824+0000", 2024-09-17T07:30:31.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7096999999999997e-05 2024-09-17T07:30:31.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.329087+0000", 2024-09-17T07:30:31.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026249999999999998 2024-09-17T07:30:31.268 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.268 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.268 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.268 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7356416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.268 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:31.268 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.938020397000003, 2024-09-17T07:30:31.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072172616999999994, 2024-09-17T07:30:31.269 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.269 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.269 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.269 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.269 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.269 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.269 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:30:31.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.269 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:31.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:31.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:30:31.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:30:31.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:30:31.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:30:31.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:30:31.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:30:31.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:30:31.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:30:31.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334558+0000", 2024-09-17T07:30:31.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071780790999999997 2024-09-17T07:30:31.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334647+0000", 2024-09-17T07:30:31.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9448000000000001e-05 2024-09-17T07:30:31.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334934+0000", 2024-09-17T07:30:31.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028654599999999999 2024-09-17T07:30:31.272 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.272 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.272 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.273 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7585792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.273 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:31.273 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.937998684999997, 2024-09-17T07:30:31.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072606135000000002, 2024-09-17T07:30:31.273 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.273 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.273 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.273 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.273 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.273 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.273 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:30:31.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.274 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:31.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:31.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:31.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:30:31.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:30:31.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:30:31.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:30:31.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:30:31.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:30:31.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:30:31.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335116+0000", 2024-09-17T07:30:31.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072323897999999998 2024-09-17T07:30:31.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335166+0000", 2024-09-17T07:30:31.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9882000000000001e-05 2024-09-17T07:30:31.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335389+0000", 2024-09-17T07:30:31.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022334300000000001 2024-09-17T07:30:31.277 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.277 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.277 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.277 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7815168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.278 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:31.278 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.937986486, 2024-09-17T07:30:31.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073051230999999994, 2024-09-17T07:30:31.278 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.278 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.278 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.278 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:30:31.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.278 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:31.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:31.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:31.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:31.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:30:31.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:30:31.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:30:31.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:30:31.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:30:31.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:30:31.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335431+0000", 2024-09-17T07:30:31.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072627852000000007 2024-09-17T07:30:31.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335525+0000", 2024-09-17T07:30:31.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3708000000000005e-05 2024-09-17T07:30:31.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335846+0000", 2024-09-17T07:30:31.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032182000000000001 2024-09-17T07:30:31.282 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.282 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.282 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.282 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.283 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:31.283 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.937975628000004, 2024-09-17T07:30:31.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073284199999999994, 2024-09-17T07:30:31.283 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.283 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.283 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.283 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:30:31.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.283 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:31.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:31.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:31.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:30:31.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:30:31.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:30:31.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:30:31.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:30:31.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:30:31.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:30:31.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335863+0000", 2024-09-17T07:30:31.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073048921000000003 2024-09-17T07:30:31.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335904+0000", 2024-09-17T07:30:31.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1545000000000003e-05 2024-09-17T07:30:31.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336090+0000", 2024-09-17T07:30:31.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185787 2024-09-17T07:30:31.287 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.287 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.287 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.287 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8273920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:31.287 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:31.287 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.937961368000003, 2024-09-17T07:30:31.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073670478999999997, 2024-09-17T07:30:31.287 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.287 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.287 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.288 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.288 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.288 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:30:31.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.288 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:31.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:31.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:31.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:30:31.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:30:31.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:30:31.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:30:31.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:30:31.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:30:31.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:30:31.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336126+0000", 2024-09-17T07:30:31.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073297254000000006 2024-09-17T07:30:31.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336211+0000", 2024-09-17T07:30:31.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4728000000000006e-05 2024-09-17T07:30:31.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336491+0000", 2024-09-17T07:30:31.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027979399999999998 2024-09-17T07:30:31.291 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.291 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.291 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.291 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:116163 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-17T07:30:31.291 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:30:31.292 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.9356264999999997, 2024-09-17T07:30:31.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8549867870000001, 2024-09-17T07:30:31.292 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.292 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.292 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.292 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116163 2024-09-17T07:30:31.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.292 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:30:31.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:30:31.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265157+0000", 2024-09-17T07:30:31.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-06 2024-09-17T07:30:31.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265168+0000", 2024-09-17T07:30:31.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0827e-05 2024-09-17T07:30:31.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265198+0000", 2024-09-17T07:30:31.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0383e-05 2024-09-17T07:30:31.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265212+0000", 2024-09-17T07:30:31.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3799e-05 2024-09-17T07:30:31.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265338+0000", 2024-09-17T07:30:31.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012576800000000001 2024-09-17T07:30:31.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:31.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265357+0000", 2024-09-17T07:30:31.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9417000000000001e-05 2024-09-17T07:30:31.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:27.083102+0000", 2024-09-17T07:30:31.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.81774456299999998 2024-09-17T07:30:31.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:31.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:27.083113+0000", 2024-09-17T07:30:31.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1154e-05 2024-09-17T07:30:31.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:28.119793+0000", 2024-09-17T07:30:31.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.036680365 2024-09-17T07:30:31.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:28.119943+0000", 2024-09-17T07:30:31.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014940999999999999 2024-09-17T07:30:31.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:28.120142+0000", 2024-09-17T07:30:31.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00019897099999999999 2024-09-17T07:30:31.297 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.297 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.297 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.297 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.297 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.297 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:31.298 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 5, 6, 7, 3, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-17T07:30:31.298 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:30:31.298 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T07:30:31.298 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T07:30:31.299 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:31.298+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:31.300 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:31.400 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:31.399+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:31.403 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:31.404 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:31.404 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:31.404 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:31.412 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:31.501 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:31.500+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:31.515 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:31.515 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:31.516 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:31.516 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:31.524 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:31.601 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:31.601+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:31.604 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:30:31.604 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:31.604 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 122, 2024-09-17T07:30:31.604 INFO:teuthology.orchestra.run.smithi045.stdout: "fsid": "b283e6d5-0f4a-4271-a443-15e695e35364", 2024-09-17T07:30:31.604 INFO:teuthology.orchestra.run.smithi045.stdout: "created": "2024-09-17T07:28:12.276015+0000", 2024-09-17T07:30:31.604 INFO:teuthology.orchestra.run.smithi045.stdout: "modified": "2024-09-17T07:30:31.082939+0000", 2024-09-17T07:30:31.604 INFO:teuthology.orchestra.run.smithi045.stdout: "last_up_change": "2024-09-17T07:29:58.120417+0000", 2024-09-17T07:30:31.604 INFO:teuthology.orchestra.run.smithi045.stdout: "last_in_change": "2024-09-17T07:30:26.234253+0000", 2024-09-17T07:30:31.604 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T07:30:31.605 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_num": 5801984, 2024-09-17T07:30:31.605 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_set": [ 2024-09-17T07:30:31.605 INFO:teuthology.orchestra.run.smithi045.stdout: "noscrub", 2024-09-17T07:30:31.605 INFO:teuthology.orchestra.run.smithi045.stdout: "pglog_hardlimit", 2024-09-17T07:30:31.605 INFO:teuthology.orchestra.run.smithi045.stdout: "purged_snapdirs", 2024-09-17T07:30:31.605 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_deletes", 2024-09-17T07:30:31.605 INFO:teuthology.orchestra.run.smithi045.stdout: "sortbitwise" 2024-09-17T07:30:31.605 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:30:31.605 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_version": 6, 2024-09-17T07:30:31.605 INFO:teuthology.orchestra.run.smithi045.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T07:30:31.606 INFO:teuthology.orchestra.run.smithi045.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T07:30:31.606 INFO:teuthology.orchestra.run.smithi045.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T07:30:31.606 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_snapshot": "", 2024-09-17T07:30:31.606 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_max": 3, 2024-09-17T07:30:31.606 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd": 8, 2024-09-17T07:30:31.606 INFO:teuthology.orchestra.run.smithi045.stdout: "require_min_compat_client": "luminous", 2024-09-17T07:30:31.606 INFO:teuthology.orchestra.run.smithi045.stdout: "min_compat_client": "luminous", 2024-09-17T07:30:31.606 INFO:teuthology.orchestra.run.smithi045.stdout: "require_osd_release": "squid", 2024-09-17T07:30:31.606 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_crimson": false, 2024-09-17T07:30:31.606 INFO:teuthology.orchestra.run.smithi045.stdout: "pools": [ 2024-09-17T07:30:31.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.606 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 1, 2024-09-17T07:30:31.607 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": ".mgr", 2024-09-17T07:30:31.607 INFO:teuthology.orchestra.run.smithi045.stdout: "create_time": "2024-09-17T07:28:24.767696+0000", 2024-09-17T07:30:31.607 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-17T07:30:31.607 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_names": "hashpspool", 2024-09-17T07:30:31.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": 1, 2024-09-17T07:30:31.607 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 2, 2024-09-17T07:30:31.607 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": 1, 2024-09-17T07:30:31.607 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_rule": 0, 2024-09-17T07:30:31.607 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_count": 0, 2024-09-17T07:30:31.607 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_target": 0, 2024-09-17T07:30:31.607 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T07:30:31.607 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T07:30:31.607 INFO:teuthology.orchestra.run.smithi045.stdout: "is_stretch_pool": false, 2024-09-17T07:30:31.608 INFO:teuthology.orchestra.run.smithi045.stdout: "object_hash": 2, 2024-09-17T07:30:31.608 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscale_mode": "off", 2024-09-17T07:30:31.608 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": 1, 2024-09-17T07:30:31.608 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num": 1, 2024-09-17T07:30:31.608 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num_target": 1, 2024-09-17T07:30:31.608 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_target": 1, 2024-09-17T07:30:31.608 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_pending": 1, 2024-09-17T07:30:31.608 INFO:teuthology.orchestra.run.smithi045.stdout: "last_pg_merge_meta": { 2024-09-17T07:30:31.608 INFO:teuthology.orchestra.run.smithi045.stdout: "source_pgid": "0.0", 2024-09-17T07:30:31.608 INFO:teuthology.orchestra.run.smithi045.stdout: "ready_epoch": 0, 2024-09-17T07:30:31.608 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_started": 0, 2024-09-17T07:30:31.608 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_clean": 0, 2024-09-17T07:30:31.609 INFO:teuthology.orchestra.run.smithi045.stdout: "source_version": "0'0", 2024-09-17T07:30:31.609 INFO:teuthology.orchestra.run.smithi045.stdout: "target_version": "0'0" 2024-09-17T07:30:31.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.609 INFO:teuthology.orchestra.run.smithi045.stdout: "last_change": "18", 2024-09-17T07:30:31.609 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend": "0", 2024-09-17T07:30:31.609 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T07:30:31.609 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T07:30:31.609 INFO:teuthology.orchestra.run.smithi045.stdout: "auid": 0, 2024-09-17T07:30:31.609 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_mode": "selfmanaged", 2024-09-17T07:30:31.609 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_seq": 0, 2024-09-17T07:30:31.609 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_epoch": 0, 2024-09-17T07:30:31.609 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_snaps": [], 2024-09-17T07:30:31.609 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps": "[]", 2024-09-17T07:30:31.609 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_bytes": 0, 2024-09-17T07:30:31.610 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_objects": 0, 2024-09-17T07:30:31.610 INFO:teuthology.orchestra.run.smithi045.stdout: "tiers": [], 2024-09-17T07:30:31.610 INFO:teuthology.orchestra.run.smithi045.stdout: "tier_of": -1, 2024-09-17T07:30:31.610 INFO:teuthology.orchestra.run.smithi045.stdout: "read_tier": -1, 2024-09-17T07:30:31.610 INFO:teuthology.orchestra.run.smithi045.stdout: "write_tier": -1, 2024-09-17T07:30:31.610 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_mode": "none", 2024-09-17T07:30:31.610 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_bytes": 0, 2024-09-17T07:30:31.610 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_objects": 0, 2024-09-17T07:30:31.610 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T07:30:31.610 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T07:30:31.610 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T07:30:31.610 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_flush_age": 0, 2024-09-17T07:30:31.610 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_evict_age": 0, 2024-09-17T07:30:31.611 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profile": "", 2024-09-17T07:30:31.611 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_params": { 2024-09-17T07:30:31.611 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "none" 2024-09-17T07:30:31.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.611 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_period": 0, 2024-09-17T07:30:31.611 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_count": 0, 2024-09-17T07:30:31.611 INFO:teuthology.orchestra.run.smithi045.stdout: "use_gmt_hitset": true, 2024-09-17T07:30:31.611 INFO:teuthology.orchestra.run.smithi045.stdout: "min_read_recency_for_promote": 0, 2024-09-17T07:30:31.611 INFO:teuthology.orchestra.run.smithi045.stdout: "min_write_recency_for_promote": 0, 2024-09-17T07:30:31.611 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T07:30:31.611 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_search_last_n": 0, 2024-09-17T07:30:31.611 INFO:teuthology.orchestra.run.smithi045.stdout: "grade_table": [], 2024-09-17T07:30:31.611 INFO:teuthology.orchestra.run.smithi045.stdout: "stripe_width": 0, 2024-09-17T07:30:31.612 INFO:teuthology.orchestra.run.smithi045.stdout: "expected_num_objects": 0, 2024-09-17T07:30:31.612 INFO:teuthology.orchestra.run.smithi045.stdout: "fast_read": false, 2024-09-17T07:30:31.612 INFO:teuthology.orchestra.run.smithi045.stdout: "options": { 2024-09-17T07:30:31.612 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_max": 32, 2024-09-17T07:30:31.612 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_min": 1 2024-09-17T07:30:31.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.612 INFO:teuthology.orchestra.run.smithi045.stdout: "application_metadata": { 2024-09-17T07:30:31.612 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": {} 2024-09-17T07:30:31.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.612 INFO:teuthology.orchestra.run.smithi045.stdout: "read_balance": { 2024-09-17T07:30:31.612 INFO:teuthology.orchestra.run.smithi045.stdout: "score_type": "Fair distribution", 2024-09-17T07:30:31.612 INFO:teuthology.orchestra.run.smithi045.stdout: "score_acting": 8, 2024-09-17T07:30:31.612 INFO:teuthology.orchestra.run.smithi045.stdout: "score_stable": 8, 2024-09-17T07:30:31.613 INFO:teuthology.orchestra.run.smithi045.stdout: "optimal_score": 0.25, 2024-09-17T07:30:31.613 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_acting": 2, 2024-09-17T07:30:31.613 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_stable": 2, 2024-09-17T07:30:31.613 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity_weighted": 1, 2024-09-17T07:30:31.613 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity": 1, 2024-09-17T07:30:31.613 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity_weighted": 1 2024-09-17T07:30:31.613 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.613 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 2, 2024-09-17T07:30:31.613 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": "rbd", 2024-09-17T07:30:31.613 INFO:teuthology.orchestra.run.smithi045.stdout: "create_time": "2024-09-17T07:28:30.249487+0000", 2024-09-17T07:30:31.613 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 8193, 2024-09-17T07:30:31.614 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T07:30:31.614 INFO:teuthology.orchestra.run.smithi045.stdout: "type": 1, 2024-09-17T07:30:31.614 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 2, 2024-09-17T07:30:31.614 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": 1, 2024-09-17T07:30:31.614 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_rule": 0, 2024-09-17T07:30:31.614 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_count": 0, 2024-09-17T07:30:31.614 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_target": 0, 2024-09-17T07:30:31.614 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T07:30:31.614 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T07:30:31.614 INFO:teuthology.orchestra.run.smithi045.stdout: "is_stretch_pool": false, 2024-09-17T07:30:31.614 INFO:teuthology.orchestra.run.smithi045.stdout: "object_hash": 2, 2024-09-17T07:30:31.614 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscale_mode": "off", 2024-09-17T07:30:31.614 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": 8, 2024-09-17T07:30:31.615 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num": 8, 2024-09-17T07:30:31.615 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num_target": 8, 2024-09-17T07:30:31.615 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_target": 8, 2024-09-17T07:30:31.615 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_pending": 8, 2024-09-17T07:30:31.615 INFO:teuthology.orchestra.run.smithi045.stdout: "last_pg_merge_meta": { 2024-09-17T07:30:31.615 INFO:teuthology.orchestra.run.smithi045.stdout: "source_pgid": "0.0", 2024-09-17T07:30:31.615 INFO:teuthology.orchestra.run.smithi045.stdout: "ready_epoch": 0, 2024-09-17T07:30:31.615 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_started": 0, 2024-09-17T07:30:31.615 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_clean": 0, 2024-09-17T07:30:31.615 INFO:teuthology.orchestra.run.smithi045.stdout: "source_version": "0'0", 2024-09-17T07:30:31.615 INFO:teuthology.orchestra.run.smithi045.stdout: "target_version": "0'0" 2024-09-17T07:30:31.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.615 INFO:teuthology.orchestra.run.smithi045.stdout: "last_change": "22", 2024-09-17T07:30:31.615 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend": "0", 2024-09-17T07:30:31.616 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T07:30:31.616 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T07:30:31.616 INFO:teuthology.orchestra.run.smithi045.stdout: "auid": 0, 2024-09-17T07:30:31.616 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_mode": "selfmanaged", 2024-09-17T07:30:31.616 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_seq": 2, 2024-09-17T07:30:31.616 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_epoch": 22, 2024-09-17T07:30:31.616 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_snaps": [], 2024-09-17T07:30:31.616 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps": "[]", 2024-09-17T07:30:31.616 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_bytes": 0, 2024-09-17T07:30:31.616 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_objects": 0, 2024-09-17T07:30:31.616 INFO:teuthology.orchestra.run.smithi045.stdout: "tiers": [], 2024-09-17T07:30:31.616 INFO:teuthology.orchestra.run.smithi045.stdout: "tier_of": -1, 2024-09-17T07:30:31.616 INFO:teuthology.orchestra.run.smithi045.stdout: "read_tier": -1, 2024-09-17T07:30:31.617 INFO:teuthology.orchestra.run.smithi045.stdout: "write_tier": -1, 2024-09-17T07:30:31.617 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_mode": "none", 2024-09-17T07:30:31.617 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_bytes": 0, 2024-09-17T07:30:31.617 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_objects": 0, 2024-09-17T07:30:31.617 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T07:30:31.617 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T07:30:31.617 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T07:30:31.617 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_flush_age": 0, 2024-09-17T07:30:31.617 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_evict_age": 0, 2024-09-17T07:30:31.617 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profile": "", 2024-09-17T07:30:31.617 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_params": { 2024-09-17T07:30:31.617 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "none" 2024-09-17T07:30:31.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.617 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_period": 0, 2024-09-17T07:30:31.618 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_count": 0, 2024-09-17T07:30:31.618 INFO:teuthology.orchestra.run.smithi045.stdout: "use_gmt_hitset": true, 2024-09-17T07:30:31.618 INFO:teuthology.orchestra.run.smithi045.stdout: "min_read_recency_for_promote": 0, 2024-09-17T07:30:31.618 INFO:teuthology.orchestra.run.smithi045.stdout: "min_write_recency_for_promote": 0, 2024-09-17T07:30:31.618 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T07:30:31.618 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_search_last_n": 0, 2024-09-17T07:30:31.618 INFO:teuthology.orchestra.run.smithi045.stdout: "grade_table": [], 2024-09-17T07:30:31.618 INFO:teuthology.orchestra.run.smithi045.stdout: "stripe_width": 0, 2024-09-17T07:30:31.618 INFO:teuthology.orchestra.run.smithi045.stdout: "expected_num_objects": 0, 2024-09-17T07:30:31.618 INFO:teuthology.orchestra.run.smithi045.stdout: "fast_read": false, 2024-09-17T07:30:31.618 INFO:teuthology.orchestra.run.smithi045.stdout: "options": {}, 2024-09-17T07:30:31.618 INFO:teuthology.orchestra.run.smithi045.stdout: "application_metadata": { 2024-09-17T07:30:31.618 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd": {} 2024-09-17T07:30:31.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.619 INFO:teuthology.orchestra.run.smithi045.stdout: "read_balance": { 2024-09-17T07:30:31.619 INFO:teuthology.orchestra.run.smithi045.stdout: "score_type": "Fair distribution", 2024-09-17T07:30:31.619 INFO:teuthology.orchestra.run.smithi045.stdout: "score_acting": 2.9900000095367432, 2024-09-17T07:30:31.619 INFO:teuthology.orchestra.run.smithi045.stdout: "score_stable": 2.9900000095367432, 2024-09-17T07:30:31.619 INFO:teuthology.orchestra.run.smithi045.stdout: "optimal_score": 0.87999999523162842, 2024-09-17T07:30:31.619 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_acting": 2.630000114440918, 2024-09-17T07:30:31.619 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_stable": 2.630000114440918, 2024-09-17T07:30:31.620 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity_weighted": 1, 2024-09-17T07:30:31.620 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity": 1, 2024-09-17T07:30:31.620 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity_weighted": 1 2024-09-17T07:30:31.620 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.621 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 3, 2024-09-17T07:30:31.621 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": "pool_client.0", 2024-09-17T07:30:31.621 INFO:teuthology.orchestra.run.smithi045.stdout: "create_time": "2024-09-17T07:28:41.831004+0000", 2024-09-17T07:30:31.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 8193, 2024-09-17T07:30:31.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T07:30:31.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type": 1, 2024-09-17T07:30:31.621 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 2, 2024-09-17T07:30:31.622 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": 1, 2024-09-17T07:30:31.622 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_rule": 0, 2024-09-17T07:30:31.622 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_count": 0, 2024-09-17T07:30:31.622 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_target": 0, 2024-09-17T07:30:31.622 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T07:30:31.622 INFO:teuthology.orchestra.run.smithi045.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T07:30:31.622 INFO:teuthology.orchestra.run.smithi045.stdout: "is_stretch_pool": false, 2024-09-17T07:30:31.622 INFO:teuthology.orchestra.run.smithi045.stdout: "object_hash": 2, 2024-09-17T07:30:31.623 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscale_mode": "off", 2024-09-17T07:30:31.623 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": 32, 2024-09-17T07:30:31.623 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num": 32, 2024-09-17T07:30:31.623 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_placement_num_target": 32, 2024-09-17T07:30:31.623 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_target": 32, 2024-09-17T07:30:31.623 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num_pending": 32, 2024-09-17T07:30:31.623 INFO:teuthology.orchestra.run.smithi045.stdout: "last_pg_merge_meta": { 2024-09-17T07:30:31.623 INFO:teuthology.orchestra.run.smithi045.stdout: "source_pgid": "0.0", 2024-09-17T07:30:31.624 INFO:teuthology.orchestra.run.smithi045.stdout: "ready_epoch": 0, 2024-09-17T07:30:31.624 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_started": 0, 2024-09-17T07:30:31.624 INFO:teuthology.orchestra.run.smithi045.stdout: "last_epoch_clean": 0, 2024-09-17T07:30:31.624 INFO:teuthology.orchestra.run.smithi045.stdout: "source_version": "0'0", 2024-09-17T07:30:31.624 INFO:teuthology.orchestra.run.smithi045.stdout: "target_version": "0'0" 2024-09-17T07:30:31.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.624 INFO:teuthology.orchestra.run.smithi045.stdout: "last_change": "120", 2024-09-17T07:30:31.624 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend": "0", 2024-09-17T07:30:31.625 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T07:30:31.625 INFO:teuthology.orchestra.run.smithi045.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T07:30:31.625 INFO:teuthology.orchestra.run.smithi045.stdout: "auid": 0, 2024-09-17T07:30:31.625 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_mode": "selfmanaged", 2024-09-17T07:30:31.625 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_seq": 48, 2024-09-17T07:30:31.625 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_epoch": 120, 2024-09-17T07:30:31.625 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_snaps": [], 2024-09-17T07:30:31.625 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps": "[]", 2024-09-17T07:30:31.626 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_bytes": 0, 2024-09-17T07:30:31.626 INFO:teuthology.orchestra.run.smithi045.stdout: "quota_max_objects": 0, 2024-09-17T07:30:31.626 INFO:teuthology.orchestra.run.smithi045.stdout: "tiers": [], 2024-09-17T07:30:31.626 INFO:teuthology.orchestra.run.smithi045.stdout: "tier_of": -1, 2024-09-17T07:30:31.626 INFO:teuthology.orchestra.run.smithi045.stdout: "read_tier": -1, 2024-09-17T07:30:31.626 INFO:teuthology.orchestra.run.smithi045.stdout: "write_tier": -1, 2024-09-17T07:30:31.626 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_mode": "none", 2024-09-17T07:30:31.626 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_bytes": 0, 2024-09-17T07:30:31.626 INFO:teuthology.orchestra.run.smithi045.stdout: "target_max_objects": 0, 2024-09-17T07:30:31.626 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T07:30:31.626 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T07:30:31.627 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T07:30:31.627 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_flush_age": 0, 2024-09-17T07:30:31.627 INFO:teuthology.orchestra.run.smithi045.stdout: "cache_min_evict_age": 0, 2024-09-17T07:30:31.627 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profile": "", 2024-09-17T07:30:31.627 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_params": { 2024-09-17T07:30:31.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "none" 2024-09-17T07:30:31.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.627 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_period": 0, 2024-09-17T07:30:31.628 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_count": 0, 2024-09-17T07:30:31.628 INFO:teuthology.orchestra.run.smithi045.stdout: "use_gmt_hitset": true, 2024-09-17T07:30:31.628 INFO:teuthology.orchestra.run.smithi045.stdout: "min_read_recency_for_promote": 0, 2024-09-17T07:30:31.628 INFO:teuthology.orchestra.run.smithi045.stdout: "min_write_recency_for_promote": 0, 2024-09-17T07:30:31.628 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T07:30:31.628 INFO:teuthology.orchestra.run.smithi045.stdout: "hit_set_search_last_n": 0, 2024-09-17T07:30:31.628 INFO:teuthology.orchestra.run.smithi045.stdout: "grade_table": [], 2024-09-17T07:30:31.628 INFO:teuthology.orchestra.run.smithi045.stdout: "stripe_width": 0, 2024-09-17T07:30:31.628 INFO:teuthology.orchestra.run.smithi045.stdout: "expected_num_objects": 0, 2024-09-17T07:30:31.629 INFO:teuthology.orchestra.run.smithi045.stdout: "fast_read": false, 2024-09-17T07:30:31.629 INFO:teuthology.orchestra.run.smithi045.stdout: "options": {}, 2024-09-17T07:30:31.629 INFO:teuthology.orchestra.run.smithi045.stdout: "application_metadata": { 2024-09-17T07:30:31.629 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd": {} 2024-09-17T07:30:31.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.629 INFO:teuthology.orchestra.run.smithi045.stdout: "read_balance": { 2024-09-17T07:30:31.629 INFO:teuthology.orchestra.run.smithi045.stdout: "score_type": "Fair distribution", 2024-09-17T07:30:31.629 INFO:teuthology.orchestra.run.smithi045.stdout: "score_acting": 1.5, 2024-09-17T07:30:31.630 INFO:teuthology.orchestra.run.smithi045.stdout: "score_stable": 1.5, 2024-09-17T07:30:31.630 INFO:teuthology.orchestra.run.smithi045.stdout: "optimal_score": 1, 2024-09-17T07:30:31.630 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_acting": 1.5, 2024-09-17T07:30:31.630 INFO:teuthology.orchestra.run.smithi045.stdout: "raw_score_stable": 1.5, 2024-09-17T07:30:31.630 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity_weighted": 1, 2024-09-17T07:30:31.630 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity": 1, 2024-09-17T07:30:31.630 INFO:teuthology.orchestra.run.smithi045.stdout: "average_primary_affinity_weighted": 1 2024-09-17T07:30:31.630 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.631 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.631 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:30:31.631 INFO:teuthology.orchestra.run.smithi045.stdout: "osds": [ 2024-09-17T07:30:31.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.631 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 0, 2024-09-17T07:30:31.631 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "7794e6a8-ebf4-4b7c-bf3d-63cd015234ef", 2024-09-17T07:30:31.631 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:31.632 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-17T07:30:31.632 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0.845458984375, 2024-09-17T07:30:31.632 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:31.632 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-17T07:30:31.632 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-17T07:30:31.632 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 14, 2024-09-17T07:30:31.633 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 120, 2024-09-17T07:30:31.633 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-17T07:30:31.633 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:31.633 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:31.633 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.633 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.633 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6800", 2024-09-17T07:30:31.633 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:31.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.634 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.634 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6801", 2024-09-17T07:30:31.634 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:31.634 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.634 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.634 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:31.634 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.634 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.634 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6802", 2024-09-17T07:30:31.634 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:31.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.635 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6803", 2024-09-17T07:30:31.635 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:31.635 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.635 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.635 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:31.635 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.635 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6806", 2024-09-17T07:30:31.635 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:31.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.636 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6807", 2024-09-17T07:30:31.636 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:31.636 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.636 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.636 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:31.636 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.636 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6804", 2024-09-17T07:30:31.636 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:31.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.637 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6805", 2024-09-17T07:30:31.637 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2039818160 2024-09-17T07:30:31.637 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.637 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.637 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6801/2039818160", 2024-09-17T07:30:31.637 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6803/2039818160", 2024-09-17T07:30:31.637 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6807/2039818160", 2024-09-17T07:30:31.637 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6805/2039818160", 2024-09-17T07:30:31.637 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:31.637 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:31.638 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:31.638 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.638 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 1, 2024-09-17T07:30:31.638 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5", 2024-09-17T07:30:31.638 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:31.638 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-17T07:30:31.638 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-17T07:30:31.638 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:31.638 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-17T07:30:31.638 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-17T07:30:31.638 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 14, 2024-09-17T07:30:31.638 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 121, 2024-09-17T07:30:31.639 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-17T07:30:31.639 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:31.639 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:31.639 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.639 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.639 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6808", 2024-09-17T07:30:31.639 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:31.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.639 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.639 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6809", 2024-09-17T07:30:31.639 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:31.639 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.640 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.640 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:31.640 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.640 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6810", 2024-09-17T07:30:31.640 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:31.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.640 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6811", 2024-09-17T07:30:31.640 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:31.640 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.641 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.641 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:31.641 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.641 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.641 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6814", 2024-09-17T07:30:31.641 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:31.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.641 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.641 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6815", 2024-09-17T07:30:31.641 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:31.641 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.642 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.642 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:31.642 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.642 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.642 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6812", 2024-09-17T07:30:31.642 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:31.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.642 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.642 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6813", 2024-09-17T07:30:31.642 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1158817756 2024-09-17T07:30:31.642 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.643 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.643 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6809/1158817756", 2024-09-17T07:30:31.643 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6811/1158817756", 2024-09-17T07:30:31.643 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6815/1158817756", 2024-09-17T07:30:31.643 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6813/1158817756", 2024-09-17T07:30:31.643 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:31.643 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:31.643 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:31.643 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.644 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 2, 2024-09-17T07:30:31.644 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "195a9db1-568d-459a-bb50-2a1bb2445f53", 2024-09-17T07:30:31.644 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:31.644 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-17T07:30:31.644 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-17T07:30:31.644 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:31.645 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-17T07:30:31.645 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-17T07:30:31.645 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 14, 2024-09-17T07:30:31.645 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 119, 2024-09-17T07:30:31.645 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-17T07:30:31.645 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:31.645 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:31.645 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.646 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.646 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6824", 2024-09-17T07:30:31.646 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:31.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.646 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.646 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6825", 2024-09-17T07:30:31.647 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:31.647 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.647 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.647 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:31.647 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.648 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6826", 2024-09-17T07:30:31.648 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:31.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.648 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6827", 2024-09-17T07:30:31.648 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:31.648 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.649 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.649 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:31.649 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.649 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.649 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6830", 2024-09-17T07:30:31.649 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:31.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.650 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.650 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6831", 2024-09-17T07:30:31.650 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:31.650 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.650 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.650 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:31.650 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.650 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.650 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6828", 2024-09-17T07:30:31.650 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:31.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.651 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.651 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6829", 2024-09-17T07:30:31.651 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3298886784 2024-09-17T07:30:31.651 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.651 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.651 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6825/3298886784", 2024-09-17T07:30:31.651 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6827/3298886784", 2024-09-17T07:30:31.651 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6831/3298886784", 2024-09-17T07:30:31.651 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6829/3298886784", 2024-09-17T07:30:31.651 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:31.651 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:31.652 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:31.652 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.652 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 3, 2024-09-17T07:30:31.652 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "b3fda98b-a10a-4354-905e-1b37a5124b98", 2024-09-17T07:30:31.652 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:31.652 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-17T07:30:31.652 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-17T07:30:31.652 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:31.652 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 14, 2024-09-17T07:30:31.653 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 36, 2024-09-17T07:30:31.653 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 90, 2024-09-17T07:30:31.653 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 110, 2024-09-17T07:30:31.653 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 37, 2024-09-17T07:30:31.653 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:31.653 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:31.653 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.653 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.654 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6816", 2024-09-17T07:30:31.654 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:31.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.654 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.654 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6817", 2024-09-17T07:30:31.654 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:31.654 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.654 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.655 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:31.655 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.655 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.655 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6818", 2024-09-17T07:30:31.655 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:31.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6819", 2024-09-17T07:30:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.657 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.657 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6822", 2024-09-17T07:30:31.657 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:31.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.657 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.657 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6823", 2024-09-17T07:30:31.657 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:31.657 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.657 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.657 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:31.657 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.658 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6820", 2024-09-17T07:30:31.658 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:31.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.658 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6821", 2024-09-17T07:30:31.658 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 724784918 2024-09-17T07:30:31.658 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.658 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.658 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.45:6817/724784918", 2024-09-17T07:30:31.658 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.45:6819/724784918", 2024-09-17T07:30:31.659 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.45:6823/724784918", 2024-09-17T07:30:31.659 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.45:6821/724784918", 2024-09-17T07:30:31.659 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:31.659 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:31.659 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:31.659 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.659 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 4, 2024-09-17T07:30:31.659 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "6ab67b48-2d83-4e68-8898-302a1adc0a52", 2024-09-17T07:30:31.659 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:31.659 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-17T07:30:31.659 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0.783294677734375, 2024-09-17T07:30:31.659 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:31.660 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-17T07:30:31.660 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-17T07:30:31.660 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 14, 2024-09-17T07:30:31.660 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 121, 2024-09-17T07:30:31.660 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-17T07:30:31.660 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:31.660 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:31.660 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.660 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.660 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6824", 2024-09-17T07:30:31.660 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:31.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.661 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6825", 2024-09-17T07:30:31.661 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:31.661 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.661 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.661 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:31.661 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.661 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6826", 2024-09-17T07:30:31.661 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:31.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.662 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6827", 2024-09-17T07:30:31.662 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:31.662 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.662 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.662 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:31.662 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.662 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6830", 2024-09-17T07:30:31.662 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:31.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.663 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.663 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6831", 2024-09-17T07:30:31.663 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:31.663 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.663 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.663 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:31.663 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.663 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.663 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6828", 2024-09-17T07:30:31.663 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:31.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.664 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.664 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6829", 2024-09-17T07:30:31.664 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1644784047 2024-09-17T07:30:31.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.664 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.664 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.192:6825/1644784047", 2024-09-17T07:30:31.664 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.192:6827/1644784047", 2024-09-17T07:30:31.664 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.192:6831/1644784047", 2024-09-17T07:30:31.664 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.192:6829/1644784047", 2024-09-17T07:30:31.664 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:31.664 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:31.665 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:31.665 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.665 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 5, 2024-09-17T07:30:31.665 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "ae3c787c-63d2-4e88-8f99-6d4f19837a3a", 2024-09-17T07:30:31.665 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:31.665 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-17T07:30:31.665 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-17T07:30:31.665 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:31.665 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-17T07:30:31.665 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-17T07:30:31.665 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 14, 2024-09-17T07:30:31.665 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 121, 2024-09-17T07:30:31.666 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-17T07:30:31.666 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:31.666 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:31.666 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.666 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6816", 2024-09-17T07:30:31.666 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:31.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.666 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6817", 2024-09-17T07:30:31.666 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:31.666 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.667 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.667 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:31.667 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.667 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6818", 2024-09-17T07:30:31.667 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:31.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.668 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6819", 2024-09-17T07:30:31.668 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:31.668 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.668 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.668 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:31.668 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.668 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.669 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6822", 2024-09-17T07:30:31.669 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:31.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.669 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6823", 2024-09-17T07:30:31.669 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:31.670 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.670 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.670 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:31.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.670 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6820", 2024-09-17T07:30:31.671 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:31.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.671 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.671 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6821", 2024-09-17T07:30:31.671 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4077426583 2024-09-17T07:30:31.671 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.671 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.672 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.192:6817/4077426583", 2024-09-17T07:30:31.672 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.192:6819/4077426583", 2024-09-17T07:30:31.672 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.192:6823/4077426583", 2024-09-17T07:30:31.672 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.192:6821/4077426583", 2024-09-17T07:30:31.672 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:31.672 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:31.672 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:31.673 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.673 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 6, 2024-09-17T07:30:31.673 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "f792f18d-ad9e-49a8-b194-24522398fb25", 2024-09-17T07:30:31.673 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:31.673 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-17T07:30:31.673 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 1, 2024-09-17T07:30:31.673 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:31.674 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-17T07:30:31.674 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-17T07:30:31.674 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 14, 2024-09-17T07:30:31.674 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 121, 2024-09-17T07:30:31.674 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-17T07:30:31.674 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:31.674 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:31.674 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.674 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.674 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6808", 2024-09-17T07:30:31.674 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:31.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.675 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6809", 2024-09-17T07:30:31.675 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:31.675 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.675 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.675 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:31.675 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.675 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6810", 2024-09-17T07:30:31.675 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:31.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.676 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.676 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6811", 2024-09-17T07:30:31.676 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:31.676 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.676 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.676 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:31.676 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.676 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.676 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6814", 2024-09-17T07:30:31.676 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:31.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.677 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6815", 2024-09-17T07:30:31.677 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:31.677 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.677 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.678 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:31.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.678 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.678 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6812", 2024-09-17T07:30:31.678 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:31.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.678 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.679 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6813", 2024-09-17T07:30:31.679 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2922229639 2024-09-17T07:30:31.679 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.679 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.679 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.192:6809/2922229639", 2024-09-17T07:30:31.679 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.192:6811/2922229639", 2024-09-17T07:30:31.679 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.192:6815/2922229639", 2024-09-17T07:30:31.680 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.192:6813/2922229639", 2024-09-17T07:30:31.680 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:31.680 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:31.680 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:31.680 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.681 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 7, 2024-09-17T07:30:31.681 INFO:teuthology.orchestra.run.smithi045.stdout: "uuid": "4a589d43-b0b9-42de-989c-46928d693c24", 2024-09-17T07:30:31.681 INFO:teuthology.orchestra.run.smithi045.stdout: "up": 1, 2024-09-17T07:30:31.681 INFO:teuthology.orchestra.run.smithi045.stdout: "in": 1, 2024-09-17T07:30:31.681 INFO:teuthology.orchestra.run.smithi045.stdout: "weight": 0.8525848388671875, 2024-09-17T07:30:31.681 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_affinity": 1, 2024-09-17T07:30:31.681 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_begin": 0, 2024-09-17T07:30:31.681 INFO:teuthology.orchestra.run.smithi045.stdout: "last_clean_end": 0, 2024-09-17T07:30:31.682 INFO:teuthology.orchestra.run.smithi045.stdout: "up_from": 14, 2024-09-17T07:30:31.682 INFO:teuthology.orchestra.run.smithi045.stdout: "up_thru": 121, 2024-09-17T07:30:31.682 INFO:teuthology.orchestra.run.smithi045.stdout: "down_at": 0, 2024-09-17T07:30:31.682 INFO:teuthology.orchestra.run.smithi045.stdout: "lost_at": 0, 2024-09-17T07:30:31.682 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addrs": { 2024-09-17T07:30:31.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.682 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6800", 2024-09-17T07:30:31.683 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:31.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6801", 2024-09-17T07:30:31.683 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:31.683 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.683 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.683 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addrs": { 2024-09-17T07:30:31.683 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6802", 2024-09-17T07:30:31.684 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:31.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6803", 2024-09-17T07:30:31.684 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:31.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.684 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.684 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addrs": { 2024-09-17T07:30:31.684 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.685 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.685 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6806", 2024-09-17T07:30:31.685 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:31.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.685 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.685 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6807", 2024-09-17T07:30:31.685 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:31.685 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.685 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.685 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addrs": { 2024-09-17T07:30:31.685 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-17T07:30:31.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-17T07:30:31.686 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6804", 2024-09-17T07:30:31.686 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:31.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-17T07:30:31.686 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.192:6805", 2024-09-17T07:30:31.686 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 780298944 2024-09-17T07:30:31.686 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.686 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.686 INFO:teuthology.orchestra.run.smithi045.stdout: "public_addr": "172.21.15.192:6801/780298944", 2024-09-17T07:30:31.686 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_addr": "172.21.15.192:6803/780298944", 2024-09-17T07:30:31.686 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_back_addr": "172.21.15.192:6807/780298944", 2024-09-17T07:30:31.687 INFO:teuthology.orchestra.run.smithi045.stdout: "heartbeat_front_addr": "172.21.15.192:6805/780298944", 2024-09-17T07:30:31.687 INFO:teuthology.orchestra.run.smithi045.stdout: "state": [ 2024-09-17T07:30:31.687 INFO:teuthology.orchestra.run.smithi045.stdout: "exists", 2024-09-17T07:30:31.687 INFO:teuthology.orchestra.run.smithi045.stdout: "up" 2024-09-17T07:30:31.687 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.687 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.687 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:30:31.687 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_xinfo": [ 2024-09-17T07:30:31.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.687 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 0, 2024-09-17T07:30:31.687 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:31.687 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:31.687 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:31.687 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:31.688 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-17T07:30:31.688 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:20.586514+0000", 2024-09-17T07:30:31.688 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-17T07:30:31.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.688 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 1, 2024-09-17T07:30:31.688 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:31.688 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:31.688 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:31.688 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:31.688 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-17T07:30:31.688 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:20.833505+0000", 2024-09-17T07:30:31.688 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-17T07:30:31.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 2, 2024-09-17T07:30:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-17T07:30:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:21.059899+0000", 2024-09-17T07:30:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-17T07:30:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 3, 2024-09-17T07:30:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:31.690 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:31.690 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:31.690 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-17T07:30:31.690 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:20.499470+0000", 2024-09-17T07:30:31.690 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 37 2024-09-17T07:30:31.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.690 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 4, 2024-09-17T07:30:31.690 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:31.690 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:31.690 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:31.690 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:31.690 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-17T07:30:31.690 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:21.353342+0000", 2024-09-17T07:30:31.691 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-17T07:30:31.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.691 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 5, 2024-09-17T07:30:31.691 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:31.691 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:31.691 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:31.691 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:31.691 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-17T07:30:31.691 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:20.586673+0000", 2024-09-17T07:30:31.691 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-17T07:30:31.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.691 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 6, 2024-09-17T07:30:31.692 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:31.692 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:31.692 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:31.692 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:31.692 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-17T07:30:31.692 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:20.767847+0000", 2024-09-17T07:30:31.692 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-17T07:30:31.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.692 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": 7, 2024-09-17T07:30:31.692 INFO:teuthology.orchestra.run.smithi045.stdout: "down_stamp": "0.000000", 2024-09-17T07:30:31.692 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_probability": 0, 2024-09-17T07:30:31.692 INFO:teuthology.orchestra.run.smithi045.stdout: "laggy_interval": 0, 2024-09-17T07:30:31.693 INFO:teuthology.orchestra.run.smithi045.stdout: "features": 4540719139924082687, 2024-09-17T07:30:31.693 INFO:teuthology.orchestra.run.smithi045.stdout: "old_weight": 0, 2024-09-17T07:30:31.693 INFO:teuthology.orchestra.run.smithi045.stdout: "last_purged_snaps_scrub": "2024-09-17T07:28:20.859615+0000", 2024-09-17T07:30:31.693 INFO:teuthology.orchestra.run.smithi045.stdout: "dead_epoch": 0 2024-09-17T07:30:31.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.693 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:30:31.693 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_upmap": [ 2024-09-17T07:30:31.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.693 INFO:teuthology.orchestra.run.smithi045.stdout: "pgid": "2.4", 2024-09-17T07:30:31.693 INFO:teuthology.orchestra.run.smithi045.stdout: "osds": [ 2024-09-17T07:30:31.693 INFO:teuthology.orchestra.run.smithi045.stdout: 5, 2024-09-17T07:30:31.693 INFO:teuthology.orchestra.run.smithi045.stdout: 3 2024-09-17T07:30:31.693 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.694 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:30:31.694 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_upmap_items": [], 2024-09-17T07:30:31.694 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_upmap_primaries": [], 2024-09-17T07:30:31.694 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_temp": [], 2024-09-17T07:30:31.694 INFO:teuthology.orchestra.run.smithi045.stdout: "primary_temp": [], 2024-09-17T07:30:31.694 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklist": {}, 2024-09-17T07:30:31.694 INFO:teuthology.orchestra.run.smithi045.stdout: "range_blocklist": {}, 2024-09-17T07:30:31.694 INFO:teuthology.orchestra.run.smithi045.stdout: "erasure_code_profiles": { 2024-09-17T07:30:31.694 INFO:teuthology.orchestra.run.smithi045.stdout: "default": { 2024-09-17T07:30:31.694 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-failure-domain": "osd", 2024-09-17T07:30:31.694 INFO:teuthology.orchestra.run.smithi045.stdout: "k": "2", 2024-09-17T07:30:31.694 INFO:teuthology.orchestra.run.smithi045.stdout: "m": "1", 2024-09-17T07:30:31.694 INFO:teuthology.orchestra.run.smithi045.stdout: "plugin": "jerasure", 2024-09-17T07:30:31.694 INFO:teuthology.orchestra.run.smithi045.stdout: "technique": "reed_sol_van" 2024-09-17T07:30:31.695 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:31.695 INFO:teuthology.orchestra.run.smithi045.stdout: "removed_snaps_queue": [ 2024-09-17T07:30:31.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.695 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 3, 2024-09-17T07:30:31.695 INFO:teuthology.orchestra.run.smithi045.stdout: "snaps": [ 2024-09-17T07:30:31.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:31.695 INFO:teuthology.orchestra.run.smithi045.stdout: "begin": 34, 2024-09-17T07:30:31.695 INFO:teuthology.orchestra.run.smithi045.stdout: "length": 4 2024-09-17T07:30:31.695 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.695 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:31.695 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.695 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-17T07:30:31.695 INFO:teuthology.orchestra.run.smithi045.stdout: "new_removed_snaps": [], 2024-09-17T07:30:31.696 INFO:teuthology.orchestra.run.smithi045.stdout: "new_purged_snaps": [], 2024-09-17T07:30:31.696 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_node_flags": {}, 2024-09-17T07:30:31.696 INFO:teuthology.orchestra.run.smithi045.stdout: "device_class_flags": {}, 2024-09-17T07:30:31.696 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode": { 2024-09-17T07:30:31.696 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode_enabled": false, 2024-09-17T07:30:31.696 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_bucket_count": 0, 2024-09-17T07:30:31.696 INFO:teuthology.orchestra.run.smithi045.stdout: "degraded_stretch_mode": 0, 2024-09-17T07:30:31.696 INFO:teuthology.orchestra.run.smithi045.stdout: "recovering_stretch_mode": 0, 2024-09-17T07:30:31.696 INFO:teuthology.orchestra.run.smithi045.stdout: "stretch_mode_bucket": 0 2024-09-17T07:30:31.696 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:31.696 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:31.699 INFO:tasks.thrashosds.thrasher:j is {'epoch': 122, 'fsid': 'b283e6d5-0f4a-4271-a443-15e695e35364', 'created': '2024-09-17T07:28:12.276015+0000', 'modified': '2024-09-17T07:30:31.082939+0000', 'last_up_change': '2024-09-17T07:29:58.120417+0000', 'last_in_change': '2024-09-17T07:30:26.234253+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T07:28:24.767696+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T07:28:30.249487+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T07:28:41.831004+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 48, 'snap_epoch': 120, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '7794e6a8-ebf4-4b7c-bf3d-63cd015234ef', 'up': 1, 'in': 1, 'weight': 0.845458984375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 120, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6800', 'nonce': 2039818160}, {'type': 'v1', 'addr': '172.21.15.45:6801', 'nonce': 2039818160}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6802', 'nonce': 2039818160}, {'type': 'v1', 'addr': '172.21.15.45:6803', 'nonce': 2039818160}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6806', 'nonce': 2039818160}, {'type': 'v1', 'addr': '172.21.15.45:6807', 'nonce': 2039818160}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6804', 'nonce': 2039818160}, {'type': 'v1', 'addr': '172.21.15.45:6805', 'nonce': 2039818160}]}, 'public_addr': '172.21.15.45:6801/2039818160', 'cluster_addr': '172.21.15.45:6803/2039818160', 'heartbeat_back_addr': '172.21.15.45:6807/2039818160', 'heartbeat_front_addr': '172.21.15.45:6805/2039818160', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6808', 'nonce': 1158817756}, {'type': 'v1', 'addr': '172.21.15.45:6809', 'nonce': 1158817756}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6810', 'nonce': 1158817756}, {'type': 'v1', 'addr': '172.21.15.45:6811', 'nonce': 1158817756}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6814', 'nonce': 1158817756}, {'type': 'v1', 'addr': '172.21.15.45:6815', 'nonce': 1158817756}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6812', 'nonce': 1158817756}, {'type': 'v1', 'addr': '172.21.15.45:6813', 'nonce': 1158817756}]}, 'public_addr': '172.21.15.45:6809/1158817756', 'cluster_addr': '172.21.15.45:6811/1158817756', 'heartbeat_back_addr': '172.21.15.45:6815/1158817756', 'heartbeat_front_addr': '172.21.15.45:6813/1158817756', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '195a9db1-568d-459a-bb50-2a1bb2445f53', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 119, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6824', 'nonce': 3298886784}, {'type': 'v1', 'addr': '172.21.15.45:6825', 'nonce': 3298886784}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6826', 'nonce': 3298886784}, {'type': 'v1', 'addr': '172.21.15.45:6827', 'nonce': 3298886784}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6830', 'nonce': 3298886784}, {'type': 'v1', 'addr': '172.21.15.45:6831', 'nonce': 3298886784}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6828', 'nonce': 3298886784}, {'type': 'v1', 'addr': '172.21.15.45:6829', 'nonce': 3298886784}]}, 'public_addr': '172.21.15.45:6825/3298886784', 'cluster_addr': '172.21.15.45:6827/3298886784', 'heartbeat_back_addr': '172.21.15.45:6831/3298886784', 'heartbeat_front_addr': '172.21.15.45:6829/3298886784', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b3fda98b-a10a-4354-905e-1b37a5124b98', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 36, 'up_from': 90, 'up_thru': 110, 'down_at': 37, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6816', 'nonce': 724784918}, {'type': 'v1', 'addr': '172.21.15.45:6817', 'nonce': 724784918}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6818', 'nonce': 724784918}, {'type': 'v1', 'addr': '172.21.15.45:6819', 'nonce': 724784918}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6822', 'nonce': 724784918}, {'type': 'v1', 'addr': '172.21.15.45:6823', 'nonce': 724784918}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6820', 'nonce': 724784918}, {'type': 'v1', 'addr': '172.21.15.45:6821', 'nonce': 724784918}]}, 'public_addr': '172.21.15.45:6817/724784918', 'cluster_addr': '172.21.15.45:6819/724784918', 'heartbeat_back_addr': '172.21.15.45:6823/724784918', 'heartbeat_front_addr': '172.21.15.45:6821/724784918', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '6ab67b48-2d83-4e68-8898-302a1adc0a52', 'up': 1, 'in': 1, 'weight': 0.783294677734375, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6824', 'nonce': 1644784047}, {'type': 'v1', 'addr': '172.21.15.192:6825', 'nonce': 1644784047}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6826', 'nonce': 1644784047}, {'type': 'v1', 'addr': '172.21.15.192:6827', 'nonce': 1644784047}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6830', 'nonce': 1644784047}, {'type': 'v1', 'addr': '172.21.15.192:6831', 'nonce': 1644784047}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6828', 'nonce': 1644784047}, {'type': 'v1', 'addr': '172.21.15.192:6829', 'nonce': 1644784047}]}, 'public_addr': '172.21.15.192:6825/1644784047', 'cluster_addr': '172.21.15.192:6827/1644784047', 'heartbeat_back_addr': '172.21.15.192:6831/1644784047', 'heartbeat_front_addr': '172.21.15.192:6829/1644784047', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ae3c787c-63d2-4e88-8f99-6d4f19837a3a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 4077426583}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 4077426583}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 4077426583}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 4077426583}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 4077426583}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 4077426583}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 4077426583}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 4077426583}]}, 'public_addr': '172.21.15.192:6817/4077426583', 'cluster_addr': '172.21.15.192:6819/4077426583', 'heartbeat_back_addr': '172.21.15.192:6823/4077426583', 'heartbeat_front_addr': '172.21.15.192:6821/4077426583', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'f792f18d-ad9e-49a8-b194-24522398fb25', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 2922229639}, {'type': 'v1', 'addr': '172.21.15.192:6809', 'nonce': 2922229639}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6810', 'nonce': 2922229639}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 2922229639}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6814', 'nonce': 2922229639}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 2922229639}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 2922229639}, {'type': 'v1', 'addr': '172.21.15.192:6813', 'nonce': 2922229639}]}, 'public_addr': '172.21.15.192:6809/2922229639', 'cluster_addr': '172.21.15.192:6811/2922229639', 'heartbeat_back_addr': '172.21.15.192:6815/2922229639', 'heartbeat_front_addr': '172.21.15.192:6813/2922229639', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '4a589d43-b0b9-42de-989c-46928d693c24', 'up': 1, 'in': 1, 'weight': 0.8525848388671875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 780298944}, {'type': 'v1', 'addr': '172.21.15.192:6801', 'nonce': 780298944}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6802', 'nonce': 780298944}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 780298944}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6806', 'nonce': 780298944}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 780298944}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 780298944}, {'type': 'v1', 'addr': '172.21.15.192:6805', 'nonce': 780298944}]}, 'public_addr': '172.21.15.192:6801/780298944', 'cluster_addr': '172.21.15.192:6803/780298944', 'heartbeat_back_addr': '172.21.15.192:6807/780298944', 'heartbeat_front_addr': '172.21.15.192:6805/780298944', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T07:28:20.586514+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T07:28:20.833505+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T07:28:21.059899+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T07:28:20.499470+0000', 'dead_epoch': 37}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T07:28:21.353342+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T07:28:20.586673+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T07:28:20.767847+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T07:28:20.859615+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.4', 'osds': [5, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 34, 'length': 4}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T07:30:31.699 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:30:31.700 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:31.700 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:31.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:31.700 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:31.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.700 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:31.700 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:31.701 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.763406641000003, 2024-09-17T07:30:31.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:30:31.701 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.701 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.701 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.701 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:30:31.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.702 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:31.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:31.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:31.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:30:31.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:30:31.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:30:31.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:30:31.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:30:31.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:30:31.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:30:31.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:30:31.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:30:31.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:30:31.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:30:31.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:30:31.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:30:31.706 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.706 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.706 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.763387964000003, 2024-09-17T07:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:31.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:31.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:30:31.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:31.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:30:31.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:30:31.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:30:31.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:30:31.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:30:31.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:30:31.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:30:31.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:30:31.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:30:31.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:30:31.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:30:31.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:30:31.711 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.711 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.711 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.711 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:31.711 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:31.711 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.763369611000002, 2024-09-17T07:30:31.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:30:31.711 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.711 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.711 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.712 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.712 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.712 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.712 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:30:31.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.712 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:31.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:31.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:31.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:30:31.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:30:31.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:30:31.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:30:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:30:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:30:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:30:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:30:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:30:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:30:31.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:30:31.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:30:31.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:30:31.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.715 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.715 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:31.716 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:31.716 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.763350012999993, 2024-09-17T07:30:31.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:30:31.716 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.716 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.716 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.716 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:30:31.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.716 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:31.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:31.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:30:31.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:30:31.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:30:31.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:30:31.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:30:31.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:30:31.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:30:31.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:30:31.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:30:31.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:30:31.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:30:31.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:30:31.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:30:31.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:30:31.720 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.720 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.720 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.720 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:31.720 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:31.720 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.763330057999994, 2024-09-17T07:30:31.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:30:31.720 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.720 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.720 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.721 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:30:31.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.721 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:31.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:31.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:31.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:30:31.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:30:31.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:30:31.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:30:31.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:30:31.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:30:31.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:30:31.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:30:31.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:30:31.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:30:31.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:30:31.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:30:31.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:30:31.724 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.724 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.724 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.725 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:31.725 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:31.725 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.763311403000003, 2024-09-17T07:30:31.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:30:31.725 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.725 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.725 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.726 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.726 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.726 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.726 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:30:31.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.726 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:31.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:31.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:31.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:30:31.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:30:31.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:30:31.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:30:31.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:30:31.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:30:31.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:30:31.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:30:31.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:30:31.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:30:31.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:30:31.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:30:31.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:30:31.731 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.731 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.731 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.732 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:31.732 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:31.732 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.763292926999995, 2024-09-17T07:30:31.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:30:31.732 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.732 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.732 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.732 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.732 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.732 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.732 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:30:31.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.732 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:31.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:31.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:30:31.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:30:31.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:30:31.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:30:31.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:30:31.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:30:31.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:30:31.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:30:31.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:30:31.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:30:31.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:30:31.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:30:31.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:30:31.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:30:31.736 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.736 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.736 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.736 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:31.736 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:31.736 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.763273327999997, 2024-09-17T07:30:31.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:30:31.736 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.736 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.736 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.737 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.737 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.737 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.737 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:30:31.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.737 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:31.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:31.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:31.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:30:31.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:30:31.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:30:31.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:30:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:30:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:30:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:30:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:30:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:30:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:30:31.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:30:31.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:30:31.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:30:31.740 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.740 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.740 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.740 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:31.740 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:31.741 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.763254752999998, 2024-09-17T07:30:31.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:30:31.741 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.741 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.741 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.741 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.741 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.741 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.741 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:30:31.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.741 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:31.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:31.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:30:31.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:30:31.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:30:31.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:30:31.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:30:31.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:30:31.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:30:31.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:30:31.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:30:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:30:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:30:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:30:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:30:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:30:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.745 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.745 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:31.745 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:31.745 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.763232325999994, 2024-09-17T07:30:31.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:30:31.745 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.745 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.745 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.745 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.745 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:30:31.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.746 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:31.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:31.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:31.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:31.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:30:31.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:30:31.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:30:31.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:30:31.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:30:31.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:30:31.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:30:31.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:30:31.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:30:31.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:30:31.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:30:31.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:30:31.749 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.749 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.749 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.750 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:31.750 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:31.750 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.763204509000005, 2024-09-17T07:30:31.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:30:31.750 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.750 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.750 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.750 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.751 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.751 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.751 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:30:31.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.751 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:31.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:31.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:31.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:30:31.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:30:31.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:30:31.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:30:31.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:30:31.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:30:31.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:30:31.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:30:31.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:30:31.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:30:31.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:30:31.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:30:31.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:30:31.756 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.756 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.756 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.756 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:31.756 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:31.756 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.763182564999994, 2024-09-17T07:30:31.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:30:31.757 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.757 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.757 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.757 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:30:31.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.757 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:31.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:31.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:31.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:30:31.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:30:31.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:30:31.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:30:31.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:30:31.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:30:31.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:30:31.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:30:31.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:30:31.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:30:31.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:30:31.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:30:31.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:30:31.760 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.760 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.760 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.761 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:31.761 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:31.761 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.763161460999996, 2024-09-17T07:30:31.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:30:31.761 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.761 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.761 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.761 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.761 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.761 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.761 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:30:31.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.761 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:30:31.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:30:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:30:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:30:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:30:31.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:30:31.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:30:31.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:30:31.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:30:31.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:30:31.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:30:31.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:30:31.765 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.765 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.765 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.765 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:31.765 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:31.765 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.763141857999997, 2024-09-17T07:30:31.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:30:31.765 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.765 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.765 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.766 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.766 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.766 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.766 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:30:31.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.766 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:31.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:31.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:30:31.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:30:31.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:30:31.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:30:31.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:30:31.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:30:31.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:30:31.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:30:31.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:30:31.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:30:31.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:30:31.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:30:31.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:30:31.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:30:31.769 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.769 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.769 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.770 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:31.770 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:31.770 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.810200745000003, 2024-09-17T07:30:31.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:30:31.770 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.770 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.770 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.770 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.770 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.770 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.770 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:30:31.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.771 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:31.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:31.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:30:31.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:30:31.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:30:31.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:31.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:30:31.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:30:31.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:30:31.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:30:31.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:30:31.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:30:31.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:30:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:30:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:30:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:30:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.775 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.810187084999995, 2024-09-17T07:30:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:30:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.776 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:30:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:30:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:30:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:30:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:30:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:30:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:30:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:30:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:30:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:30:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:30:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:30:31.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:30:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:30:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.810172303000002, 2024-09-17T07:30:31.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:30:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:30:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:30:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:30:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:30:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:30:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:30:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:30:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:30:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:30:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:30:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:30:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:30:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:30:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.784 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.810161411999999, 2024-09-17T07:30:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:30:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.785 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:30:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:30:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:30:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:30:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:30:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:30:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:30:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:30:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:30:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:30:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:30:31.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:30:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:30:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.810151852000004, 2024-09-17T07:30:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:30:31.789 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:30:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:30:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:30:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:30:31.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:30:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:30:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:30:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:30:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:30:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:30:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:30:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:30:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:30:31.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:30:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.810141291999997, 2024-09-17T07:30:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:30:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:31.794 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:30:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:30:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:30:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:30:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:30:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:30:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:30:31.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:30:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:30:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:30:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:30:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:30:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:30:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:30:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:30:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.798 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.798 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:31.799 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:31.800 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:31.800+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:31.901 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:31.901+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:31.903 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:31.903 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:31.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.903 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:116418 3.13 3.b4964fb3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T07:30:31.903 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:31.903 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 0.73292522100000002, 2024-09-17T07:30:31.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.73294574599999995, 2024-09-17T07:30:31.903 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:31.903 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:31.903 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "waiting for peered", 2024-09-17T07:30:31.903 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:31.904 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:31.904 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:31.904 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116418 2024-09-17T07:30:31.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.904 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:31.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:31.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:31.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:31.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:31.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:31.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:31.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170140+0000", 2024-09-17T07:30:31.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.398e-06 2024-09-17T07:30:31.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:31.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170144+0000", 2024-09-17T07:30:31.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4399999999999998e-06 2024-09-17T07:30:31.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:31.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170157+0000", 2024-09-17T07:30:31.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2133999999999999e-05 2024-09-17T07:30:31.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:31.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170164+0000", 2024-09-17T07:30:31.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7200000000000006e-06 2024-09-17T07:30:31.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:31.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170227+0000", 2024-09-17T07:30:31.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3035999999999998e-05 2024-09-17T07:30:31.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:31.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:31.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:31.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170238+0000", 2024-09-17T07:30:31.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0397000000000001e-05 2024-09-17T07:30:31.907 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.907 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:31.907 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.907 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:31.907 INFO:teuthology.orchestra.run.smithi192.stdout: ], 2024-09-17T07:30:31.907 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 1 2024-09-17T07:30:31.907 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:31.912 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:32.002 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:32.001+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:32.015 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:32.015 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:32.015 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:32.015 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:32.015 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:32.023 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:32.103 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:30:32.104 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:30:32.107 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:32.107+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:32.116 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-17T07:30:32.080092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1414565965,"num_objects":1901,"num_object_clones":444,"num_object_copies":3802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":587,"num_objects_misplaced":124,"num_objects_unfound":0,"num_objects_dirty":1901,"num_whiteouts":994,"num_read":157302,"num_read_kb":1200729,"num_write":21741,"num_write_kb":793997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4511,"num_bytes_recovered":461983418,"num_keys_recovered":567,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18063,"ondisk_log_size":18063,"up":82,"acting":76,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":472256,"kb_used_data":96216,"kb_used_omap":27,"kb_used_meta":375780,"kb_avail":754502464,"statfs":{"total":773094113280,"available":772610523136,"internally_reserved":0,"allocated":98525184,"data_stored":1046036520,"data_compressed":64314797,"data_compressed_allocated":67698688,"data_compressed_original":1022279542,"omap_allocated":28217,"internal_metadata":384799175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8685542,"num_objects":-42,"num_object_clones":-9,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":587,"num_objects_misplaced":124,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":-9,"num_read":436,"num_read_kb":373,"num_write":93,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":607,"num_bytes_recovered":43884481,"num_keys_recovered":47,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.999394"},"pg_stats":[{"pgid":"3.1f","version":"117'778","reported_seq":4171,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.089795+0000","last_change":"2024-09-17T07:30:29.090354+0000","last_active":"2024-09-17T07:30:29.089795+0000","last_peered":"2024-09-17T07:30:29.089795+0000","last_clean":"2024-09-17T07:30:29.089795+0000","last_became_active":"2024-09-17T07:29:15.075896+0000","last_became_peered":"2024-09-17T07:29:15.075896+0000","last_unstale":"2024-09-17T07:30:29.089795+0000","last_undegraded":"2024-09-17T07:30:29.089795+0000","last_fullsized":"2024-09-17T07:30:29.089795+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":778,"log_dups_size":0,"ondisk_log_size":778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:50.853641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000783843,"stat_sum":{"num_bytes":56956133,"num_objects":63,"num_object_clones":16,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":33,"num_read":5527,"num_read_kb":34082,"num_write":939,"num_write_kb":41967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":12072365,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.1e","version":"117'500","reported_seq":2840,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T07:30:26.262194+0000","last_change":"2024-09-17T07:30:26.262503+0000","last_active":"2024-09-17T07:30:26.262194+0000","last_peered":"2024-09-17T07:30:26.262194+0000","last_clean":"2024-09-17T07:30:26.262194+0000","last_became_active":"2024-09-17T07:30:19.087213+0000","last_became_peered":"2024-09-17T07:30:19.087213+0000","last_unstale":"2024-09-17T07:30:26.262194+0000","last_undegraded":"2024-09-17T07:30:26.262194+0000","last_fullsized":"2024-09-17T07:30:26.262194+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:10.408226+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.018614624999999999,"stat_sum":{"num_bytes":23110841,"num_objects":65,"num_object_clones":10,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":44,"num_read":3479,"num_read_kb":30092,"num_write":500,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":15743718,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.1d","version":"115'488","reported_seq":2611,"reported_epoch":119,"state":"peering","last_fresh":"2024-09-17T07:30:28.351015+0000","last_change":"2024-09-17T07:30:29.089994+0000","last_active":"2024-09-17T07:30:29.089994+0000","last_peered":"2024-09-17T07:30:28.351015+0000","last_clean":"2024-09-17T07:30:25.236530+0000","last_became_active":"2024-09-17T07:30:28.101255+0000","last_became_peered":"2024-09-17T07:30:28.101255+0000","last_unstale":"2024-09-17T07:30:28.351015+0000","last_undegraded":"2024-09-17T07:30:28.351015+0000","last_fullsized":"2024-09-17T07:30:28.082047+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010620440000000001,"stat_sum":{"num_bytes":30064459,"num_objects":68,"num_object_clones":13,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":41,"num_read":1884,"num_read_kb":28710,"num_write":539,"num_write_kb":27233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":27967854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"24","length":"2"}]},{"pgid":"3.1c","version":"119'671","reported_seq":5225,"reported_epoch":120,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T07:30:29.089485+0000","last_change":"2024-09-17T07:30:28.119660+0000","last_active":"2024-09-17T07:30:29.089485+0000","last_peered":"2024-09-17T07:30:29.089485+0000","last_clean":"2024-09-17T07:30:25.281920+0000","last_became_active":"2024-09-17T07:30:28.118630+0000","last_became_peered":"2024-09-17T07:30:28.118630+0000","last_unstale":"2024-09-17T07:30:29.089485+0000","last_undegraded":"2024-09-17T07:30:28.083739+0000","last_fullsized":"2024-09-17T07:30:28.082118+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019020549999999999,"stat_sum":{"num_bytes":39849983,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":99,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":23,"num_read":6775,"num_read_kb":41792,"num_write":919,"num_write_kb":38700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":99,"num_bytes_recovered":13330825,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5],"avail_no_missing":["5","1"],"object_location_counts":[{"shards":"1,5","objects":48},{"shards":"5","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"24","length":"2"}]},{"pgid":"3.1b","version":"115'528","reported_seq":4249,"reported_epoch":120,"state":"active+recovery_wait+remapped","last_fresh":"2024-09-17T07:30:29.086057+0000","last_change":"2024-09-17T07:30:28.124094+0000","last_active":"2024-09-17T07:30:29.086057+0000","last_peered":"2024-09-17T07:30:29.086057+0000","last_clean":"2024-09-17T07:30:24.304355+0000","last_became_active":"2024-09-17T07:30:28.123034+0000","last_became_peered":"2024-09-17T07:30:28.123034+0000","last_unstale":"2024-09-17T07:30:29.086057+0000","last_undegraded":"2024-09-17T07:30:29.086057+0000","last_fullsized":"2024-09-17T07:30:29.086057+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001321247,"stat_sum":{"num_bytes":38077370,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":124,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":22,"num_read":5346,"num_read_kb":52067,"num_write":550,"num_write_kb":22867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":33455393,"num_keys_recovered":42,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[{"shards":"1,7","objects":47}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[{"start":"24","length":"2"}]},{"pgid":"3.1a","version":"115'833","reported_seq":14188,"reported_epoch":120,"state":"active+undersized+remapped","last_fresh":"2024-09-17T07:30:29.206585+0000","last_change":"2024-09-17T07:30:29.206806+0000","last_active":"2024-09-17T07:30:29.206585+0000","last_peered":"2024-09-17T07:30:29.206585+0000","last_clean":"2024-09-17T07:30:25.293421+0000","last_became_active":"2024-09-17T07:30:28.120729+0000","last_became_peered":"2024-09-17T07:30:28.120729+0000","last_unstale":"2024-09-17T07:30:29.206585+0000","last_undegraded":"2024-09-17T07:30:29.206585+0000","last_fullsized":"2024-09-17T07:30:28.084611+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010245440000000001,"stat_sum":{"num_bytes":71530956,"num_objects":61,"num_object_clones":14,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":30,"num_read":16205,"num_read_kb":60522,"num_write":1131,"num_write_kb":20582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":19561990,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[1],"avail_no_missing":["1","2","4"],"object_location_counts":[{"shards":"1","objects":61}],"blocked_by":[],"up_primary":4,"acting_primary":1,"purged_snaps":[{"start":"24","length":"2"}]},{"pgid":"3.19","version":"117'403","reported_seq":5352,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.084013+0000","last_change":"2024-09-17T07:30:29.084511+0000","last_active":"2024-09-17T07:30:29.084013+0000","last_peered":"2024-09-17T07:30:29.084013+0000","last_clean":"2024-09-17T07:30:29.084013+0000","last_became_active":"2024-09-17T07:30:18.083918+0000","last_became_peered":"2024-09-17T07:30:18.083918+0000","last_unstale":"2024-09-17T07:30:29.084013+0000","last_undegraded":"2024-09-17T07:30:29.084013+0000","last_fullsized":"2024-09-17T07:30:29.084013+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:10:32.876975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088949099999999996,"stat_sum":{"num_bytes":36709709,"num_objects":62,"num_object_clones":11,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":37,"num_read":4339,"num_read_kb":37349,"num_write":386,"num_write_kb":43089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":25536146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.18","version":"117'632","reported_seq":3599,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T07:30:26.245998+0000","last_change":"2024-09-17T07:30:26.246581+0000","last_active":"2024-09-17T07:30:26.245998+0000","last_peered":"2024-09-17T07:30:26.245998+0000","last_clean":"2024-09-17T07:30:26.245998+0000","last_became_active":"2024-09-17T07:30:20.091034+0000","last_became_peered":"2024-09-17T07:30:20.091034+0000","last_unstale":"2024-09-17T07:30:26.245998+0000","last_undegraded":"2024-09-17T07:30:26.245998+0000","last_fullsized":"2024-09-17T07:30:26.245998+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:13.422253+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0032417399999999999,"stat_sum":{"num_bytes":25985926,"num_objects":62,"num_object_clones":15,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":31,"num_read":5112,"num_read_kb":24587,"num_write":712,"num_write_kb":16579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":22420580,"num_keys_recovered":129,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.17","version":"119'353","reported_seq":3645,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T07:30:28.084531+0000","last_change":"2024-09-17T07:30:26.257681+0000","last_active":"2024-09-17T07:30:28.084531+0000","last_peered":"2024-09-17T07:30:28.084531+0000","last_clean":"2024-09-17T07:30:28.084531+0000","last_became_active":"2024-09-17T07:28:43.080428+0000","last_became_peered":"2024-09-17T07:28:43.080428+0000","last_unstale":"2024-09-17T07:30:28.084531+0000","last_undegraded":"2024-09-17T07:30:28.084531+0000","last_fullsized":"2024-09-17T07:30:28.084531+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:05.496082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025135959999999999,"stat_sum":{"num_bytes":43270254,"num_objects":63,"num_object_clones":15,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":37,"num_read":4417,"num_read_kb":22249,"num_write":322,"num_write_kb":13519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"23","length":"3"}]},{"pgid":"3.16","version":"119'381","reported_seq":3370,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.089885+0000","last_change":"2024-09-17T07:30:29.090486+0000","last_active":"2024-09-17T07:30:29.089885+0000","last_peered":"2024-09-17T07:30:29.089885+0000","last_clean":"2024-09-17T07:30:29.089885+0000","last_became_active":"2024-09-17T07:28:43.082313+0000","last_became_peered":"2024-09-17T07:28:43.082313+0000","last_unstale":"2024-09-17T07:30:29.089885+0000","last_undegraded":"2024-09-17T07:30:29.089885+0000","last_fullsized":"2024-09-17T07:30:29.089885+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:05.437018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091487100000000004,"stat_sum":{"num_bytes":32054138,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":27,"num_read":5061,"num_read_kb":57431,"num_write":362,"num_write_kb":31831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.15","version":"119'343","reported_seq":3935,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.084156+0000","last_change":"2024-09-17T07:30:29.084784+0000","last_active":"2024-09-17T07:30:29.084156+0000","last_peered":"2024-09-17T07:30:29.084156+0000","last_clean":"2024-09-17T07:30:29.084156+0000","last_became_active":"2024-09-17T07:30:18.088846+0000","last_became_peered":"2024-09-17T07:30:18.088846+0000","last_unstale":"2024-09-17T07:30:29.084156+0000","last_undegraded":"2024-09-17T07:30:29.084156+0000","last_fullsized":"2024-09-17T07:30:29.084156+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:58.557102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097105200000000003,"stat_sum":{"num_bytes":49233699,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":29,"num_read":3578,"num_read_kb":32348,"num_write":358,"num_write_kb":22781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":35856553,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.14","version":"115'409","reported_seq":1760,"reported_epoch":120,"state":"peering","last_fresh":"2024-09-17T07:30:28.557003+0000","last_change":"2024-09-17T07:30:29.317205+0000","last_active":"2024-09-17T07:30:29.317205+0000","last_peered":"2024-09-17T07:30:28.557003+0000","last_clean":"2024-09-17T07:30:24.304594+0000","last_became_active":"2024-09-17T07:30:28.115977+0000","last_became_peered":"2024-09-17T07:30:28.115977+0000","last_unstale":"2024-09-17T07:30:28.557003+0000","last_undegraded":"2024-09-17T07:30:28.557003+0000","last_fullsized":"2024-09-17T07:30:28.557003+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076541599999999999,"stat_sum":{"num_bytes":20480595,"num_objects":59,"num_object_clones":17,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":31,"num_read":1372,"num_read_kb":17945,"num_write":459,"num_write_kb":24984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":16518806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":220,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.088294+0000","last_change":"2024-09-17T07:29:12.072080+0000","last_active":"2024-09-17T07:30:29.088294+0000","last_peered":"2024-09-17T07:30:29.088294+0000","last_clean":"2024-09-17T07:30:29.088294+0000","last_became_active":"2024-09-17T07:29:12.071738+0000","last_became_peered":"2024-09-17T07:29:12.071738+0000","last_unstale":"2024-09-17T07:30:29.088294+0000","last_undegraded":"2024-09-17T07:30:29.088294+0000","last_fullsized":"2024-09-17T07:30:29.088294+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:37.818818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"119'949","reported_seq":2378,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T07:30:28.180755+0000","last_change":"2024-09-17T07:30:26.259346+0000","last_active":"2024-09-17T07:30:28.180755+0000","last_peered":"2024-09-17T07:30:28.180755+0000","last_clean":"2024-09-17T07:30:28.180755+0000","last_became_active":"2024-09-17T07:28:43.079229+0000","last_became_peered":"2024-09-17T07:28:43.079229+0000","last_unstale":"2024-09-17T07:30:28.180755+0000","last_undegraded":"2024-09-17T07:30:28.180755+0000","last_fullsized":"2024-09-17T07:30:28.180755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":949,"log_dups_size":0,"ondisk_log_size":949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:03.740687+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0048500510000000002,"stat_sum":{"num_bytes":19654028,"num_objects":55,"num_object_clones":15,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":32,"num_read":4584,"num_read_kb":48715,"num_write":1458,"num_write_kb":29292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"23","length":"3"}]},{"pgid":"2.6","version":"0'0","reported_seq":216,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.084342+0000","last_change":"2024-09-17T07:28:33.361412+0000","last_active":"2024-09-17T07:30:29.084342+0000","last_peered":"2024-09-17T07:30:29.084342+0000","last_clean":"2024-09-17T07:30:29.084342+0000","last_became_active":"2024-09-17T07:28:31.362639+0000","last_became_peered":"2024-09-17T07:28:31.362639+0000","last_unstale":"2024-09-17T07:30:29.084342+0000","last_undegraded":"2024-09-17T07:30:29.084342+0000","last_fullsized":"2024-09-17T07:30:29.084342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:42.297390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049448899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"117'499","reported_seq":3227,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T07:30:26.258505+0000","last_change":"2024-09-17T07:30:26.259020+0000","last_active":"2024-09-17T07:30:26.258505+0000","last_peered":"2024-09-17T07:30:26.258505+0000","last_clean":"2024-09-17T07:30:26.258505+0000","last_became_active":"2024-09-17T07:30:19.087787+0000","last_became_peered":"2024-09-17T07:30:19.087787+0000","last_unstale":"2024-09-17T07:30:26.258505+0000","last_undegraded":"2024-09-17T07:30:26.258505+0000","last_fullsized":"2024-09-17T07:30:26.258505+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:20.737249+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015330372,"stat_sum":{"num_bytes":55821458,"num_objects":64,"num_object_clones":15,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":33,"num_read":3712,"num_read_kb":36756,"num_write":569,"num_write_kb":22611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":15169480,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":215,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.083985+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:30:29.083985+0000","last_peered":"2024-09-17T07:30:29.083985+0000","last_clean":"2024-09-17T07:30:29.083985+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:30:29.083985+0000","last_undegraded":"2024-09-17T07:30:29.083985+0000","last_fullsized":"2024-09-17T07:30:29.083985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"113'982","reported_seq":3156,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.084881+0000","last_change":"2024-09-17T07:30:29.085456+0000","last_active":"2024-09-17T07:30:29.084881+0000","last_peered":"2024-09-17T07:30:29.084881+0000","last_clean":"2024-09-17T07:30:29.084881+0000","last_became_active":"2024-09-17T07:28:43.239387+0000","last_became_peered":"2024-09-17T07:28:43.239387+0000","last_unstale":"2024-09-17T07:30:29.084881+0000","last_undegraded":"2024-09-17T07:30:29.084881+0000","last_fullsized":"2024-09-17T07:30:29.084881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:16:13.684838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016536669999999999,"stat_sum":{"num_bytes":16664349,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":29,"num_read":6577,"num_read_kb":16679,"num_write":1390,"num_write_kb":10798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.089414+0000","last_change":"2024-09-17T07:30:15.200774+0000","last_active":"2024-09-17T07:30:29.089414+0000","last_peered":"2024-09-17T07:30:29.089414+0000","last_clean":"2024-09-17T07:30:29.089414+0000","last_became_active":"2024-09-17T07:30:15.200470+0000","last_became_peered":"2024-09-17T07:30:15.200470+0000","last_unstale":"2024-09-17T07:30:29.089414+0000","last_undegraded":"2024-09-17T07:30:29.089414+0000","last_fullsized":"2024-09-17T07:30:29.089414+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:58.984439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"115'505","reported_seq":2933,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.091063+0000","last_change":"2024-09-17T07:30:29.091631+0000","last_active":"2024-09-17T07:30:29.091063+0000","last_peered":"2024-09-17T07:30:29.091063+0000","last_clean":"2024-09-17T07:30:29.091063+0000","last_became_active":"2024-09-17T07:30:19.085184+0000","last_became_peered":"2024-09-17T07:30:19.085184+0000","last_unstale":"2024-09-17T07:30:29.091063+0000","last_undegraded":"2024-09-17T07:30:29.091063+0000","last_fullsized":"2024-09-17T07:30:29.091063+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:43:44.818013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090038,"stat_sum":{"num_bytes":43719179,"num_objects":63,"num_object_clones":14,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":33,"num_read":2622,"num_read_kb":46253,"num_write":596,"num_write_kb":27527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":35691417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.3","version":"119'503","reported_seq":2310,"reported_epoch":119,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T07:30:28.199347+0000","last_change":"2024-09-17T07:30:28.124280+0000","last_active":"2024-09-17T07:30:28.199347+0000","last_peered":"2024-09-17T07:30:28.199347+0000","last_clean":"2024-09-17T07:30:25.270148+0000","last_became_active":"2024-09-17T07:30:28.123152+0000","last_became_peered":"2024-09-17T07:30:28.123152+0000","last_unstale":"2024-09-17T07:30:28.199347+0000","last_undegraded":"2024-09-17T07:30:28.085996+0000","last_fullsized":"2024-09-17T07:30:28.084438+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001863137,"stat_sum":{"num_bytes":46574206,"num_objects":61,"num_object_clones":10,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":119,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":33,"num_read":2036,"num_read_kb":35180,"num_write":487,"num_write_kb":16588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":13363879,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[0],"avail_no_missing":["0","6"],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,6","objects":60}],"blocked_by":[],"up_primary":4,"acting_primary":0,"purged_snaps":[{"start":"24","length":"2"}]},{"pgid":"2.2","version":"22'2","reported_seq":218,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.089640+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:30:29.089640+0000","last_peered":"2024-09-17T07:30:29.089640+0000","last_clean":"2024-09-17T07:30:29.089640+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:30:29.089640+0000","last_undegraded":"2024-09-17T07:30:29.089640+0000","last_fullsized":"2024-09-17T07:30:29.089640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"119'761","reported_seq":10981,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.084951+0000","last_change":"2024-09-17T07:30:29.085725+0000","last_active":"2024-09-17T07:30:29.084951+0000","last_peered":"2024-09-17T07:30:29.084951+0000","last_clean":"2024-09-17T07:30:29.084951+0000","last_became_active":"2024-09-17T07:28:43.239409+0000","last_became_peered":"2024-09-17T07:28:43.239409+0000","last_unstale":"2024-09-17T07:30:29.084951+0000","last_undegraded":"2024-09-17T07:30:29.084951+0000","last_fullsized":"2024-09-17T07:30:29.084951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:54:24.421043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018015660000000001,"stat_sum":{"num_bytes":115581482,"num_objects":71,"num_object_clones":14,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":41,"num_read":12555,"num_read_kb":91660,"num_write":1200,"num_write_kb":58357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"2.1","version":"0'0","reported_seq":227,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T07:30:28.082263+0000","last_change":"2024-09-17T07:30:15.201999+0000","last_active":"2024-09-17T07:30:28.082263+0000","last_peered":"2024-09-17T07:30:28.082263+0000","last_clean":"2024-09-17T07:30:28.082263+0000","last_became_active":"2024-09-17T07:30:15.201642+0000","last_became_peered":"2024-09-17T07:30:15.201642+0000","last_unstale":"2024-09-17T07:30:28.082263+0000","last_undegraded":"2024-09-17T07:30:28.082263+0000","last_fullsized":"2024-09-17T07:30:28.082263+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:47.871565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"117'367","reported_seq":4244,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T07:30:28.084347+0000","last_change":"2024-09-17T07:30:26.257176+0000","last_active":"2024-09-17T07:30:28.084347+0000","last_peered":"2024-09-17T07:30:28.084347+0000","last_clean":"2024-09-17T07:30:28.084347+0000","last_became_active":"2024-09-17T07:30:18.088918+0000","last_became_peered":"2024-09-17T07:30:18.088918+0000","last_unstale":"2024-09-17T07:30:28.084347+0000","last_undegraded":"2024-09-17T07:30:28.084347+0000","last_fullsized":"2024-09-17T07:30:28.084347+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:43.801671+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0027338919999999999,"stat_sum":{"num_bytes":49998777,"num_objects":52,"num_object_clones":10,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":23,"num_read":3608,"num_read_kb":51530,"num_write":389,"num_write_kb":28006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":36305471,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"23","length":"3"}]},{"pgid":"2.0","version":"0'0","reported_seq":215,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.083100+0000","last_change":"2024-09-17T07:28:33.951319+0000","last_active":"2024-09-17T07:30:29.083100+0000","last_peered":"2024-09-17T07:30:29.083100+0000","last_clean":"2024-09-17T07:30:29.083100+0000","last_became_active":"2024-09-17T07:28:31.364065+0000","last_became_peered":"2024-09-17T07:28:31.364065+0000","last_unstale":"2024-09-17T07:30:29.083100+0000","last_undegraded":"2024-09-17T07:30:29.083100+0000","last_fullsized":"2024-09-17T07:30:29.083100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:40.816064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"117'345","reported_seq":2019,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T07:30:26.251214+0000","last_change":"2024-09-17T07:30:26.251816+0000","last_active":"2024-09-17T07:30:26.251214+0000","last_peered":"2024-09-17T07:30:26.251214+0000","last_clean":"2024-09-17T07:30:26.251214+0000","last_became_active":"2024-09-17T07:30:18.084247+0000","last_became_peered":"2024-09-17T07:30:18.084247+0000","last_unstale":"2024-09-17T07:30:26.251214+0000","last_undegraded":"2024-09-17T07:30:26.251214+0000","last_fullsized":"2024-09-17T07:30:26.251214+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:42.103295+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0084000410000000005,"stat_sum":{"num_bytes":39770770,"num_objects":53,"num_object_clones":11,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":25,"num_read":2641,"num_read_kb":8993,"num_write":331,"num_write_kb":6311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":6805602,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"2.3","version":"20'1","reported_seq":217,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.089415+0000","last_change":"2024-09-17T07:28:33.361820+0000","last_active":"2024-09-17T07:30:29.089415+0000","last_peered":"2024-09-17T07:30:29.089415+0000","last_clean":"2024-09-17T07:30:29.089415+0000","last_became_active":"2024-09-17T07:28:31.359703+0000","last_became_peered":"2024-09-17T07:28:31.359703+0000","last_unstale":"2024-09-17T07:30:29.089415+0000","last_undegraded":"2024-09-17T07:30:29.089415+0000","last_fullsized":"2024-09-17T07:30:29.089415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:59.468099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000487639,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":307,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.083159+0000","last_change":"2024-09-17T07:28:26.547435+0000","last_active":"2024-09-17T07:30:29.083159+0000","last_peered":"2024-09-17T07:30:29.083159+0000","last_clean":"2024-09-17T07:30:29.083159+0000","last_became_active":"2024-09-17T07:28:26.546300+0000","last_became_peered":"2024-09-17T07:28:26.546300+0000","last_unstale":"2024-09-17T07:30:29.083159+0000","last_undegraded":"2024-09-17T07:30:29.083159+0000","last_fullsized":"2024-09-17T07:30:29.083159+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:02:23.225041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"117'464","reported_seq":2069,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T07:30:26.256463+0000","last_change":"2024-09-17T07:30:26.256973+0000","last_active":"2024-09-17T07:30:26.256463+0000","last_peered":"2024-09-17T07:30:26.256463+0000","last_clean":"2024-09-17T07:30:26.256463+0000","last_became_active":"2024-09-17T07:30:19.086777+0000","last_became_peered":"2024-09-17T07:30:19.086777+0000","last_unstale":"2024-09-17T07:30:26.256463+0000","last_undegraded":"2024-09-17T07:30:26.256463+0000","last_fullsized":"2024-09-17T07:30:26.256463+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:10.033199+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.01335271,"stat_sum":{"num_bytes":43700092,"num_objects":63,"num_object_clones":16,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":32,"num_read":1606,"num_read_kb":24362,"num_write":504,"num_write_kb":33950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":24920122,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.9","version":"116'480","reported_seq":3204,"reported_epoch":120,"state":"peering","last_fresh":"2024-09-17T07:30:28.999352+0000","last_change":"2024-09-17T07:30:29.317189+0000","last_active":"2024-09-17T07:30:29.317189+0000","last_peered":"2024-09-17T07:30:28.999352+0000","last_clean":"2024-09-17T07:30:25.265201+0000","last_became_active":"2024-09-17T07:30:28.117743+0000","last_became_peered":"2024-09-17T07:30:28.117743+0000","last_unstale":"2024-09-17T07:30:28.999352+0000","last_undegraded":"2024-09-17T07:30:28.999352+0000","last_fullsized":"2024-09-17T07:30:28.082220+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023857230000000002,"stat_sum":{"num_bytes":29368707,"num_objects":61,"num_object_clones":18,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":29,"num_read":4239,"num_read_kb":20839,"num_write":500,"num_write_kb":12190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":11434290,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"24","length":"2"}]},{"pgid":"3.a","version":"119'604","reported_seq":8041,"reported_epoch":120,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T07:30:29.087798+0000","last_change":"2024-09-17T07:30:28.121755+0000","last_active":"2024-09-17T07:30:29.087798+0000","last_peered":"2024-09-17T07:30:29.087798+0000","last_clean":"2024-09-17T07:30:25.240923+0000","last_became_active":"2024-09-17T07:30:28.120530+0000","last_became_peered":"2024-09-17T07:30:28.120530+0000","last_unstale":"2024-09-17T07:30:29.087798+0000","last_undegraded":"2024-09-17T07:30:28.085542+0000","last_fullsized":"2024-09-17T07:30:28.083524+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084291900000000003,"stat_sum":{"num_bytes":107715189,"num_objects":85,"num_object_clones":21,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":46,"num_read":8761,"num_read_kb":52399,"num_write":604,"num_write_kb":21077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":13150748,"num_keys_recovered":47,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6],"avail_no_missing":["6","1"],"object_location_counts":[{"shards":"1,6","objects":84},{"shards":"6","objects":1}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"24","length":"2"}]},{"pgid":"3.b","version":"113'500","reported_seq":2543,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-17T07:30:26.247442+0000","last_change":"2024-09-17T07:30:26.248492+0000","last_active":"2024-09-17T07:30:26.247442+0000","last_peered":"2024-09-17T07:30:26.247442+0000","last_clean":"2024-09-17T07:30:26.247442+0000","last_became_active":"2024-09-17T07:30:19.087515+0000","last_became_peered":"2024-09-17T07:30:19.087515+0000","last_unstale":"2024-09-17T07:30:26.247442+0000","last_undegraded":"2024-09-17T07:30:26.247442+0000","last_fullsized":"2024-09-17T07:30:26.247442+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:55:50.763623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050508150000000002,"stat_sum":{"num_bytes":20690401,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":25,"num_read":2352,"num_read_kb":29864,"num_write":631,"num_write_kb":15628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":11178824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"9"}]},{"pgid":"3.c","version":"119'778","reported_seq":4958,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.090446+0000","last_change":"2024-09-17T07:30:29.090976+0000","last_active":"2024-09-17T07:30:29.090446+0000","last_peered":"2024-09-17T07:30:29.090446+0000","last_clean":"2024-09-17T07:30:29.090446+0000","last_became_active":"2024-09-17T07:30:19.085668+0000","last_became_peered":"2024-09-17T07:30:19.085668+0000","last_unstale":"2024-09-17T07:30:29.090446+0000","last_undegraded":"2024-09-17T07:30:29.090446+0000","last_fullsized":"2024-09-17T07:30:29.090446+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":778,"log_dups_size":0,"ondisk_log_size":778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:04.534854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091114400000000002,"stat_sum":{"num_bytes":19289945,"num_objects":57,"num_object_clones":11,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":25,"num_read":5809,"num_read_kb":27507,"num_write":1043,"num_write_kb":14812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":15619889,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.d","version":"113'659","reported_seq":4184,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.084621+0000","last_change":"2024-09-17T07:30:29.085041+0000","last_active":"2024-09-17T07:30:29.084621+0000","last_peered":"2024-09-17T07:30:29.084621+0000","last_clean":"2024-09-17T07:30:29.084621+0000","last_became_active":"2024-09-17T07:28:43.080816+0000","last_became_peered":"2024-09-17T07:28:43.080816+0000","last_unstale":"2024-09-17T07:30:29.084621+0000","last_undegraded":"2024-09-17T07:30:29.084621+0000","last_fullsized":"2024-09-17T07:30:29.084621+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:55.833773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010723670000000001,"stat_sum":{"num_bytes":95045662,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":5827,"num_read_kb":48806,"num_write":802,"num_write_kb":31218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.e","version":"115'415","reported_seq":7323,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.084094+0000","last_change":"2024-09-17T07:30:29.084549+0000","last_active":"2024-09-17T07:30:29.084094+0000","last_peered":"2024-09-17T07:30:29.084094+0000","last_clean":"2024-09-17T07:30:29.084094+0000","last_became_active":"2024-09-17T07:29:12.076559+0000","last_became_peered":"2024-09-17T07:29:12.076559+0000","last_unstale":"2024-09-17T07:30:29.084094+0000","last_undegraded":"2024-09-17T07:30:29.084094+0000","last_fullsized":"2024-09-17T07:30:29.084094+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:50.941299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076342599999999999,"stat_sum":{"num_bytes":64449564,"num_objects":58,"num_object_clones":16,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":31,"num_read":7070,"num_read_kb":55682,"num_write":381,"num_write_kb":41376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":12168265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.f","version":"119'719","reported_seq":5026,"reported_epoch":119,"state":"peering","last_fresh":"2024-09-17T07:30:28.813376+0000","last_change":"2024-09-17T07:30:29.083977+0000","last_active":"2024-09-17T07:30:29.083977+0000","last_peered":"2024-09-17T07:30:28.813376+0000","last_clean":"2024-09-17T07:30:25.286170+0000","last_became_active":"2024-09-17T07:30:28.116955+0000","last_became_peered":"2024-09-17T07:30:28.116955+0000","last_unstale":"2024-09-17T07:30:28.813376+0000","last_undegraded":"2024-09-17T07:30:28.813376+0000","last_fullsized":"2024-09-17T07:30:28.088485+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":719,"log_dups_size":0,"ondisk_log_size":719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013688940000000001,"stat_sum":{"num_bytes":46274018,"num_objects":75,"num_object_clones":16,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":38,"num_read":6075,"num_read_kb":22879,"num_write":851,"num_write_kb":18525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":16463942,"num_keys_recovered":45,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"2"}]},{"pgid":"3.10","version":"119'346","reported_seq":2344,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-17T07:30:29.088089+0000","last_change":"2024-09-17T07:30:29.088520+0000","last_active":"2024-09-17T07:30:29.088089+0000","last_peered":"2024-09-17T07:30:29.088089+0000","last_clean":"2024-09-17T07:30:29.088089+0000","last_became_active":"2024-09-17T07:28:43.077360+0000","last_became_peered":"2024-09-17T07:28:43.077360+0000","last_unstale":"2024-09-17T07:30:29.088089+0000","last_undegraded":"2024-09-17T07:30:29.088089+0000","last_fullsized":"2024-09-17T07:30:29.088089+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:28.256149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067053800000000004,"stat_sum":{"num_bytes":25781021,"num_objects":49,"num_object_clones":10,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":25,"num_read":2654,"num_read_kb":26041,"num_write":420,"num_write_kb":12229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"22","length":"4"}]},{"pgid":"3.11","version":"116'735","reported_seq":3170,"reported_epoch":120,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T07:30:29.083143+0000","last_change":"2024-09-17T07:30:28.120848+0000","last_active":"2024-09-17T07:30:29.083143+0000","last_peered":"2024-09-17T07:30:29.083143+0000","last_clean":"2024-09-17T07:30:25.302319+0000","last_became_active":"2024-09-17T07:30:28.119643+0000","last_became_peered":"2024-09-17T07:30:28.119643+0000","last_unstale":"2024-09-17T07:30:29.083143+0000","last_undegraded":"2024-09-17T07:30:28.090617+0000","last_fullsized":"2024-09-17T07:30:28.088181+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012493529999999999,"stat_sum":{"num_bytes":42814083,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":25,"num_read":5106,"num_read_kb":45039,"num_write":900,"num_write_kb":26410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":16346328,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","6"],"object_location_counts":[{"shards":"6,7","objects":46}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"2"}]},{"pgid":"3.12","version":"115'338","reported_seq":1665,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T07:30:28.084308+0000","last_change":"2024-09-17T07:30:26.256520+0000","last_active":"2024-09-17T07:30:28.084308+0000","last_peered":"2024-09-17T07:30:28.084308+0000","last_clean":"2024-09-17T07:30:28.084308+0000","last_became_active":"2024-09-17T07:28:43.081466+0000","last_became_peered":"2024-09-17T07:28:43.081466+0000","last_unstale":"2024-09-17T07:30:28.084308+0000","last_undegraded":"2024-09-17T07:30:28.084308+0000","last_fullsized":"2024-09-17T07:30:28.084308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:13:23.757613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020556839999999999,"stat_sum":{"num_bytes":45121457,"num_objects":70,"num_object_clones":19,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":35,"num_read":1861,"num_read_kb":44241,"num_write":364,"num_write_kb":29136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"23","length":"3"}]},{"pgid":"3.13","version":"119'720","reported_seq":3248,"reported_epoch":120,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-17T07:30:29.083390+0000","last_change":"2024-09-17T07:30:28.124088+0000","last_active":"2024-09-17T07:30:29.083390+0000","last_peered":"2024-09-17T07:30:29.083390+0000","last_clean":"2024-09-17T07:30:25.243452+0000","last_became_active":"2024-09-17T07:30:28.123233+0000","last_became_peered":"2024-09-17T07:30:28.123233+0000","last_unstale":"2024-09-17T07:30:29.083390+0000","last_undegraded":"2024-09-17T07:30:28.092769+0000","last_fullsized":"2024-09-17T07:30:28.090321+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062957499999999995,"stat_sum":{"num_bytes":18617127,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":101,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":24,"num_read":4446,"num_read_kb":28075,"num_write":1041,"num_write_kb":16606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":101,"num_bytes_recovered":10900931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7],"avail_no_missing":["7","2"],"object_location_counts":[{"shards":"2,7","objects":43},{"shards":"7","objects":-2}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"24","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1413975578,"num_objects":1897,"num_object_clones":444,"num_object_copies":3794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":587,"num_objects_misplaced":124,"num_objects_unfound":0,"num_objects_dirty":1897,"num_whiteouts":994,"num_read":157236,"num_read_kb":1200674,"num_write":21638,"num_write_kb":792367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4511,"num_bytes_recovered":461983418,"num_keys_recovered":567,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":89677824,"data_stored":1061402932,"data_compressed":65797075,"data_compressed_allocated":69529600,"data_compressed_original":1042472822,"omap_allocated":7902,"internal_metadata":0},"log_size":17988,"ondisk_log_size":17988,"up":64,"acting":58,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65196,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70824,"kb_used_data":17420,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94301016,"statfs":{"total":96636764160,"available":96564240384,"internally_reserved":0,"allocated":17838080,"data_stored":217072270,"data_compressed":13214621,"data_compressed_allocated":13135872,"data_compressed_original":213479068,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49160,"kb_used_data":9644,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322680,"statfs":{"total":96636764160,"available":96586424320,"internally_reserved":0,"allocated":9875456,"data_stored":85859033,"data_compressed":5273608,"data_compressed_allocated":6201344,"data_compressed_original":82657280,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62476,"kb_used_data":13228,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94309364,"statfs":{"total":96636764160,"available":96572788736,"internally_reserved":0,"allocated":13545472,"data_stored":161731226,"data_compressed":9662733,"data_compressed_allocated":9678848,"data_compressed_original":158878273,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50272,"kb_used_data":5192,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321568,"statfs":{"total":96636764160,"available":96585285632,"internally_reserved":0,"allocated":5316608,"data_stored":36440811,"data_compressed":2342138,"data_compressed_allocated":2998272,"data_compressed_original":35005235,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056647,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50544,"kb_used_data":13644,"kb_used_omap":16,"kb_used_meta":36847,"kb_avail":94321296,"statfs":{"total":96636764160,"available":96585007104,"internally_reserved":0,"allocated":13971456,"data_stored":133125527,"data_compressed":8617834,"data_compressed_allocated":9932800,"data_compressed_original":130210212,"omap_allocated":17058,"internal_metadata":37731678},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58664,"kb_used_data":9428,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94313176,"statfs":{"total":96636764160,"available":96576692224,"internally_reserved":0,"allocated":9654272,"data_stored":94406866,"data_compressed":5966565,"data_compressed_allocated":6447104,"data_compressed_original":92084878,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87624,"kb_used_data":16172,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94284216,"statfs":{"total":96636764160,"available":96547037184,"internally_reserved":0,"allocated":16560128,"data_stored":190109292,"data_compressed":11895821,"data_compressed_allocated":11767808,"data_compressed_original":186424936,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542171,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42692,"kb_used_data":11488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329148,"statfs":{"total":96636764160,"available":96593047552,"internally_reserved":0,"allocated":11763712,"data_stored":127291495,"data_compressed":7341477,"data_compressed_allocated":7536640,"data_compressed_original":123539660,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10235904,"data_stored":125774381,"data_compressed":7259852,"data_compressed_allocated":7438336,"data_compressed_original":122704076,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15138816,"data_stored":189438526,"data_compressed":11895821,"data_compressed_allocated":11767808,"data_compressed_original":186424936,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8237056,"data_stored":93736119,"data_compressed":5966565,"data_compressed_allocated":6447104,"data_compressed_original":92084878,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12578816,"data_stored":132467425,"data_compressed":8617834,"data_compressed_allocated":9932800,"data_compressed_original":130210212,"omap_allocated":7902,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6549504,"data_stored":57926580,"data_compressed":3938639,"data_compressed_allocated":4972544,"data_compressed_original":56607539,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12124160,"data_stored":161060460,"data_compressed":9662733,"data_compressed_allocated":9678848,"data_compressed_original":158878273,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8458240,"data_stored":85188286,"data_compressed":5273608,"data_compressed_allocated":6201344,"data_compressed_original":82657280,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16355328,"data_stored":215811155,"data_compressed":13182023,"data_compressed_allocated":13090816,"data_compressed_original":212905628,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:30:32.118 INFO:tasks.thrashosds.thrasher:Setting 2.3 to [5, 0, 4, 6] 2024-09-17T07:30:32.118 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.3', '5', '0', '4', '6'] 2024-09-17T07:30:32.119 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.3 5 0 4 6 2024-09-17T07:30:32.177 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.177 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:30:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 125.840552649, 2024-09-17T07:30:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:30:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:30:32.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:30:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:30:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:30:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:30:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:30:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:30:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:30:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:30:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:30:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:30:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:30:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:30:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:30:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:30:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:30:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:30:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:30:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:30:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:30:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:30:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:30:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:30:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:30:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:30:32.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:30:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:30:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:30:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:30:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.251500499000002, 2024-09-17T07:30:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:30:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.186 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:30:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:30:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:30:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:30:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:30:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:30:32.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:30:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:30:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:30:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:30:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:30:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:30:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:30:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:30:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:30:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.191 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.246217383000001, 2024-09-17T07:30:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:30:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.193 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:30:32.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.193 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:32.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:32.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:32.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:30:32.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:30:32.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:30:32.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:30:32.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:30:32.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:30:32.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:30:32.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:30:32.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:30:32.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:30:32.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:30:32.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:30:32.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:30:32.197 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.197 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.197 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.198 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.198 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:32.198 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.246186402999996, 2024-09-17T07:30:32.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:30:32.198 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.198 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.198 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.198 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.198 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.198 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.199 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:30:32.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.199 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:32.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:32.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:30:32.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:30:32.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:30:32.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:30:32.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:30:32.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:30:32.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:30:32.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:30:32.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:30:32.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:30:32.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:30:32.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:30:32.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:30:32.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:30:32.202 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.203 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.203 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.203 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.203 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:32.203 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.246156353999993, 2024-09-17T07:30:32.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:30:32.203 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.203 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.203 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.203 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.203 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.203 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.204 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:30:32.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.204 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:32.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:32.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:30:32.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:30:32.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:30:32.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:30:32.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:30:32.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:30:32.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:30:32.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:30:32.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:30:32.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:30:32.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:30:32.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:30:32.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:30:32.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:30:32.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.208 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.208 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.208 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.209 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:32.209 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.246127947000005, 2024-09-17T07:30:32.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:30:32.209 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.209 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.209 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.209 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.209 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:30:32.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.210 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:32.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:32.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:32.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:32.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:30:32.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:30:32.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:30:32.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:30:32.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:30:32.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:30:32.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:30:32.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:30:32.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:30:32.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:30:32.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:30:32.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:30:32.214 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.214 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.214 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.214 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.214 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:32.214 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.246108125999996, 2024-09-17T07:30:32.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:30:32.214 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.215 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.215 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.215 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.215 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.215 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.215 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:30:32.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.215 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:32.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:32.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:30:32.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:30:32.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:30:32.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:30:32.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:30:32.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:30:32.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:30:32.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:30:32.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:30:32.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:30:32.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:30:32.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:30:32.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:30:32.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:30:32.219 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.219 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.219 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.219 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.219 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:32.219 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.246083874000007, 2024-09-17T07:30:32.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:30:32.219 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.219 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.220 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.220 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.220 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.220 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.220 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:30:32.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.220 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:32.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:32.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:30:32.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:30:32.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:30:32.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:30:32.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:30:32.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:30:32.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:30:32.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:30:32.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:30:32.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:30:32.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:30:32.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:30:32.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:30:32.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:30:32.224 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.224 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.224 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.224 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.225 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:32.225 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.240661286000005, 2024-09-17T07:30:32.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:30:32.225 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.225 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.225 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.225 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.225 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.225 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.225 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:30:32.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.225 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:32.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:32.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:30:32.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:30:32.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:30:32.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:30:32.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:30:32.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:30:32.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:30:32.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:30:32.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:30:32.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:30:32.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:30:32.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:30:32.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:30:32.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:30:32.228 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.229 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.229 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.229 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:32.229 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.240614410999996, 2024-09-17T07:30:32.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:30:32.229 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.229 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.229 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.229 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.229 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.229 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.230 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:30:32.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.230 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:32.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:32.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:32.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:30:32.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:30:32.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:30:32.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:30:32.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:30:32.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:30:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:30:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:30:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:30:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:30:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:30:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:30:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:30:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.233 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:32.234 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.240599707000001, 2024-09-17T07:30:32.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:30:32.234 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.234 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.234 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.234 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.235 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.235 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.235 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:30:32.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.235 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:32.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:32.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:32.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:32.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:30:32.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:30:32.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:30:32.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:30:32.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:30:32.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:30:32.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:30:32.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:30:32.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:30:32.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:30:32.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:30:32.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:30:32.238 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.238 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.238 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.238 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.239 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:32.239 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.240576611999998, 2024-09-17T07:30:32.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:30:32.239 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.239 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.239 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.239 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:30:32.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.239 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:32.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:32.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:32.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:30:32.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:30:32.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:30:32.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:30:32.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:30:32.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:30:32.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:30:32.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:30:32.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:30:32.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:30:32.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:30:32.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:30:32.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:30:32.243 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.243 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.243 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.243 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.243 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.240545237000006, 2024-09-17T07:30:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:30:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:30:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:30:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:30:32.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:30:32.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:30:32.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:30:32.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:30:32.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:30:32.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:30:32.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:30:32.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:30:32.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:30:32.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:30:32.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:30:32.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:30:32.249 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.249 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.249 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.249 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.249 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:32.249 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.240529976999994, 2024-09-17T07:30:32.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:30:32.249 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:30:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:32.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:32.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:30:32.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:30:32.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:30:32.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:30:32.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:30:32.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:30:32.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:30:32.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:30:32.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:30:32.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:30:32.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:30:32.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:30:32.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:30:32.254 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.254 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.254 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.254 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.254 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:32.254 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.240509907000003, 2024-09-17T07:30:32.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:30:32.255 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.255 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.255 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.255 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.255 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.255 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.255 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:30:32.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.255 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:32.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:32.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:32.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:30:32.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:30:32.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:30:32.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:30:32.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:30:32.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:30:32.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:30:32.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:30:32.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:30:32.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:30:32.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:30:32.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:30:32.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:30:32.258 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.259 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.259 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.259 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:32.259 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.240497966999996, 2024-09-17T07:30:32.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:30:32.259 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.259 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.259 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.260 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:30:32.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.260 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:32.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:32.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:32.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:30:32.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:30:32.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:30:32.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:30:32.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:30:32.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:30:32.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:30:32.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:30:32.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:30:32.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:30:32.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:30:32.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:30:32.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:30:32.263 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.264 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.264 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.264 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:32.264 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:32.264 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.240474809000006, 2024-09-17T07:30:32.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:30:32.264 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.264 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.264 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.264 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.264 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.264 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.265 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:30:32.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.265 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:32.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:32.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:30:32.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:30:32.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:30:32.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:30:32.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:30:32.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:30:32.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:30:32.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:30:32.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:30:32.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:30:32.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:30:32.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:30:32.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:30:32.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:30:32.268 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.268 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.268 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.269 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:115644 3.15 3:af37e0f6:::rbd_header.118311e0d6f8:head [watch reconnect cookie 140227662450992 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T07:30:32.269 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:30:32.269 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 15.097317915, 2024-09-17T07:30:32.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.013658722, 2024-09-17T07:30:32.270 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.270 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.270 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:32.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.270 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115644 2024-09-17T07:30:32.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.270 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:30:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:30:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077907+0000", 2024-09-17T07:30:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.08e-06 2024-09-17T07:30:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077916+0000", 2024-09-17T07:30:32.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-09-17T07:30:32.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077938+0000", 2024-09-17T07:30:32.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1883999999999999e-05 2024-09-17T07:30:32.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077950+0000", 2024-09-17T07:30:32.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2298e-05 2024-09-17T07:30:32.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.078037+0000", 2024-09-17T07:30:32.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6726999999999998e-05 2024-09-17T07:30:32.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-17T07:30:32.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.078046+0000", 2024-09-17T07:30:32.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6010000000000001e-06 2024-09-17T07:30:32.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085705+0000", 2024-09-17T07:30:32.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007659084 2024-09-17T07:30:32.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:32.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085718+0000", 2024-09-17T07:30:32.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2081e-05 2024-09-17T07:30:32.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.089697+0000", 2024-09-17T07:30:32.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.003979373 2024-09-17T07:30:32.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.089875+0000", 2024-09-17T07:30:32.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017760000000000001 2024-09-17T07:30:32.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.091563+0000", 2024-09-17T07:30:32.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001688836 2024-09-17T07:30:32.275 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.275 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.275 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.276 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:115645 3.15 3:af37e0f6:::rbd_header.118311e0d6f8:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T07:30:32.276 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:30:32.276 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 15.094500395000001, 2024-09-17T07:30:32.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.011649171, 2024-09-17T07:30:32.276 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.276 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.276 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:32.277 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.277 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.277 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.277 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115645 2024-09-17T07:30:32.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.277 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:30:32.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:30:32.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080724+0000", 2024-09-17T07:30:32.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-17T07:30:32.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080735+0000", 2024-09-17T07:30:32.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1544e-05 2024-09-17T07:30:32.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080751+0000", 2024-09-17T07:30:32.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5244e-05 2024-09-17T07:30:32.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080761+0000", 2024-09-17T07:30:32.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0372e-05 2024-09-17T07:30:32.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080834+0000", 2024-09-17T07:30:32.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3263999999999995e-05 2024-09-17T07:30:32.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:32.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080842+0000", 2024-09-17T07:30:32.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2209999999999997e-06 2024-09-17T07:30:32.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085755+0000", 2024-09-17T07:30:32.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049130729999999996 2024-09-17T07:30:32.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:32.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085766+0000", 2024-09-17T07:30:32.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0288e-05 2024-09-17T07:30:32.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.090746+0000", 2024-09-17T07:30:32.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0049801190000001 2024-09-17T07:30:32.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.090798+0000", 2024-09-17T07:30:32.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2502e-05 2024-09-17T07:30:32.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-09-17T07:30:32.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.091037+0000", 2024-09-17T07:30:32.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0002385 2024-09-17T07:30:32.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:32.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092119+0000", 2024-09-17T07:30:32.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010820249999999999 2024-09-17T07:30:32.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:32.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092276+0000", 2024-09-17T07:30:32.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000156737 2024-09-17T07:30:32.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:32.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092330+0000", 2024-09-17T07:30:32.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4666999999999997e-05 2024-09-17T07:30:32.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092371+0000", 2024-09-17T07:30:32.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1043e-05 2024-09-17T07:30:32.283 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.283 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.283 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.284 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:116418 3.13 3:cdf2692d:::rbd_data.1183cb746bb8.0000000000000024:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T07:30:32.284 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:32.284 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 1.0050839979999999, 2024-09-17T07:30:32.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92140365899999999, 2024-09-17T07:30:32.284 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:32.284 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:32.284 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:32.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:32.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:32.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:32.284 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116418 2024-09-17T07:30:32.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.285 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:32.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:32.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:32.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:32.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:32.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:32.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:32.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170140+0000", 2024-09-17T07:30:32.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.398e-06 2024-09-17T07:30:32.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:32.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170144+0000", 2024-09-17T07:30:32.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4399999999999998e-06 2024-09-17T07:30:32.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:32.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170157+0000", 2024-09-17T07:30:32.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2133999999999999e-05 2024-09-17T07:30:32.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:32.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170164+0000", 2024-09-17T07:30:32.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7200000000000006e-06 2024-09-17T07:30:32.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170227+0000", 2024-09-17T07:30:32.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3035999999999998e-05 2024-09-17T07:30:32.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:32.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170238+0000", 2024-09-17T07:30:32.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0397000000000001e-05 2024-09-17T07:30:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086367+0000", 2024-09-17T07:30:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.916128889 2024-09-17T07:30:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086523+0000", 2024-09-17T07:30:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015638800000000001 2024-09-17T07:30:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:30:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086652+0000", 2024-09-17T07:30:32.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000128806 2024-09-17T07:30:32.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:32.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091261+0000", 2024-09-17T07:30:32.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046091020000000003 2024-09-17T07:30:32.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:32.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091449+0000", 2024-09-17T07:30:32.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000188486 2024-09-17T07:30:32.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:32.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091503+0000", 2024-09-17T07:30:32.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3254999999999997e-05 2024-09-17T07:30:32.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:32.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:32.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:32.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091542+0000", 2024-09-17T07:30:32.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9607999999999998e-05 2024-09-17T07:30:32.290 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.290 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.290 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.290 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:32.290 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:32.290 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:32.292 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:32.292 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:32.291+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:32.392 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:32.392+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:32.400 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:32.400 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:32.400 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:32.409 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:32.494 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:32.494+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:32.518 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:30:32.558 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:32.558 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:32.558 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:32.567 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:32.608 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:32.607+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:32.647 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:32.647 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:32.664 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:32.664 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:32.682 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:32.682 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:32.695 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:32.694+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:32.699 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:32.699 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:32.716 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:32.717 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:32.720 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:32.720 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:32.728 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:32.734 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:32.734 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:32.751 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:32.751 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:32.769 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:32.769 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:32.796 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:32.795+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:32.843 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:32.843 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:32.852 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:32.896 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:32.896+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:32.959 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:32.959 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:32.967 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:33.012 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:33.012+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:33.075 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:33.075 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:33.084 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:33.098 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:33.098+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:33.099 INFO:teuthology.orchestra.run.smithi045.stderr:set 2.3 pg_upmap_items mapping to [5->0,4->6] 2024-09-17T07:30:33.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T07:30:33.206 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:33.206+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:33.225 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:33.225 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:33.235 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:33.319 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:33.318+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:33.383 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:33.384 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:33.393 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:33.402 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:33.401+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:33.503 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:33.502+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:33.540 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:33.541 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:33.549 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:33.632 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:33.632+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:33.697 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:33.697 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:33.706 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:33.706 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:33.705+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:33.806 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:33.806+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:33.852 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:33.853 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:33.861 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:33.907 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:33.907+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:33.967 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:33.968 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:33.976 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:34.021 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:34.021+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:34.091 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:34.091 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:34.100 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:34.105 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-17T07:30:34.110 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:34.109+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:34.210 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:34.210+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:34.254 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:34.255 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:34.264 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:34.311 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:34.310+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:34.406 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:34.406 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:34.412 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:34.411+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:34.415 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:34.513 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:34.512+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:34.518 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:34.518 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:34.526 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:34.614 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:34.614+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:34.670 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:34.670 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:34.678 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:34.715 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:34.715+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:34.781 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:30:34.821 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:34.821 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:34.828 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:34.828+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:34.829 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:34.916 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:34.917 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:34.917 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:34.916+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:34.932 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:34.932 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:34.933 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:34.933 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:34.941 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:34.949 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:34.950 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:34.966 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:34.966 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:34.983 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:34.984 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:35.000 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:35.001 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:35.018 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:35.018 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:35.025 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:35.025+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:35.035 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:35.035 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:35.086 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:35.086 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:35.086 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:35.086 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:35.086 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:35.095 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:35.135 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:35.135+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:35.220 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:35.219+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:35.252 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:35.252 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:35.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:35.252 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:35.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.252 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:35.252 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:35.253 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.344544400000004, 2024-09-17T07:30:35.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:30:35.253 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.253 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.254 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.254 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.254 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.254 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.255 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:30:35.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.255 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:35.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:35.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:35.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:30:35.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:30:35.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:30:35.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:30:35.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:30:35.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:30:35.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:30:35.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:30:35.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:30:35.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:30:35.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:30:35.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:30:35.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:30:35.260 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.260 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.261 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:35.261 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:35.261 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.344525723000004, 2024-09-17T07:30:35.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:30:35.261 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.262 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.262 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.262 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:30:35.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.262 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:35.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:35.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:30:35.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:35.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:30:35.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:30:35.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:30:35.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:30:35.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:30:35.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:30:35.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:30:35.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:30:35.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:30:35.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:30:35.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:30:35.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:30:35.267 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.267 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.267 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.268 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:35.268 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:35.268 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.344507370000002, 2024-09-17T07:30:35.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:30:35.268 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.268 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.268 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.268 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.268 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.268 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.268 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:30:35.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.269 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:35.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:35.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:35.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:30:35.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:30:35.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:30:35.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:30:35.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:30:35.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:30:35.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:30:35.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:30:35.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:30:35.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:30:35.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:30:35.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:30:35.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:30:35.272 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.272 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.272 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.272 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:35.272 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:35.272 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.344487771999994, 2024-09-17T07:30:35.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:30:35.273 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.273 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.273 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.273 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.273 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.273 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.273 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:30:35.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.273 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:35.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:35.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:30:35.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:30:35.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:30:35.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:30:35.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:30:35.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:30:35.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:30:35.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:30:35.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:30:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:30:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:30:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:30:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:30:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:30:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.344467816999995, 2024-09-17T07:30:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:30:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.277 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:30:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:30:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:30:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:30:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:30:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:30:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:30:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:30:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:30:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:30:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:30:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:30:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:30:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:30:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.344449162000004, 2024-09-17T07:30:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:30:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:30:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:30:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:30:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:30:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:30:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:30:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:30:35.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:30:35.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:30:35.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:30:35.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:30:35.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:30:35.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:30:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:30:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.344430685999995, 2024-09-17T07:30:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:30:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.287 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.287 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.287 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:30:35.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.287 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:35.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:35.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:30:35.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:30:35.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:30:35.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:30:35.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:30:35.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:30:35.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:30:35.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:30:35.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:30:35.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:30:35.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:30:35.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:30:35.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:30:35.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:30:35.291 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.291 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.291 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.291 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:35.291 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:35.291 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.344411086999997, 2024-09-17T07:30:35.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:30:35.291 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.291 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.292 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.292 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:30:35.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.292 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:35.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:35.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:35.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:30:35.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:30:35.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:30:35.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:30:35.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:30:35.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:30:35.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:30:35.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:30:35.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:30:35.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:30:35.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:30:35.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:30:35.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:30:35.295 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.296 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.296 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.296 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:35.296 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:35.296 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.344392511999999, 2024-09-17T07:30:35.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:30:35.296 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.296 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.296 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.297 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.297 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.297 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.297 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:30:35.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.297 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:35.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:35.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:30:35.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:30:35.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:30:35.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:30:35.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:30:35.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:30:35.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:30:35.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:30:35.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:30:35.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:30:35.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:30:35.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:30:35.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:30:35.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:30:35.300 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.300 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.300 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.301 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:35.301 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:35.301 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.344370084999994, 2024-09-17T07:30:35.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:30:35.301 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.301 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.301 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.301 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.301 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.301 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.301 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:30:35.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.302 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:35.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:35.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:35.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:35.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:30:35.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:30:35.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:30:35.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:30:35.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:30:35.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:30:35.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:30:35.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:30:35.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:30:35.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:30:35.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:30:35.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:30:35.305 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.305 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.305 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.305 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:35.305 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:35.305 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.344342268000005, 2024-09-17T07:30:35.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:30:35.305 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.305 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.305 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.306 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.306 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.306 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.306 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:30:35.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.306 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:35.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:35.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:35.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:30:35.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:30:35.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:30:35.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:30:35.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:30:35.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:30:35.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:30:35.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:30:35.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:30:35.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:30:35.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:30:35.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:30:35.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:30:35.310 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.310 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.310 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.310 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:35.310 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:35.310 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.344320323999995, 2024-09-17T07:30:35.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:30:35.310 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.310 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.310 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:35.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:30:35.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:30:35.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:30:35.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:30:35.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:30:35.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:30:35.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:30:35.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:30:35.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:30:35.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:30:35.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:30:35.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:30:35.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:30:35.314 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.314 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.314 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.314 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:35.314 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:35.314 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.344299219999996, 2024-09-17T07:30:35.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:30:35.315 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.315 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.315 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.315 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.315 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.315 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.315 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:30:35.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.315 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:35.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:35.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:35.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:30:35.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:30:35.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:30:35.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:30:35.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:35.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:30:35.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:30:35.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:30:35.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:30:35.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:30:35.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:30:35.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:30:35.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:30:35.318 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.318 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.318 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.319 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:35.319 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:35.319 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.344279616999998, 2024-09-17T07:30:35.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:30:35.319 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.319 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.319 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.319 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.319 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.319 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.319 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:30:35.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.319 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:35.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:35.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:30:35.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:30:35.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:30:35.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:30:35.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:30:35.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:30:35.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:30:35.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:30:35.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:30:35.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:30:35.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:30:35.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:30:35.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:30:35.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:30:35.323 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.323 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.323 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.323 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:35.323 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:35.323 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.391338504000004, 2024-09-17T07:30:35.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:30:35.323 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.323 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.324 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.324 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.324 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.324 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.324 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:30:35.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.324 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:35.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:35.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:30:35.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:30:35.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:30:35.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:35.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:30:35.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:30:35.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:30:35.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:30:35.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:30:35.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:30:35.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:30:35.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:30:35.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:30:35.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:30:35.327 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.327 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.327 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.328 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:35.328 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:35.328 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.391324843999996, 2024-09-17T07:30:35.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:30:35.328 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.328 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.328 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.328 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.328 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.328 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.328 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:30:35.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.328 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:35.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:35.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:35.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:30:35.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:30:35.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:30:35.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:30:35.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:30:35.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:30:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:30:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:30:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:30:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:30:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:30:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:30:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:30:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.391310062000002, 2024-09-17T07:30:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:30:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.332 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.333 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:30:35.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.333 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:35.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:35.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:35.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:30:35.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:30:35.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:30:35.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:30:35.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:35.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:30:35.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:30:35.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:30:35.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:30:35.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:30:35.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:30:35.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:30:35.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:30:35.336 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.336 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.336 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.337 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:35.337 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:35.337 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.391299171, 2024-09-17T07:30:35.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:30:35.337 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.337 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.337 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.337 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:30:35.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.338 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:35.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:35.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:35.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:35.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:30:35.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:30:35.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:30:35.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:30:35.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:30:35.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:30:35.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:30:35.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:30:35.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:30:35.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:30:35.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:30:35.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:30:35.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.341 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.342 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:35.342 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:35.342 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.391289611000005, 2024-09-17T07:30:35.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:30:35.342 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.342 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.342 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.342 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:30:35.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.343 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:35.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:35.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:30:35.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:35.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:30:35.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:30:35.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:30:35.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:30:35.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:30:35.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:30:35.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:30:35.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:30:35.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:30:35.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:30:35.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:30:35.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:30:35.346 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.346 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.346 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.346 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:35.346 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:35.347 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.391279050999998, 2024-09-17T07:30:35.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:30:35.347 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.347 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.347 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.347 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.347 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.347 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.347 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:30:35.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.347 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:35.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:35.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:30:35.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:30:35.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:30:35.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:30:35.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:30:35.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:30:35.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:30:35.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:30:35.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:30:35.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:30:35.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:30:35.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:30:35.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:30:35.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:30:35.350 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.351 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.351 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.351 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:35.352 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:35.353 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:35.352+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:35.452 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:35.452+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:35.497 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:35.497 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:35.497 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:35.497 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:35.505 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:35.553 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:35.553+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:35.650 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:35.650 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:35.654 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:35.653+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:35.659 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:35.754 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:35.754+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:35.763 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:35.763 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:35.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:35.764 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:35.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.764 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:30:35.764 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:35.764 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.42763884999999, 2024-09-17T07:30:35.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:30:35.764 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.764 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.764 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:35.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:30:35.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:30:35.764 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:30:35.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.765 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:35.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:35.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:30:35.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:30:35.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:30:35.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:30:35.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:30:35.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:30:35.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:30:35.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:30:35.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:30:35.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:30:35.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:35.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:30:35.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:30:35.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:30:35.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:30:35.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:30:35.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:30:35.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:35.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:30:35.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:30:35.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:35.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:30:35.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:30:35.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:35.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:30:35.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:30:35.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:35.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:30:35.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:30:35.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:30:35.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:30:35.770 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.770 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.770 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.770 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.770 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:35.770 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.83858669999999, 2024-09-17T07:30:35.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:30:35.770 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.770 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.770 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.771 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.771 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.771 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.771 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:30:35.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.771 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:35.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:35.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:30:35.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:30:35.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:30:35.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:30:35.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:30:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:30:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:30:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:30:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.774 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.83330358400001, 2024-09-17T07:30:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:30:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:30:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:30:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:30:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:30:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:30:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:30:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:30:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:30:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:30:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:30:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:30:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:30:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:30:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:30:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.778 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.833272604, 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.779 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:30:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:30:35.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:30:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:30:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:30:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:30:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:30:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:30:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.782 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.833242555, 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.783 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:30:35.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:30:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:30:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:30:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:30:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:30:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:30:35.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:30:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.833214148, 2024-09-17T07:30:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:30:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.787 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:30:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:30:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:30:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:30:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:30:35.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:30:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:30:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.833194327, 2024-09-17T07:30:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:30:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.791 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:30:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:30:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:30:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:30:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:30:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:30:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:30:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:30:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:30:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:30:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:30:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.833170075, 2024-09-17T07:30:35.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:30:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:30:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.827747487, 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:30:35.799 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:30:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:30:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:30:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:30:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:30:35.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:30:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:30:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:30:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:30:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:30:35.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:30:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:30:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:30:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:30:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:30:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.827700612, 2024-09-17T07:30:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:30:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:30:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.804 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:35.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:30:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:30:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:30:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:30:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:30:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:30:35.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:30:35.807 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.82768590800001, 2024-09-17T07:30:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:30:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.808 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:30:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:30:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:30:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:30:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:30:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:30:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:30:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:30:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:30:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:30:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:30:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:30:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:30:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.827662813, 2024-09-17T07:30:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:30:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.812 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:30:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:30:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:30:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:30:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:30:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:30:35.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.827631438, 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.816 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:35.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:30:35.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:30:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:30:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.827616178, 2024-09-17T07:30:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:30:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.820 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:30:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:30:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:30:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:30:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:30:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:30:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:30:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:30:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:30:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:30:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:30:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:30:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:30:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:30:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.824 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.82759610799999, 2024-09-17T07:30:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:30:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:30:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:30:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:30:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:30:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:30:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:30:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:30:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:30:35.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:30:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:30:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:30:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:30:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:30:35.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:30:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.827584168, 2024-09-17T07:30:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:30:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.829 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:30:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:30:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:30:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:30:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:30:35.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:30:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:30:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:30:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:30:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:30:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:30:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:30:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:30:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:30:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:35.833 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.82756101, 2024-09-17T07:30:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:30:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:30:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:35.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:30:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:30:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:30:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:30:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:30:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:30:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:30:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:30:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:30:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:30:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:30:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:30:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:30:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:30:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.837 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:115644 3.15 3:af37e0f6:::rbd_header.118311e0d6f8:head [watch reconnect cookie 140227662450992 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T07:30:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:30:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.684404116, 2024-09-17T07:30:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.013658722, 2024-09-17T07:30:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115644 2024-09-17T07:30:35.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:30:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:30:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077907+0000", 2024-09-17T07:30:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.08e-06 2024-09-17T07:30:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077916+0000", 2024-09-17T07:30:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-09-17T07:30:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077938+0000", 2024-09-17T07:30:35.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1883999999999999e-05 2024-09-17T07:30:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077950+0000", 2024-09-17T07:30:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2298e-05 2024-09-17T07:30:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.078037+0000", 2024-09-17T07:30:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6726999999999998e-05 2024-09-17T07:30:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-17T07:30:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.078046+0000", 2024-09-17T07:30:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6010000000000001e-06 2024-09-17T07:30:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085705+0000", 2024-09-17T07:30:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007659084 2024-09-17T07:30:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085718+0000", 2024-09-17T07:30:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2081e-05 2024-09-17T07:30:35.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.089697+0000", 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.003979373 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.089875+0000", 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017760000000000001 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.091563+0000", 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001688836 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.843 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:115645 3.15 3:af37e0f6:::rbd_header.118311e0d6f8:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T07:30:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:30:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.681586595999999, 2024-09-17T07:30:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.011649171, 2024-09-17T07:30:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115645 2024-09-17T07:30:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:30:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:30:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080724+0000", 2024-09-17T07:30:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-17T07:30:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080735+0000", 2024-09-17T07:30:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1544e-05 2024-09-17T07:30:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080751+0000", 2024-09-17T07:30:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5244e-05 2024-09-17T07:30:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080761+0000", 2024-09-17T07:30:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0372e-05 2024-09-17T07:30:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080834+0000", 2024-09-17T07:30:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3263999999999995e-05 2024-09-17T07:30:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080842+0000", 2024-09-17T07:30:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2209999999999997e-06 2024-09-17T07:30:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085755+0000", 2024-09-17T07:30:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049130729999999996 2024-09-17T07:30:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085766+0000", 2024-09-17T07:30:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0288e-05 2024-09-17T07:30:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.090746+0000", 2024-09-17T07:30:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0049801190000001 2024-09-17T07:30:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.090798+0000", 2024-09-17T07:30:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2502e-05 2024-09-17T07:30:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-09-17T07:30:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.091037+0000", 2024-09-17T07:30:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0002385 2024-09-17T07:30:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092119+0000", 2024-09-17T07:30:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010820249999999999 2024-09-17T07:30:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092276+0000", 2024-09-17T07:30:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000156737 2024-09-17T07:30:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092330+0000", 2024-09-17T07:30:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4666999999999997e-05 2024-09-17T07:30:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092371+0000", 2024-09-17T07:30:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1043e-05 2024-09-17T07:30:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:116418 3.13 3:cdf2692d:::rbd_data.1183cb746bb8.0000000000000024:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T07:30:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.5921701989999999, 2024-09-17T07:30:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92140365899999999, 2024-09-17T07:30:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:35.851 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116418 2024-09-17T07:30:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:35.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170140+0000", 2024-09-17T07:30:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.398e-06 2024-09-17T07:30:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170144+0000", 2024-09-17T07:30:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4399999999999998e-06 2024-09-17T07:30:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170157+0000", 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2133999999999999e-05 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170164+0000", 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7200000000000006e-06 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170227+0000", 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3035999999999998e-05 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:35.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170238+0000", 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0397000000000001e-05 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086367+0000", 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.916128889 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086523+0000", 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015638800000000001 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086652+0000", 2024-09-17T07:30:35.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000128806 2024-09-17T07:30:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091261+0000", 2024-09-17T07:30:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046091020000000003 2024-09-17T07:30:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091449+0000", 2024-09-17T07:30:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000188486 2024-09-17T07:30:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091503+0000", 2024-09-17T07:30:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3254999999999997e-05 2024-09-17T07:30:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091542+0000", 2024-09-17T07:30:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9607999999999998e-05 2024-09-17T07:30:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:35.857 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:35.858 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:35.859 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:35.859 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:35.858+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:35.960 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:35.960+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:35.977 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:35.978 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:35.978 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:35.978 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:35.987 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:36.061 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:36.060+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:36.094 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:36.094 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:36.095 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:36.095 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:36.095 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:36.103 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:36.126 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T07:30:36.191 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:36.191+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:36.227 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:36.227 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:36.227 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:36.227 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:36.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.228 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ae97839c:::rbd_data.11839a97b753.0000000000000004:head, version: 55'111, data_included: [0~16384,81920~16384,163840~16384,196608~16384,245760~16384,278528~16384,360448~16384,442368~16384,524288~16384,704512~16384,786432~16384,868352~16384,950272~16384,983040~16384,1032192~16384,1064960~16384,1146880~16384,1228800~16384,1310720~16384,1343488~16384,1392640~16384,1425408~32768,1523712~16384], data_size: 393216, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:ae97839c:::rbd_data.11839a97b753.0000000000000004:head@55'111, size: 1540096, copy_subset: [(0, 1540096)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1540096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:36.228 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.884937+0000", 2024-09-17T07:30:36.228 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.340358131999999, 2024-09-17T07:30:36.228 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025497419999999998, 2024-09-17T07:30:36.228 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.228 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.228 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:36.228 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.228 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.228 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.229 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.884937+0000", 2024-09-17T07:30:36.229 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.229 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.229 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.884937+0000", 2024-09-17T07:30:36.229 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.229 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.229 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.884938+0000", 2024-09-17T07:30:36.229 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5499999999999998e-07 2024-09-17T07:30:36.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.229 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.230 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.230 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885283+0000", 2024-09-17T07:30:36.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00034585300000000002 2024-09-17T07:30:36.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.230 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.230 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885301+0000", 2024-09-17T07:30:36.230 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7207999999999999e-05 2024-09-17T07:30:36.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.230 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.230 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885316+0000", 2024-09-17T07:30:36.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4873e-05 2024-09-17T07:30:36.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885416+0000", 2024-09-17T07:30:36.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000100456 2024-09-17T07:30:36.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.231 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.231 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.231 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885432+0000", 2024-09-17T07:30:36.231 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6390000000000001e-05 2024-09-17T07:30:36.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.232 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.232 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.887487+0000", 2024-09-17T07:30:36.232 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020545070000000001 2024-09-17T07:30:36.232 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.232 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.232 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.232 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.233 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ab91cfbe:::rbd_data.118394b259f9.0000000000000000:1e, version: 68'159, data_included: [16384~81920,180224~81920,344064~92788,507904~81920,595536~49296,671744~81920,835584~81920,999424~49152,5169889~28787], data_size: 629623, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:ab91cfbe:::rbd_data.118394b259f9.0000000000000000:1e@68'159, size: 5242880, copy_subset: [(0, 5242880)], clone_subset: {}, snapset: 1e=[]:{1e: [1e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5242880, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:36.233 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.917685+0000", 2024-09-17T07:30:36.233 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.307610368999999, 2024-09-17T07:30:36.233 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023928310000000002, 2024-09-17T07:30:36.233 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.234 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:36.234 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.234 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.234 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.234 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.917685+0000", 2024-09-17T07:30:36.234 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.235 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.235 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.917685+0000", 2024-09-17T07:30:36.235 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.235 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.236 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.917686+0000", 2024-09-17T07:30:36.236 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2399999999999998e-07 2024-09-17T07:30:36.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.236 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.236 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.918262+0000", 2024-09-17T07:30:36.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00057645199999999998 2024-09-17T07:30:36.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.237 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.237 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.918279+0000", 2024-09-17T07:30:36.237 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7277000000000001e-05 2024-09-17T07:30:36.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.238 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.238 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.918292+0000", 2024-09-17T07:30:36.238 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2303000000000001e-05 2024-09-17T07:30:36.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.239 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.918382+0000", 2024-09-17T07:30:36.239 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0469000000000004e-05 2024-09-17T07:30:36.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.239 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.240 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.918399+0000", 2024-09-17T07:30:36.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6565000000000001e-05 2024-09-17T07:30:36.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.240 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.240 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.920078+0000", 2024-09-17T07:30:36.240 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016791410000000001 2024-09-17T07:30:36.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.240 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.241 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.241 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9, version: 36'125, data_included: [1048576~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9@36'125, size: 10246384, copy_subset: [(0, 10246384)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:36.241 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.176147+0000", 2024-09-17T07:30:36.241 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.049148224, 2024-09-17T07:30:36.241 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022794080000000001, 2024-09-17T07:30:36.241 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.241 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.241 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:36.241 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.176147+0000", 2024-09-17T07:30:36.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.176147+0000", 2024-09-17T07:30:36.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.242 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.242 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.176149+0000", 2024-09-17T07:30:36.242 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4699999999999999e-06 2024-09-17T07:30:36.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.177787+0000", 2024-09-17T07:30:36.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001638371 2024-09-17T07:30:36.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.177802+0000", 2024-09-17T07:30:36.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5336e-05 2024-09-17T07:30:36.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.243 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.243 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.177812+0000", 2024-09-17T07:30:36.243 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8079999999999999e-06 2024-09-17T07:30:36.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.244 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.177878+0000", 2024-09-17T07:30:36.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5926999999999994e-05 2024-09-17T07:30:36.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.244 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.244 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.177886+0000", 2024-09-17T07:30:36.244 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2630000000000003e-06 2024-09-17T07:30:36.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.245 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.245 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.178427+0000", 2024-09-17T07:30:36.245 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054023299999999999 2024-09-17T07:30:36.245 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.245 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.245 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.245 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9, version: 36'125, data_included: [5478708~26316,5767168~524288,6553600~497972], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9@36'125, size: 10246384, copy_subset: [(0, 10246384)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7051572, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5478708, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:36.245 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:36.245 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.035890639000002, 2024-09-17T07:30:36.245 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024847990000000002, 2024-09-17T07:30:36.245 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.246 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.246 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:36.246 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:36.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.246 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.246 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:36.246 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:36.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:30:36.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191002+0000", 2024-09-17T07:30:36.247 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015967100000000001 2024-09-17T07:30:36.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.247 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.247 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191017+0000", 2024-09-17T07:30:36.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4717e-05 2024-09-17T07:30:36.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.248 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.248 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191036+0000", 2024-09-17T07:30:36.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9882999999999999e-05 2024-09-17T07:30:36.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.248 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.248 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.248 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191120+0000", 2024-09-17T07:30:36.248 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3313999999999995e-05 2024-09-17T07:30:36.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.249 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.249 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191128+0000", 2024-09-17T07:30:36.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5499999999999995e-06 2024-09-17T07:30:36.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.249 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.249 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.249 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191889+0000", 2024-09-17T07:30:36.249 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076114100000000001 2024-09-17T07:30:36.249 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.249 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.249 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.250 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.250 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head, version: 36'126, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head@36'126, size: 9431460, copy_subset: [], clone_subset: {3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9: [(0, 9431460)]}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:36.250 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:36.250 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.021505820000002, 2024-09-17T07:30:36.250 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002812681, 2024-09-17T07:30:36.250 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.251 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:36.251 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:36.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.251 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:36.251 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.251 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.251 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:36.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9100000000000004e-07 2024-09-17T07:30:36.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.252 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203801+0000", 2024-09-17T07:30:36.252 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0696000000000001e-05 2024-09-17T07:30:36.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.252 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.252 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203817+0000", 2024-09-17T07:30:36.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6480000000000001e-05 2024-09-17T07:30:36.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203829+0000", 2024-09-17T07:30:36.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1935e-05 2024-09-17T07:30:36.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.253 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.253 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203891+0000", 2024-09-17T07:30:36.253 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2079e-05 2024-09-17T07:30:36.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203900+0000", 2024-09-17T07:30:36.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4999999999999999e-06 2024-09-17T07:30:36.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.254 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.254 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.206602+0000", 2024-09-17T07:30:36.254 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027024000000000002 2024-09-17T07:30:36.254 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.254 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.255 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.255 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head, version: 82'426, data_included: [0~786432,981325~41682,1966080~6721], data_size: 834835, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head@82'426, size: 3276800, copy_subset: [(0, 3276800)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3276800, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:36.255 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.308228+0000", 2024-09-17T07:30:36.255 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.917066832, 2024-09-17T07:30:36.255 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025329110000000001, 2024-09-17T07:30:36.255 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.255 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:36.255 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.255 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.255 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.308228+0000", 2024-09-17T07:30:36.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.308228+0000", 2024-09-17T07:30:36.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.256 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.256 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.308229+0000", 2024-09-17T07:30:36.256 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:30:36.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.256 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.257 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309432+0000", 2024-09-17T07:30:36.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001202704 2024-09-17T07:30:36.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.257 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.257 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.257 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309447+0000", 2024-09-17T07:30:36.257 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5184000000000001e-05 2024-09-17T07:30:36.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.258 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.258 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.259 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309457+0000", 2024-09-17T07:30:36.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9129999999999998e-06 2024-09-17T07:30:36.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309544+0000", 2024-09-17T07:30:36.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7249000000000004e-05 2024-09-17T07:30:36.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.260 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.260 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.260 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309558+0000", 2024-09-17T07:30:36.260 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3781000000000001e-05 2024-09-17T07:30:36.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.261 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.261 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.310761+0000", 2024-09-17T07:30:36.261 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012035839999999999 2024-09-17T07:30:36.261 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.261 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.261 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.261 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.261 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.c 108/106 [PushOp(3:3034d26c:::rbd_data.11835a3b5a27.0000000000000039:1c, version: 66'255, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:3034d26c:::rbd_data.11835a3b5a27.0000000000000039:1c@66'255, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 1c=[]:{1c: [1c]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:36.261 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.781537+0000", 2024-09-17T07:30:36.261 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.443757835, 2024-09-17T07:30:36.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0026008479999999998, 2024-09-17T07:30:36.262 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.262 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.262 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:36.262 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.781537+0000", 2024-09-17T07:30:36.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.262 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.262 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.781537+0000", 2024-09-17T07:30:36.262 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.781538+0000", 2024-09-17T07:30:36.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0399999999999996e-07 2024-09-17T07:30:36.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.263 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.783137+0000", 2024-09-17T07:30:36.263 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015992739999999999 2024-09-17T07:30:36.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.263 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.263 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.783153+0000", 2024-09-17T07:30:36.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5608e-05 2024-09-17T07:30:36.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.783163+0000", 2024-09-17T07:30:36.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0028e-05 2024-09-17T07:30:36.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.264 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.264 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.783247+0000", 2024-09-17T07:30:36.264 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4099999999999998e-05 2024-09-17T07:30:36.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.264 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.783261+0000", 2024-09-17T07:30:36.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4191e-05 2024-09-17T07:30:36.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.265 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.265 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.784138+0000", 2024-09-17T07:30:36.265 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087714300000000004 2024-09-17T07:30:36.265 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.265 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.265 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.265 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.265 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.c 108/106 [PushOp(3:34edcbef:::rbd_data.11831719e038.0000000000000000:head, version: 68'273, data_included: [16384~81920,180224~81920,344064~92788,507904~81920,595536~49296,671744~81920,835584~81920,999424~81920,2605548~8985,2708290~8096,2995581~42933], data_size: 693618, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:34edcbef:::rbd_data.11831719e038.0000000000000000:head@68'273, size: 3038514, copy_subset: [(0, 3038514)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3038514, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:36.266 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.799290+0000", 2024-09-17T07:30:36.266 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.426005471, 2024-09-17T07:30:36.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025209730000000001, 2024-09-17T07:30:36.266 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.266 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:36.266 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799290+0000", 2024-09-17T07:30:36.266 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.266 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.266 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.266 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799290+0000", 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799290+0000", 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799886+0000", 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00059524899999999995 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799903+0000", 2024-09-17T07:30:36.267 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7689999999999998e-05 2024-09-17T07:30:36.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799917+0000", 2024-09-17T07:30:36.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3189e-05 2024-09-17T07:30:36.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.800010+0000", 2024-09-17T07:30:36.268 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3690999999999995e-05 2024-09-17T07:30:36.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.268 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.268 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.268 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.800025+0000", 2024-09-17T07:30:36.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4453999999999999e-05 2024-09-17T07:30:36.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.269 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.269 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.801811+0000", 2024-09-17T07:30:36.269 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001786096 2024-09-17T07:30:36.269 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.269 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.269 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.270 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.18 109/106 [PushOp(3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e, version: 44'134, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e@44'134, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:36.270 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:18.083461+0000", 2024-09-17T07:30:36.270 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.141834344999999, 2024-09-17T07:30:36.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003057976, 2024-09-17T07:30:36.270 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.270 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:36.270 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.270 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.270 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.270 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083461+0000", 2024-09-17T07:30:36.270 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083461+0000", 2024-09-17T07:30:36.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083462+0000", 2024-09-17T07:30:36.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1500000000000004e-07 2024-09-17T07:30:36.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.271 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.271 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.271 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083603+0000", 2024-09-17T07:30:36.271 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140863 2024-09-17T07:30:36.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083623+0000", 2024-09-17T07:30:36.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.003e-05 2024-09-17T07:30:36.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.272 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.272 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083642+0000", 2024-09-17T07:30:36.272 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9734000000000001e-05 2024-09-17T07:30:36.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.272 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.084564+0000", 2024-09-17T07:30:36.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00092196200000000002 2024-09-17T07:30:36.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.084580+0000", 2024-09-17T07:30:36.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5464e-05 2024-09-17T07:30:36.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.273 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.273 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.273 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.086519+0000", 2024-09-17T07:30:36.273 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001939208 2024-09-17T07:30:36.274 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.274 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.274 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.274 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115819 3.1e 3:78168739:::rbd_header.118340e4fbf:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:36.274 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.123523+0000", 2024-09-17T07:30:36.274 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.101772793, 2024-09-17T07:30:36.274 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002282184, 2024-09-17T07:30:36.274 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:36.274 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.274 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.275 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.275 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115819 2024-09-17T07:30:36.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.275 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.123523+0000", 2024-09-17T07:30:36.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.275 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.275 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.275 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.123523+0000", 2024-09-17T07:30:36.275 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.123525+0000", 2024-09-17T07:30:36.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2670000000000001e-06 2024-09-17T07:30:36.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.276 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.123534+0000", 2024-09-17T07:30:36.276 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6649999999999998e-06 2024-09-17T07:30:36.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.276 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.123558+0000", 2024-09-17T07:30:36.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3044000000000002e-05 2024-09-17T07:30:36.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.123568+0000", 2024-09-17T07:30:36.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0766e-05 2024-09-17T07:30:36.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.277 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.123658+0000", 2024-09-17T07:30:36.277 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9969000000000005e-05 2024-09-17T07:30:36.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.277 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.277 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:30:36.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.123666+0000", 2024-09-17T07:30:36.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4730000000000003e-06 2024-09-17T07:30:36.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.125436+0000", 2024-09-17T07:30:36.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017703090000000001 2024-09-17T07:30:36.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.278 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.125533+0000", 2024-09-17T07:30:36.278 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6821e-05 2024-09-17T07:30:36.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.278 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.278 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.125805+0000", 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00027187000000000001 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115899 3.b 3:d406e0ff:::rbd_data.118340e4fbf.0000000000000001:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.034360673, 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0040641219999999999, 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.279 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.280 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.280 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115899 2024-09-17T07:30:36.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.280 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:36.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.280 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.280 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:36.280 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.280 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190936+0000", 2024-09-17T07:30:36.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.26e-06 2024-09-17T07:30:36.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.281 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.281 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190941+0000", 2024-09-17T07:30:36.281 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0499999999999999e-06 2024-09-17T07:30:36.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.281 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190958+0000", 2024-09-17T07:30:36.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6548000000000001e-05 2024-09-17T07:30:36.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190966+0000", 2024-09-17T07:30:36.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6799999999999999e-06 2024-09-17T07:30:36.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.282 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.282 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191028+0000", 2024-09-17T07:30:36.282 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2082999999999995e-05 2024-09-17T07:30:36.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191161+0000", 2024-09-17T07:30:36.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000132461 2024-09-17T07:30:36.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:36.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191284+0000", 2024-09-17T07:30:36.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012301200000000001 2024-09-17T07:30:36.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.283 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.283 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:36.283 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192371+0000", 2024-09-17T07:30:36.283 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00108709 2024-09-17T07:30:36.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:36.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194898+0000", 2024-09-17T07:30:36.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025272710000000002 2024-09-17T07:30:36.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:36.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194957+0000", 2024-09-17T07:30:36.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8792000000000002e-05 2024-09-17T07:30:36.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.284 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.284 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.284 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194999+0000", 2024-09-17T07:30:36.284 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1875000000000001e-05 2024-09-17T07:30:36.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.285 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.285 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115901 3.1e 3:7f62d392:::rbd_data.118340e4fbf.0000000000000003:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:36.285 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.191679+0000", 2024-09-17T07:30:36.285 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.033616329000001, 2024-09-17T07:30:36.285 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024544079999999999, 2024-09-17T07:30:36.285 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.285 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.285 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:36.285 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.285 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.286 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.286 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115901 2024-09-17T07:30:36.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.286 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191679+0000", 2024-09-17T07:30:36.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.286 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.286 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.286 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191679+0000", 2024-09-17T07:30:36.286 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191680+0000", 2024-09-17T07:30:36.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6499999999999998e-07 2024-09-17T07:30:36.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191684+0000", 2024-09-17T07:30:36.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1479999999999999e-06 2024-09-17T07:30:36.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.287 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.287 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191697+0000", 2024-09-17T07:30:36.287 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3086e-05 2024-09-17T07:30:36.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191714+0000", 2024-09-17T07:30:36.288 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7147999999999999e-05 2024-09-17T07:30:36.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.288 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.288 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.288 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192295+0000", 2024-09-17T07:30:36.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00058122000000000002 2024-09-17T07:30:36.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192413+0000", 2024-09-17T07:30:36.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000117778 2024-09-17T07:30:36.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.289 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-17T07:30:36.289 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192515+0000", 2024-09-17T07:30:36.289 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010200900000000001 2024-09-17T07:30:36.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:36.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193831+0000", 2024-09-17T07:30:36.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001315472 2024-09-17T07:30:36.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.290 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.290 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:36.290 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194034+0000", 2024-09-17T07:30:36.290 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00020312600000000001 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194083+0000", 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9132999999999999e-05 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194133+0000", 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0423000000000002e-05 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.291 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115903 3.18 3:1fc9b9ef:::rbd_data.118340e4fbf.0000000000000005:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:36.292 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:36.292 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.033562549999999, 2024-09-17T07:30:36.292 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00477466, 2024-09-17T07:30:36.292 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.292 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.292 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:36.292 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.292 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.292 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115903 2024-09-17T07:30:36.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.292 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.292 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:36.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.293 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:36.293 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.293 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.293 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:36.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9800000000000004e-07 2024-09-17T07:30:36.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191736+0000", 2024-09-17T07:30:36.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6620000000000001e-06 2024-09-17T07:30:36.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.294 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.294 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191743+0000", 2024-09-17T07:30:36.294 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6939999999999998e-06 2024-09-17T07:30:36.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191748+0000", 2024-09-17T07:30:36.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9749999999999998e-06 2024-09-17T07:30:36.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191823+0000", 2024-09-17T07:30:36.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4954000000000001e-05 2024-09-17T07:30:36.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.295 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.295 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191933+0000", 2024-09-17T07:30:36.295 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000110708 2024-09-17T07:30:36.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.295 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:30:36.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192031+0000", 2024-09-17T07:30:36.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7974999999999996e-05 2024-09-17T07:30:36.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:36.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193329+0000", 2024-09-17T07:30:36.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012976419999999999 2024-09-17T07:30:36.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.296 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.296 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:36.296 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196404+0000", 2024-09-17T07:30:36.296 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003075174 2024-09-17T07:30:36.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:36.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196461+0000", 2024-09-17T07:30:36.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6663999999999998e-05 2024-09-17T07:30:36.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.297 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.297 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.297 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196507+0000", 2024-09-17T07:30:36.297 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6714e-05 2024-09-17T07:30:36.297 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.297 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.297 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115905 3.18 3:19f08345:::rbd_data.118340e4fbf.0000000000000009:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.033531679999999, 2024-09-17T07:30:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0049339900000000001, 2024-09-17T07:30:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115905 2024-09-17T07:30:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:36.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:36.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.299 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.299 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.299 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:36.299 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0699999999999998e-07 2024-09-17T07:30:36.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191766+0000", 2024-09-17T07:30:36.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.928e-06 2024-09-17T07:30:36.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.300 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191773+0000", 2024-09-17T07:30:36.300 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2749999999999998e-06 2024-09-17T07:30:36.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.300 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.300 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191778+0000", 2024-09-17T07:30:36.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7849999999999999e-06 2024-09-17T07:30:36.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192756+0000", 2024-09-17T07:30:36.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097834399999999992 2024-09-17T07:30:36.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.301 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.301 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192868+0000", 2024-09-17T07:30:36.301 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111575 2024-09-17T07:30:36.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.301 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:30:36.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192970+0000", 2024-09-17T07:30:36.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010192300000000001 2024-09-17T07:30:36.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:36.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193865+0000", 2024-09-17T07:30:36.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089556100000000001 2024-09-17T07:30:36.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.302 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:36.302 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196583+0000", 2024-09-17T07:30:36.302 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027174180000000001 2024-09-17T07:30:36.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.302 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:36.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196648+0000", 2024-09-17T07:30:36.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4743e-05 2024-09-17T07:30:36.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.303 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.303 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.303 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196698+0000", 2024-09-17T07:30:36.303 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0031000000000003e-05 2024-09-17T07:30:36.303 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.303 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.303 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.304 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(osd.1.0:2842 3.19 e117/108 3:9ae4605d:::rbd_data.1183c869c667.0000000000000038:23 v 117'403, mlcod=103'400)", 2024-09-17T07:30:36.304 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:26.247548+0000", 2024-09-17T07:30:36.304 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 9.9777473699999994, 2024-09-17T07:30:36.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022590760000000001, 2024-09-17T07:30:36.304 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:36.304 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.304 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.304 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.304 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:26.247548+0000", 2024-09-17T07:30:36.304 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:26.247548+0000", 2024-09-17T07:30:36.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:26.247549+0000", 2024-09-17T07:30:36.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.003e-06 2024-09-17T07:30:36.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.305 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.305 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:26.247556+0000", 2024-09-17T07:30:36.305 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9399999999999996e-06 2024-09-17T07:30:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:26.247570+0000", 2024-09-17T07:30:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3803e-05 2024-09-17T07:30:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:26.247578+0000", 2024-09-17T07:30:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6710000000000005e-06 2024-09-17T07:30:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.306 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:26.247633+0000", 2024-09-17T07:30:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4775999999999999e-05 2024-09-17T07:30:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:26.247651+0000", 2024-09-17T07:30:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7980999999999999e-05 2024-09-17T07:30:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:26.249761+0000", 2024-09-17T07:30:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002110102 2024-09-17T07:30:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.307 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:26.249807+0000", 2024-09-17T07:30:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5800000000000002e-05 2024-09-17T07:30:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:116410 3.2 3:4721cecc:::rbd_data.1183cb746bb8.000000000000000a:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T07:30:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:31.167349+0000", 2024-09-17T07:30:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.0579458470000001, 2024-09-17T07:30:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00284856, 2024-09-17T07:30:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 116410 2024-09-17T07:30:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167349+0000", 2024-09-17T07:30:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167349+0000", 2024-09-17T07:30:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167351+0000", 2024-09-17T07:30:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.359e-06 2024-09-17T07:30:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167354+0000", 2024-09-17T07:30:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5920000000000001e-06 2024-09-17T07:30:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167375+0000", 2024-09-17T07:30:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0488000000000001e-05 2024-09-17T07:30:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167388+0000", 2024-09-17T07:30:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2618e-05 2024-09-17T07:30:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167451+0000", 2024-09-17T07:30:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3700000000000003e-05 2024-09-17T07:30:36.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167768+0000", 2024-09-17T07:30:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000316645 2024-09-17T07:30:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:30:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167882+0000", 2024-09-17T07:30:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011408000000000001 2024-09-17T07:30:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:36.312 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.168600+0000", 2024-09-17T07:30:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00071769200000000003 2024-09-17T07:30:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170093+0000", 2024-09-17T07:30:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001493767 2024-09-17T07:30:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170155+0000", 2024-09-17T07:30:36.313 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1779999999999995e-05 2024-09-17T07:30:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170198+0000", 2024-09-17T07:30:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2839000000000001e-05 2024-09-17T07:30:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:116419 3.2 3:447d18bc:::rbd_data.1183cb746bb8.0000000000000068:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T07:30:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:31.170268+0000", 2024-09-17T07:30:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.0550270460000002, 2024-09-17T07:30:36.314 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002372543, 2024-09-17T07:30:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 116419 2024-09-17T07:30:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.315 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170268+0000", 2024-09-17T07:30:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170268+0000", 2024-09-17T07:30:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170269+0000", 2024-09-17T07:30:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0380000000000001e-06 2024-09-17T07:30:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170274+0000", 2024-09-17T07:30:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9699999999999998e-06 2024-09-17T07:30:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170286+0000", 2024-09-17T07:30:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1902000000000001e-05 2024-09-17T07:30:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.317 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170294+0000", 2024-09-17T07:30:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7729999999999999e-06 2024-09-17T07:30:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170350+0000", 2024-09-17T07:30:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6152e-05 2024-09-17T07:30:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170422+0000", 2024-09-17T07:30:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2218999999999994e-05 2024-09-17T07:30:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.318 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:30:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170492+0000", 2024-09-17T07:30:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9406000000000006e-05 2024-09-17T07:30:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.171019+0000", 2024-09-17T07:30:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00052708900000000001 2024-09-17T07:30:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.172569+0000", 2024-09-17T07:30:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015500850000000001 2024-09-17T07:30:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.319 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.172604+0000", 2024-09-17T07:30:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5247000000000003e-05 2024-09-17T07:30:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.172641+0000", 2024-09-17T07:30:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6662000000000002e-05 2024-09-17T07:30:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.320 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:116428 3.18 3:1cb7b20e:::rbd_data.1183cb746bb8.0000000000000071:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T07:30:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:31.172786+0000", 2024-09-17T07:30:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.0525094289999997, 2024-09-17T07:30:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0022027919999999999, 2024-09-17T07:30:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 116428 2024-09-17T07:30:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.321 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.172786+0000", 2024-09-17T07:30:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.172786+0000", 2024-09-17T07:30:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.172787+0000", 2024-09-17T07:30:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2729999999999999e-06 2024-09-17T07:30:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.322 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.172792+0000", 2024-09-17T07:30:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.028e-06 2024-09-17T07:30:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.172805+0000", 2024-09-17T07:30:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2592000000000001e-05 2024-09-17T07:30:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.172813+0000", 2024-09-17T07:30:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2220000000000002e-06 2024-09-17T07:30:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.323 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.172889+0000", 2024-09-17T07:30:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5548000000000002e-05 2024-09-17T07:30:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.173036+0000", 2024-09-17T07:30:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014787399999999999 2024-09-17T07:30:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:30:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.173152+0000", 2024-09-17T07:30:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000115442 2024-09-17T07:30:36.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.174140+0000", 2024-09-17T07:30:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00098784799999999994 2024-09-17T07:30:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.174898+0000", 2024-09-17T07:30:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00075838499999999998 2024-09-17T07:30:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.174949+0000", 2024-09-17T07:30:36.325 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0927999999999998e-05 2024-09-17T07:30:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.174989+0000", 2024-09-17T07:30:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9651999999999999e-05 2024-09-17T07:30:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:116500 3.7 3:e1d03a89:::rbd_header.11832ac10dcc:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123)", 2024-09-17T07:30:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:32.148625+0000", 2024-09-17T07:30:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.0766707259999997, 2024-09-17T07:30:36.326 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025313179999999998, 2024-09-17T07:30:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 116500 2024-09-17T07:30:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148625+0000", 2024-09-17T07:30:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148625+0000", 2024-09-17T07:30:36.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148626+0000", 2024-09-17T07:30:36.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.285e-06 2024-09-17T07:30:36.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.328 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.328 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148633+0000", 2024-09-17T07:30:36.328 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5599999999999996e-06 2024-09-17T07:30:36.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148646+0000", 2024-09-17T07:30:36.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2643000000000001e-05 2024-09-17T07:30:36.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148653+0000", 2024-09-17T07:30:36.329 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3010000000000001e-06 2024-09-17T07:30:36.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.329 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.329 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.329 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148711+0000", 2024-09-17T07:30:36.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7272999999999999e-05 2024-09-17T07:30:36.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:30:36.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148715+0000", 2024-09-17T07:30:36.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2869999999999998e-06 2024-09-17T07:30:36.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.330 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.330 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.330 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.150739+0000", 2024-09-17T07:30:36.330 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020238320000000001 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.150931+0000", 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00019203199999999999 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.151156+0000", 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022510499999999999 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.331 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117007 3.8 3:11c9baa3:::rbd_object_map.118394b259f9:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-17T07:30:36.332 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:36.120261+0000", 2024-09-17T07:30:36.332 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.10503462199999999, 2024-09-17T07:30:36.332 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045697289999999998, 2024-09-17T07:30:36.332 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.332 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:36.332 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.332 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.332 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.332 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117007 2024-09-17T07:30:36.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.332 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.332 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.333 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120261+0000", 2024-09-17T07:30:36.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.333 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120261+0000", 2024-09-17T07:30:36.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.333 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.333 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.333 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120262+0000", 2024-09-17T07:30:36.333 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.77e-06 2024-09-17T07:30:36.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120270+0000", 2024-09-17T07:30:36.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0080000000000002e-06 2024-09-17T07:30:36.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120287+0000", 2024-09-17T07:30:36.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6847e-05 2024-09-17T07:30:36.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.334 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.334 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120297+0000", 2024-09-17T07:30:36.334 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8830000000000001e-06 2024-09-17T07:30:36.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120368+0000", 2024-09-17T07:30:36.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0689e-05 2024-09-17T07:30:36.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:30:36.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120375+0000", 2024-09-17T07:30:36.335 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.187e-06 2024-09-17T07:30:36.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.335 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.335 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.335 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.122574+0000", 2024-09-17T07:30:36.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021992980000000001 2024-09-17T07:30:36.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.122670+0000", 2024-09-17T07:30:36.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5830999999999998e-05 2024-09-17T07:30:36.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.336 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:30:36.336 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.122866+0000", 2024-09-17T07:30:36.336 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000195303 2024-09-17T07:30:36.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.336 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:36.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124084+0000", 2024-09-17T07:30:36.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001218076 2024-09-17T07:30:36.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:36.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124153+0000", 2024-09-17T07:30:36.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9839999999999995e-05 2024-09-17T07:30:36.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.337 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.337 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:36.337 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124784+0000", 2024-09-17T07:30:36.337 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063082299999999997 2024-09-17T07:30:36.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.338 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.338 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124830+0000", 2024-09-17T07:30:36.338 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6174000000000002e-05 2024-09-17T07:30:36.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.338 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.338 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.338 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:36.340 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:36.339+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:36.340 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:36.440 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:36.440+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:36.488 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:36.488 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:36.489 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:36.489 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:36.497 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:36.541 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:36.541+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:36.604 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:36.604 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:36.605 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:36.605 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:36.613 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:36.642 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:36.642+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:36.721 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:36.721 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 98.587793711000003, 2024-09-17T07:30:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:30:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.722 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:30:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:30:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:30:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:30:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:30:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:30:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:30:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:30:36.725 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:30:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:30:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:30:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:30:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:30:36.726 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:30:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:30:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:30:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:30:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:30:36.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:30:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:30:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:30:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:30:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.728 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:30:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:30:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.281827007000004, 2024-09-17T07:30:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:30:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:36.729 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:30:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:36.730 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:30:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:30:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:30:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:30:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:30:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:30:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:30:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:30:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:30:36.732 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:30:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:30:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:30:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:30:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:30:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.733 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.281804891999997, 2024-09-17T07:30:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:30:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:30:36.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:30:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:30:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:30:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:30:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:30:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.736 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:30:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:30:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:30:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:30:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:30:36.737 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:30:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.281780788000006, 2024-09-17T07:30:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:30:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.738 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:30:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:30:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:30:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:30:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.740 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:30:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:30:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:30:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:30:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:30:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:30:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.741 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:30:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:30:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.281756698999999, 2024-09-17T07:30:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:30:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:30:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:30:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:30:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:30:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:30:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:30:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:30:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:30:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:30:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:30:36.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:30:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:30:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.28173228, 2024-09-17T07:30:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:30:36.746 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:30:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:30:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:30:36.748 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:30:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:30:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:30:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:30:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.749 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:30:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:30:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:30:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:30:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:36.750 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.281710579999995, 2024-09-17T07:30:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:30:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:30:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:36.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:30:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:30:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:30:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:30:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:30:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:30:36.753 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:30:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:30:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:30:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:30:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:30:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.281665848000003, 2024-09-17T07:30:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:30:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:30:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.755 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:30:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:30:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:30:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:30:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:30:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:30:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.757 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:30:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:30:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:30:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:30:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:30:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:30:36.758 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.281637382, 2024-09-17T07:30:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:30:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.759 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:30:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:30:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:30:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:30:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:30:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:30:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:30:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:30:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:30:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:30:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.762 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:30:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:30:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.281520130000004, 2024-09-17T07:30:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:30:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.763 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:30:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:30:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:30:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:30:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:30:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:30:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:30:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:30:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:30:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:30:36.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:30:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:30:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:30:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.281094027999998, 2024-09-17T07:30:36.767 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:30:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:30:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:30:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:30:36.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:30:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:30:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:30:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:30:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:30:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.770 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:30:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:30:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:30:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:30:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:36.771 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.281074356999994, 2024-09-17T07:30:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:30:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:30:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:36.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:30:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:30:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:30:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:30:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:30:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:30:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:30:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:30:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:30:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:30:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:30:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.775 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.281051493000007, 2024-09-17T07:30:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:30:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:30:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.776 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:30:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:30:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:30:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:30:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:30:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:30:36.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:30:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:30:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:30:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:30:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:30:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.281030192000003, 2024-09-17T07:30:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:30:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:30:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.780 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:30:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:30:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:30:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:30:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:30:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:30:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:30:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:30:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:30:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:30:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:30:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:30:36.783 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.280982404, 2024-09-17T07:30:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:30:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.784 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:30:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:30:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:30:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:30:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:30:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:30:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:30:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:30:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:30:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:30:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:30:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.787 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:30:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:30:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.280955182, 2024-09-17T07:30:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:30:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.788 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:30:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:30:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:30:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:30:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:30:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:30:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:30:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:30:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:30:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:30:36.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:30:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:30:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.280934794999993, 2024-09-17T07:30:36.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:30:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:30:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:30:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:30:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:30:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:30:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:30:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:30:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:30:36.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:30:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:30:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:30:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:30:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:30:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.796 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.280915089999993, 2024-09-17T07:30:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:30:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.797 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:30:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:30:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:30:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:30:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:30:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:30:36.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:30:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:30:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:30:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:30:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:30:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:30:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:30:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.280892011999995, 2024-09-17T07:30:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:30:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.801 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:30:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:30:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:30:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:30:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:30:36.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:30:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:30:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:30:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:30:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:30:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:30:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:30:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:30:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:36.805 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 88.280871450000006, 2024-09-17T07:30:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:30:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:30:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:36.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:30:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:30:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:30:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:30:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:30:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:30:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:30:36.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:30:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:30:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:30:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:30:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.809 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:36.810 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:36.810 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:36.811 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:36.812 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:36.811+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:36.912 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:36.912+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:36.959 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:36.959 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:36.959 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:36.959 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:36.968 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:37.013 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:37.013+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:37.047 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:30:37.081 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-17T07:30:37.127 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:37.127 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:37.127 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:37.127 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:37.127 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:37.127 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:37.126+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:37.139 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:37.194 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:37.194 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:37.212 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:37.212 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:37.214 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:37.214+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:37.229 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:37.246 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:37.246 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:37.263 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:37.263 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:37.281 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:37.281 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:37.290 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:37.298 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:37.298 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:37.299 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:37.314 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:37.315 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:37.315 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:37.314+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:37.416 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:37.415+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:37.422 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:37.422 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:37.431 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:37.517 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:37.516+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:37.537 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:37.538 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:37.546 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:37.617 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:37.617+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:37.654 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:37.654 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:37.662 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:37.718 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:37.718+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:37.773 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:37.774 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:37.782 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:37.824 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:37.823+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:37.920 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:37.919+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:37.926 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:37.926 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:37.935 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:38.021 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:38.020+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:38.039 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:38.039 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:38.047 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:38.125 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 5, 6, 7, 3, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-17T07:30:38.125 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:30:38.125 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-17T07:30:38.126 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-17T07:30:38.132 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:38.131+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:38.204 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:38.205 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:38.214 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:38.257 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:38.256+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:38.324 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:38.323+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:38.358 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:38.358 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:38.367 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:38.424 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:38.424+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:38.470 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:38.470 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:38.479 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:38.525 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:38.525+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:38.582 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:38.582 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:38.590 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:38.631 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:38.631+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:38.727 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:38.727+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:38.733 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:38.733 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:38.742 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:38.828 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:38.828+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:38.885 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:38.886 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:38.894 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:38.936 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:38.935+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:39.030 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:39.030+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:39.039 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:39.039 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:39.048 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:39.099 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T07:30:39.129 INFO:teuthology.orchestra.run.smithi045.stdout:moved 3 / 82 (3.65854%) 2024-09-17T07:30:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:avg 10.25 2024-09-17T07:30:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:stddev 2.77263 -> 2.94746 (expected baseline 2.99479) 2024-09-17T07:30:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:min osd.6 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-17T07:30:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:max osd.7 with 14 -> 13 pgs (1.36585 -> 1.26829 * mean) 2024-09-17T07:30:39.130 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:30:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:oload 1000 2024-09-17T07:30:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:max_change 3 2024-09-17T07:30:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:max_change_osds 4 2024-09-17T07:30:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:average_utilization 0.0006 2024-09-17T07:30:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:overload_utilization 0.0063 2024-09-17T07:30:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 weight 0.8455 -> 1.0000 2024-09-17T07:30:39.132 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:39.131+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:39.153 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T07:30:39.153 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-17T07:30:39.154 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:39.154 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:39.165 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:39.232 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:39.231+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:39.273 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:39.273 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:39.282 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:39.327 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:30:39.371 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:39.371+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:39.396 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:39.396 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:39.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:39.446 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:39.446+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:39.455 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:39.455 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:39.473 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:39.474 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:39.507 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:39.507 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:39.533 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:39.534 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:39.534 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:39.533+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:39.550 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:39.626 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:39.627 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:39.627 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:39.628 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:39.628 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:39.628 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:39.628 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:39.628 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:39.629 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:39.629 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:39.630 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:39.630 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:39.631 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:39.635 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:39.635+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:39.736 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:39.735+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:39.777 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:39.777 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:39.777 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:39.778 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:39.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.778 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.15 107/106 [PushOp(3:ae97839c:::rbd_data.11839a97b753.0000000000000004:head, version: 55'111, data_included: [0~16384,81920~16384,163840~16384,196608~16384,245760~16384,278528~16384,360448~16384,442368~16384,524288~16384,704512~16384,786432~16384,868352~16384,950272~16384,983040~16384,1032192~16384,1064960~16384,1146880~16384,1228800~16384,1310720~16384,1343488~16384,1392640~16384,1425408~32768,1523712~16384], data_size: 393216, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:ae97839c:::rbd_data.11839a97b753.0000000000000004:head@55'111, size: 1540096, copy_subset: [(0, 1540096)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1540096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:39.778 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:16.884937+0000", 2024-09-17T07:30:39.778 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.890669227, 2024-09-17T07:30:39.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025497419999999998, 2024-09-17T07:30:39.778 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.778 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:39.778 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.778 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.778 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.884937+0000", 2024-09-17T07:30:39.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.884937+0000", 2024-09-17T07:30:39.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.884938+0000", 2024-09-17T07:30:39.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5499999999999998e-07 2024-09-17T07:30:39.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885283+0000", 2024-09-17T07:30:39.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00034585300000000002 2024-09-17T07:30:39.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885301+0000", 2024-09-17T07:30:39.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7207999999999999e-05 2024-09-17T07:30:39.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885316+0000", 2024-09-17T07:30:39.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4873e-05 2024-09-17T07:30:39.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885416+0000", 2024-09-17T07:30:39.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000100456 2024-09-17T07:30:39.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.885432+0000", 2024-09-17T07:30:39.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6390000000000001e-05 2024-09-17T07:30:39.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:16.887487+0000", 2024-09-17T07:30:39.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020545070000000001 2024-09-17T07:30:39.782 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.782 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.782 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.782 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9, version: 36'125, data_included: [5478708~26316,5767168~524288,6553600~497972], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9@36'125, size: 10246384, copy_subset: [(0, 10246384)], clone_subset: {}, snapset: 9=[]:{9: [9]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 7051572, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5478708, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-17T07:30:39.782 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:39.782 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.586201733999999, 2024-09-17T07:30:39.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024847990000000002, 2024-09-17T07:30:39.782 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.782 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:39.783 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:39.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:39.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.189405+0000", 2024-09-17T07:30:39.784 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:30:39.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.784 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.784 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191002+0000", 2024-09-17T07:30:39.784 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015967100000000001 2024-09-17T07:30:39.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.784 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.784 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191017+0000", 2024-09-17T07:30:39.784 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4717e-05 2024-09-17T07:30:39.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191036+0000", 2024-09-17T07:30:39.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9882999999999999e-05 2024-09-17T07:30:39.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191120+0000", 2024-09-17T07:30:39.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3313999999999995e-05 2024-09-17T07:30:39.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191128+0000", 2024-09-17T07:30:39.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5499999999999995e-06 2024-09-17T07:30:39.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.191889+0000", 2024-09-17T07:30:39.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076114100000000001 2024-09-17T07:30:39.786 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.786 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.786 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.786 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head, version: 36'126, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head@36'126, size: 9431460, copy_subset: [], clone_subset: {3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9: [(0, 9431460)]}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:39.786 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:39.786 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.571816914999999, 2024-09-17T07:30:39.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002812681, 2024-09-17T07:30:39.787 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.787 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.787 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:39.787 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.787 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.787 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:39.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.787 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.787 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:39.787 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.788 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.788 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:39.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9100000000000004e-07 2024-09-17T07:30:39.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.788 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.788 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203801+0000", 2024-09-17T07:30:39.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0696000000000001e-05 2024-09-17T07:30:39.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.788 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.788 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.788 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203817+0000", 2024-09-17T07:30:39.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6480000000000001e-05 2024-09-17T07:30:39.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203829+0000", 2024-09-17T07:30:39.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1935e-05 2024-09-17T07:30:39.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203891+0000", 2024-09-17T07:30:39.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2079e-05 2024-09-17T07:30:39.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203900+0000", 2024-09-17T07:30:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4999999999999999e-06 2024-09-17T07:30:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.206602+0000", 2024-09-17T07:30:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027024000000000002 2024-09-17T07:30:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head, version: 82'426, data_included: [0~786432,981325~41682,1966080~6721], data_size: 834835, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head@82'426, size: 3276800, copy_subset: [(0, 3276800)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3276800, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:39.790 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.308228+0000", 2024-09-17T07:30:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.467377927000001, 2024-09-17T07:30:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025329110000000001, 2024-09-17T07:30:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.308228+0000", 2024-09-17T07:30:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.308228+0000", 2024-09-17T07:30:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.308229+0000", 2024-09-17T07:30:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:30:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309432+0000", 2024-09-17T07:30:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001202704 2024-09-17T07:30:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309447+0000", 2024-09-17T07:30:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5184000000000001e-05 2024-09-17T07:30:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309457+0000", 2024-09-17T07:30:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9129999999999998e-06 2024-09-17T07:30:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309544+0000", 2024-09-17T07:30:39.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7249000000000004e-05 2024-09-17T07:30:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.309558+0000", 2024-09-17T07:30:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3781000000000001e-05 2024-09-17T07:30:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.310761+0000", 2024-09-17T07:30:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012035839999999999 2024-09-17T07:30:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.c 108/106 [PushOp(3:3034d26c:::rbd_data.11835a3b5a27.0000000000000039:1c, version: 66'255, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:3034d26c:::rbd_data.11835a3b5a27.0000000000000039:1c@66'255, size: 2097152, copy_subset: [(0, 2097152)], clone_subset: {}, snapset: 1c=[]:{1c: [1c]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.781537+0000", 2024-09-17T07:30:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 21.994068930000001, 2024-09-17T07:30:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0026008479999999998, 2024-09-17T07:30:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.781537+0000", 2024-09-17T07:30:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.781537+0000", 2024-09-17T07:30:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.781538+0000", 2024-09-17T07:30:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0399999999999996e-07 2024-09-17T07:30:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.783137+0000", 2024-09-17T07:30:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0015992739999999999 2024-09-17T07:30:39.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.783153+0000", 2024-09-17T07:30:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5608e-05 2024-09-17T07:30:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.783163+0000", 2024-09-17T07:30:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0028e-05 2024-09-17T07:30:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.783247+0000", 2024-09-17T07:30:39.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4099999999999998e-05 2024-09-17T07:30:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.783261+0000", 2024-09-17T07:30:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4191e-05 2024-09-17T07:30:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.784138+0000", 2024-09-17T07:30:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087714300000000004 2024-09-17T07:30:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.798 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.c 108/106 [PushOp(3:34edcbef:::rbd_data.11831719e038.0000000000000000:head, version: 68'273, data_included: [16384~81920,180224~81920,344064~92788,507904~81920,595536~49296,671744~81920,835584~81920,999424~81920,2605548~8985,2708290~8096,2995581~42933], data_size: 693618, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:34edcbef:::rbd_data.11831719e038.0000000000000000:head@68'273, size: 3038514, copy_subset: [(0, 3038514)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3038514, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.799290+0000", 2024-09-17T07:30:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 21.976316566000001, 2024-09-17T07:30:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025209730000000001, 2024-09-17T07:30:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799290+0000", 2024-09-17T07:30:39.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799290+0000", 2024-09-17T07:30:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799290+0000", 2024-09-17T07:30:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:30:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799886+0000", 2024-09-17T07:30:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00059524899999999995 2024-09-17T07:30:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799903+0000", 2024-09-17T07:30:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7689999999999998e-05 2024-09-17T07:30:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.799917+0000", 2024-09-17T07:30:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3189e-05 2024-09-17T07:30:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.800010+0000", 2024-09-17T07:30:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3690999999999995e-05 2024-09-17T07:30:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.800025+0000", 2024-09-17T07:30:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4453999999999999e-05 2024-09-17T07:30:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.801811+0000", 2024-09-17T07:30:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001786096 2024-09-17T07:30:39.802 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.803 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.803 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.803 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.18 109/106 [PushOp(3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e, version: 44'134, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e@44'134, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:39.803 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:18.083461+0000", 2024-09-17T07:30:39.803 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 21.692145440000001, 2024-09-17T07:30:39.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003057976, 2024-09-17T07:30:39.803 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.803 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.803 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:39.803 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083461+0000", 2024-09-17T07:30:39.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083461+0000", 2024-09-17T07:30:39.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083462+0000", 2024-09-17T07:30:39.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1500000000000004e-07 2024-09-17T07:30:39.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083603+0000", 2024-09-17T07:30:39.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140863 2024-09-17T07:30:39.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083623+0000", 2024-09-17T07:30:39.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.003e-05 2024-09-17T07:30:39.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083642+0000", 2024-09-17T07:30:39.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9734000000000001e-05 2024-09-17T07:30:39.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.084564+0000", 2024-09-17T07:30:39.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00092196200000000002 2024-09-17T07:30:39.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.084580+0000", 2024-09-17T07:30:39.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5464e-05 2024-09-17T07:30:39.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.086519+0000", 2024-09-17T07:30:39.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001939208 2024-09-17T07:30:39.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.807 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.807 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115899 3.b 3:d406e0ff:::rbd_data.118340e4fbf.0000000000000001:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:39.807 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:39.807 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.584671768, 2024-09-17T07:30:39.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0040641219999999999, 2024-09-17T07:30:39.807 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:39.807 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:39.807 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:39.808 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:39.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115899 2024-09-17T07:30:39.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.808 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:39.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:39.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190936+0000", 2024-09-17T07:30:39.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.26e-06 2024-09-17T07:30:39.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190941+0000", 2024-09-17T07:30:39.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0499999999999999e-06 2024-09-17T07:30:39.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190958+0000", 2024-09-17T07:30:39.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6548000000000001e-05 2024-09-17T07:30:39.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190966+0000", 2024-09-17T07:30:39.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6799999999999999e-06 2024-09-17T07:30:39.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191028+0000", 2024-09-17T07:30:39.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2082999999999995e-05 2024-09-17T07:30:39.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191161+0000", 2024-09-17T07:30:39.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000132461 2024-09-17T07:30:39.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:39.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191284+0000", 2024-09-17T07:30:39.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012301200000000001 2024-09-17T07:30:39.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:39.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192371+0000", 2024-09-17T07:30:39.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00108709 2024-09-17T07:30:39.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:39.812 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194898+0000", 2024-09-17T07:30:39.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025272710000000002 2024-09-17T07:30:39.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:39.812 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194957+0000", 2024-09-17T07:30:39.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8792000000000002e-05 2024-09-17T07:30:39.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.812 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194999+0000", 2024-09-17T07:30:39.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1875000000000001e-05 2024-09-17T07:30:39.813 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.813 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.813 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.813 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115901 3.1e 3:7f62d392:::rbd_data.118340e4fbf.0000000000000003:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:39.813 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.191679+0000", 2024-09-17T07:30:39.813 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.583927423999999, 2024-09-17T07:30:39.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024544079999999999, 2024-09-17T07:30:39.813 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:39.813 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:39.814 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:39.814 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:39.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115901 2024-09-17T07:30:39.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.814 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191679+0000", 2024-09-17T07:30:39.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191679+0000", 2024-09-17T07:30:39.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191680+0000", 2024-09-17T07:30:39.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6499999999999998e-07 2024-09-17T07:30:39.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191684+0000", 2024-09-17T07:30:39.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1479999999999999e-06 2024-09-17T07:30:39.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191697+0000", 2024-09-17T07:30:39.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3086e-05 2024-09-17T07:30:39.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191714+0000", 2024-09-17T07:30:39.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7147999999999999e-05 2024-09-17T07:30:39.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192295+0000", 2024-09-17T07:30:39.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00058122000000000002 2024-09-17T07:30:39.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192413+0000", 2024-09-17T07:30:39.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000117778 2024-09-17T07:30:39.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-17T07:30:39.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192515+0000", 2024-09-17T07:30:39.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010200900000000001 2024-09-17T07:30:39.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:39.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193831+0000", 2024-09-17T07:30:39.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001315472 2024-09-17T07:30:39.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:39.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194034+0000", 2024-09-17T07:30:39.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00020312600000000001 2024-09-17T07:30:39.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:39.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194083+0000", 2024-09-17T07:30:39.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9132999999999999e-05 2024-09-17T07:30:39.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194133+0000", 2024-09-17T07:30:39.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0423000000000002e-05 2024-09-17T07:30:39.819 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.819 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.819 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.819 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115903 3.18 3:1fc9b9ef:::rbd_data.118340e4fbf.0000000000000005:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:39.819 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:39.819 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.583873645000001, 2024-09-17T07:30:39.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00477466, 2024-09-17T07:30:39.819 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.819 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:39.819 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:39.820 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:39.820 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:39.820 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115903 2024-09-17T07:30:39.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.820 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:39.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:39.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:39.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9800000000000004e-07 2024-09-17T07:30:39.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191736+0000", 2024-09-17T07:30:39.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6620000000000001e-06 2024-09-17T07:30:39.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191743+0000", 2024-09-17T07:30:39.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6939999999999998e-06 2024-09-17T07:30:39.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191748+0000", 2024-09-17T07:30:39.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9749999999999998e-06 2024-09-17T07:30:39.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191823+0000", 2024-09-17T07:30:39.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4954000000000001e-05 2024-09-17T07:30:39.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191933+0000", 2024-09-17T07:30:39.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000110708 2024-09-17T07:30:39.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:30:39.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192031+0000", 2024-09-17T07:30:39.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7974999999999996e-05 2024-09-17T07:30:39.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:39.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193329+0000", 2024-09-17T07:30:39.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012976419999999999 2024-09-17T07:30:39.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:39.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196404+0000", 2024-09-17T07:30:39.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003075174 2024-09-17T07:30:39.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:39.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196461+0000", 2024-09-17T07:30:39.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6663999999999998e-05 2024-09-17T07:30:39.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196507+0000", 2024-09-17T07:30:39.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6714e-05 2024-09-17T07:30:39.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.825 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.825 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115905 3.18 3:19f08345:::rbd_data.118340e4fbf.0000000000000009:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:39.825 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:39.825 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.583842775000001, 2024-09-17T07:30:39.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0049339900000000001, 2024-09-17T07:30:39.826 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.826 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.826 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:39.826 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:39.826 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:39.826 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:39.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115905 2024-09-17T07:30:39.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.826 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:39.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:39.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:39.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0699999999999998e-07 2024-09-17T07:30:39.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191766+0000", 2024-09-17T07:30:39.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.928e-06 2024-09-17T07:30:39.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191773+0000", 2024-09-17T07:30:39.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2749999999999998e-06 2024-09-17T07:30:39.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191778+0000", 2024-09-17T07:30:39.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7849999999999999e-06 2024-09-17T07:30:39.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192756+0000", 2024-09-17T07:30:39.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097834399999999992 2024-09-17T07:30:39.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192868+0000", 2024-09-17T07:30:39.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111575 2024-09-17T07:30:39.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:30:39.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192970+0000", 2024-09-17T07:30:39.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010192300000000001 2024-09-17T07:30:39.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:39.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193865+0000", 2024-09-17T07:30:39.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089556100000000001 2024-09-17T07:30:39.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:39.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196583+0000", 2024-09-17T07:30:39.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027174180000000001 2024-09-17T07:30:39.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:39.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196648+0000", 2024-09-17T07:30:39.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4743e-05 2024-09-17T07:30:39.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196698+0000", 2024-09-17T07:30:39.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0031000000000003e-05 2024-09-17T07:30:39.831 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.831 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.831 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.831 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:116410 3.2 3:4721cecc:::rbd_data.1183cb746bb8.000000000000000a:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T07:30:39.831 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:31.167349+0000", 2024-09-17T07:30:39.831 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 8.6082569420000006, 2024-09-17T07:30:39.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00284856, 2024-09-17T07:30:39.831 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:39.832 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:39.832 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:39.832 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:39.832 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 116410 2024-09-17T07:30:39.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.832 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167349+0000", 2024-09-17T07:30:39.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167349+0000", 2024-09-17T07:30:39.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167351+0000", 2024-09-17T07:30:39.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.359e-06 2024-09-17T07:30:39.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167354+0000", 2024-09-17T07:30:39.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5920000000000001e-06 2024-09-17T07:30:39.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167375+0000", 2024-09-17T07:30:39.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0488000000000001e-05 2024-09-17T07:30:39.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167388+0000", 2024-09-17T07:30:39.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2618e-05 2024-09-17T07:30:39.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167451+0000", 2024-09-17T07:30:39.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3700000000000003e-05 2024-09-17T07:30:39.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167768+0000", 2024-09-17T07:30:39.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000316645 2024-09-17T07:30:39.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:30:39.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167882+0000", 2024-09-17T07:30:39.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011408000000000001 2024-09-17T07:30:39.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:39.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.168600+0000", 2024-09-17T07:30:39.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00071769200000000003 2024-09-17T07:30:39.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:39.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170093+0000", 2024-09-17T07:30:39.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001493767 2024-09-17T07:30:39.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:39.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170155+0000", 2024-09-17T07:30:39.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1779999999999995e-05 2024-09-17T07:30:39.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170198+0000", 2024-09-17T07:30:39.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2839000000000001e-05 2024-09-17T07:30:39.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.836 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.837 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:116500 3.7 3:e1d03a89:::rbd_header.11832ac10dcc:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123)", 2024-09-17T07:30:39.837 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:32.148625+0000", 2024-09-17T07:30:39.837 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 7.6269818210000002, 2024-09-17T07:30:39.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025313179999999998, 2024-09-17T07:30:39.837 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:39.837 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:39.837 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:39.837 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:39.837 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 116500 2024-09-17T07:30:39.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.838 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148625+0000", 2024-09-17T07:30:39.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148625+0000", 2024-09-17T07:30:39.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148626+0000", 2024-09-17T07:30:39.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.285e-06 2024-09-17T07:30:39.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148633+0000", 2024-09-17T07:30:39.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5599999999999996e-06 2024-09-17T07:30:39.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148646+0000", 2024-09-17T07:30:39.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2643000000000001e-05 2024-09-17T07:30:39.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148653+0000", 2024-09-17T07:30:39.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3010000000000001e-06 2024-09-17T07:30:39.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148711+0000", 2024-09-17T07:30:39.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7272999999999999e-05 2024-09-17T07:30:39.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:30:39.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.148715+0000", 2024-09-17T07:30:39.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2869999999999998e-06 2024-09-17T07:30:39.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.150739+0000", 2024-09-17T07:30:39.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020238320000000001 2024-09-17T07:30:39.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.150931+0000", 2024-09-17T07:30:39.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00019203199999999999 2024-09-17T07:30:39.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:32.151156+0000", 2024-09-17T07:30:39.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022510499999999999 2024-09-17T07:30:39.841 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.842 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.842 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.842 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117007 3.8 3:11c9baa3:::rbd_object_map.118394b259f9:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-17T07:30:39.842 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:36.120261+0000", 2024-09-17T07:30:39.842 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.6553457169999999, 2024-09-17T07:30:39.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045697289999999998, 2024-09-17T07:30:39.842 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.842 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:39.842 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:39.842 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:39.842 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:39.842 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117007 2024-09-17T07:30:39.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.843 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120261+0000", 2024-09-17T07:30:39.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120261+0000", 2024-09-17T07:30:39.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120262+0000", 2024-09-17T07:30:39.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.77e-06 2024-09-17T07:30:39.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120270+0000", 2024-09-17T07:30:39.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0080000000000002e-06 2024-09-17T07:30:39.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120287+0000", 2024-09-17T07:30:39.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6847e-05 2024-09-17T07:30:39.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120297+0000", 2024-09-17T07:30:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8830000000000001e-06 2024-09-17T07:30:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120368+0000", 2024-09-17T07:30:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0689e-05 2024-09-17T07:30:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:30:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120375+0000", 2024-09-17T07:30:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.187e-06 2024-09-17T07:30:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.122574+0000", 2024-09-17T07:30:39.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021992980000000001 2024-09-17T07:30:39.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.122670+0000", 2024-09-17T07:30:39.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5830999999999998e-05 2024-09-17T07:30:39.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:30:39.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.122866+0000", 2024-09-17T07:30:39.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000195303 2024-09-17T07:30:39.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124084+0000", 2024-09-17T07:30:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001218076 2024-09-17T07:30:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124153+0000", 2024-09-17T07:30:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9839999999999995e-05 2024-09-17T07:30:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124784+0000", 2024-09-17T07:30:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063082299999999997 2024-09-17T07:30:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124830+0000", 2024-09-17T07:30:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6174000000000002e-05 2024-09-17T07:30:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117147 3.7 3:e1d03a89:::rbd_header.11832ac10dcc:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T07:30:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.085788+0000", 2024-09-17T07:30:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.6898182410000002, 2024-09-17T07:30:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0032128040000000001, 2024-09-17T07:30:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.849 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:39.849 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:39.849 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:39.849 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:39.849 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117147 2024-09-17T07:30:39.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.849 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085788+0000", 2024-09-17T07:30:39.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085788+0000", 2024-09-17T07:30:39.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085790+0000", 2024-09-17T07:30:39.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1179999999999999e-06 2024-09-17T07:30:39.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085798+0000", 2024-09-17T07:30:39.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1300000000000001e-06 2024-09-17T07:30:39.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085812+0000", 2024-09-17T07:30:39.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4065e-05 2024-09-17T07:30:39.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085824+0000", 2024-09-17T07:30:39.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1307e-05 2024-09-17T07:30:39.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085883+0000", 2024-09-17T07:30:39.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9141999999999997e-05 2024-09-17T07:30:39.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085985+0000", 2024-09-17T07:30:39.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000101895 2024-09-17T07:30:39.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:39.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.086372+0000", 2024-09-17T07:30:39.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00038743600000000001 2024-09-17T07:30:39.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:39.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.087110+0000", 2024-09-17T07:30:39.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000737887 2024-09-17T07:30:39.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:39.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.088882+0000", 2024-09-17T07:30:39.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017721600000000001 2024-09-17T07:30:39.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:39.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.088947+0000", 2024-09-17T07:30:39.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4719000000000002e-05 2024-09-17T07:30:39.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.089001+0000", 2024-09-17T07:30:39.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3944999999999999e-05 2024-09-17T07:30:39.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.854 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.854 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.854 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117151 3.7 3:e1d03a89:::rbd_header.11832ac10dcc:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T07:30:39.854 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.097104+0000", 2024-09-17T07:30:39.854 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.6785020610000001, 2024-09-17T07:30:39.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062858530000000001, 2024-09-17T07:30:39.854 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:39.854 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:39.854 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:39.854 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:39.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117151 2024-09-17T07:30:39.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.855 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097104+0000", 2024-09-17T07:30:39.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097104+0000", 2024-09-17T07:30:39.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097107+0000", 2024-09-17T07:30:39.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7429999999999998e-06 2024-09-17T07:30:39.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097121+0000", 2024-09-17T07:30:39.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3774000000000001e-05 2024-09-17T07:30:39.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097150+0000", 2024-09-17T07:30:39.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8762e-05 2024-09-17T07:30:39.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097163+0000", 2024-09-17T07:30:39.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3631e-05 2024-09-17T07:30:39.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097245+0000", 2024-09-17T07:30:39.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1874999999999998e-05 2024-09-17T07:30:39.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097337+0000", 2024-09-17T07:30:39.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1465000000000006e-05 2024-09-17T07:30:39.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:39.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097553+0000", 2024-09-17T07:30:39.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021630999999999999 2024-09-17T07:30:39.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:39.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.098320+0000", 2024-09-17T07:30:39.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076674000000000002 2024-09-17T07:30:39.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:39.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.103265+0000", 2024-09-17T07:30:39.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0049452250000000001 2024-09-17T07:30:39.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:39.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.103335+0000", 2024-09-17T07:30:39.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0463999999999995e-05 2024-09-17T07:30:39.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.103390+0000", 2024-09-17T07:30:39.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4864000000000002e-05 2024-09-17T07:30:39.859 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.859 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.859 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.860 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117237 3.2 3:40c8b221:::rbd_data.11832ac10dcc.0000000000000000:head [delete] snapc 1d=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T07:30:39.860 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.169608+0000", 2024-09-17T07:30:39.860 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.605998064, 2024-09-17T07:30:39.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003512509, 2024-09-17T07:30:39.860 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.860 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.860 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:39.860 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:39.860 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:39.860 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:39.860 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117237 2024-09-17T07:30:39.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.860 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169608+0000", 2024-09-17T07:30:39.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169608+0000", 2024-09-17T07:30:39.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169610+0000", 2024-09-17T07:30:39.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8020000000000001e-06 2024-09-17T07:30:39.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169618+0000", 2024-09-17T07:30:39.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0080000000000002e-06 2024-09-17T07:30:39.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169635+0000", 2024-09-17T07:30:39.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6611000000000001e-05 2024-09-17T07:30:39.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169643+0000", 2024-09-17T07:30:39.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6549999999999993e-06 2024-09-17T07:30:39.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169723+0000", 2024-09-17T07:30:39.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0034e-05 2024-09-17T07:30:39.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169860+0000", 2024-09-17T07:30:39.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013645899999999999 2024-09-17T07:30:39.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:30:39.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169970+0000", 2024-09-17T07:30:39.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000109957 2024-09-17T07:30:39.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:39.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.171850+0000", 2024-09-17T07:30:39.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001879868 2024-09-17T07:30:39.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:39.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172391+0000", 2024-09-17T07:30:39.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054084100000000002 2024-09-17T07:30:39.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:39.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172439+0000", 2024-09-17T07:30:39.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7916e-05 2024-09-17T07:30:39.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173121+0000", 2024-09-17T07:30:39.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00068235799999999999 2024-09-17T07:30:39.866 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.866 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.866 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.866 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117245 3.1e 3:7b680344:::rbd_data.11832ac10dcc.0000000000000008:head [delete] snapc 1d=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T07:30:39.866 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.169745+0000", 2024-09-17T07:30:39.866 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.6058617850000001, 2024-09-17T07:30:39.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002854868, 2024-09-17T07:30:39.866 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.867 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.867 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:39.867 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:39.867 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:39.867 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:39.867 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117245 2024-09-17T07:30:39.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.867 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169745+0000", 2024-09-17T07:30:39.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169745+0000", 2024-09-17T07:30:39.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169745+0000", 2024-09-17T07:30:39.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7199999999999999e-07 2024-09-17T07:30:39.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169748+0000", 2024-09-17T07:30:39.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5330000000000001e-06 2024-09-17T07:30:39.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169757+0000", 2024-09-17T07:30:39.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6260000000000007e-06 2024-09-17T07:30:39.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169763+0000", 2024-09-17T07:30:39.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0780000000000001e-06 2024-09-17T07:30:39.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.170431+0000", 2024-09-17T07:30:39.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00066795099999999996 2024-09-17T07:30:39.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.170528+0000", 2024-09-17T07:30:39.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6408000000000003e-05 2024-09-17T07:30:39.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-17T07:30:39.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.170685+0000", 2024-09-17T07:30:39.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015698 2024-09-17T07:30:39.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:39.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.171771+0000", 2024-09-17T07:30:39.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010864970000000001 2024-09-17T07:30:39.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:39.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172509+0000", 2024-09-17T07:30:39.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073733399999999997 2024-09-17T07:30:39.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:39.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172558+0000", 2024-09-17T07:30:39.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9249000000000002e-05 2024-09-17T07:30:39.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172600+0000", 2024-09-17T07:30:39.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1640000000000001e-05 2024-09-17T07:30:39.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.872 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.872 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117247 3.b 3:d46c2c29:::rbd_data.11832ac10dcc.0000000000000072:head [delete] snapc 1d=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T07:30:39.872 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.172113+0000", 2024-09-17T07:30:39.872 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.603493099, 2024-09-17T07:30:39.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002623972, 2024-09-17T07:30:39.872 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:39.873 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:39.873 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:39.873 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:39.873 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117247 2024-09-17T07:30:39.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.873 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172113+0000", 2024-09-17T07:30:39.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172113+0000", 2024-09-17T07:30:39.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172115+0000", 2024-09-17T07:30:39.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.564e-06 2024-09-17T07:30:39.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172120+0000", 2024-09-17T07:30:39.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8099999999999997e-06 2024-09-17T07:30:39.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172136+0000", 2024-09-17T07:30:39.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6096e-05 2024-09-17T07:30:39.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172143+0000", 2024-09-17T07:30:39.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3329999999999999e-06 2024-09-17T07:30:39.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172241+0000", 2024-09-17T07:30:39.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8369000000000006e-05 2024-09-17T07:30:39.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172356+0000", 2024-09-17T07:30:39.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000114273 2024-09-17T07:30:39.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:39.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172455+0000", 2024-09-17T07:30:39.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9159000000000004e-05 2024-09-17T07:30:39.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:39.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173625+0000", 2024-09-17T07:30:39.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001170304 2024-09-17T07:30:39.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:39.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.174627+0000", 2024-09-17T07:30:39.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010016599999999999 2024-09-17T07:30:39.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:39.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.174685+0000", 2024-09-17T07:30:39.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8607000000000003e-05 2024-09-17T07:30:39.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.174737+0000", 2024-09-17T07:30:39.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1796999999999999e-05 2024-09-17T07:30:39.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.878 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.878 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:117250 3.5 e128/109 3:a4728dbe:::rbd_data.11832ac10dcc.0000000000000075:head v 128'521, mlcod=126'519)", 2024-09-17T07:30:39.878 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:30:39.878 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 2.6027025350000002, 2024-09-17T07:30:39.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0034023949999999999, 2024-09-17T07:30:39.878 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:39.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:39.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:39.879 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:39.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:39.879 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:30:39.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:39.879 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:30:39.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:39.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:39.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:30:39.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5900000000000002e-07 2024-09-17T07:30:39.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:39.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172910+0000", 2024-09-17T07:30:39.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2449999999999998e-06 2024-09-17T07:30:39.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:39.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172919+0000", 2024-09-17T07:30:39.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5389999999999996e-06 2024-09-17T07:30:39.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:39.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173431+0000", 2024-09-17T07:30:39.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00051224699999999996 2024-09-17T07:30:39.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:39.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173485+0000", 2024-09-17T07:30:39.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3936e-05 2024-09-17T07:30:39.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:39.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173504+0000", 2024-09-17T07:30:39.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8681e-05 2024-09-17T07:30:39.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:39.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.176259+0000", 2024-09-17T07:30:39.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027547719999999999 2024-09-17T07:30:39.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:39.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:39.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:39.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.176306+0000", 2024-09-17T07:30:39.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7515999999999997e-05 2024-09-17T07:30:39.882 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.882 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.882 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.883 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:39.883 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:39.883 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:39.884 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:39.885 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:39.883+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:39.984 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:39.984+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:40.033 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:40.033 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:40.033 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:40.033 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:40.041 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:40.085 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:40.085+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:40.131 INFO:teuthology.orchestra.run.smithi045.stdout:moved 6 / 82 (7.31707%) 2024-09-17T07:30:40.131 INFO:teuthology.orchestra.run.smithi045.stdout:avg 10.25 2024-09-17T07:30:40.131 INFO:teuthology.orchestra.run.smithi045.stdout:stddev 2.94746 -> 3.30719 (expected baseline 2.99479) 2024-09-17T07:30:40.131 INFO:teuthology.orchestra.run.smithi045.stdout:min osd.6 with 5 -> 6 pgs (0.487805 -> 0.585366 * mean) 2024-09-17T07:30:40.131 INFO:teuthology.orchestra.run.smithi045.stdout:max osd.0 with 13 -> 15 pgs (1.26829 -> 1.46341 * mean) 2024-09-17T07:30:40.131 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:30:40.131 INFO:teuthology.orchestra.run.smithi045.stdout:oload 110 2024-09-17T07:30:40.131 INFO:teuthology.orchestra.run.smithi045.stdout:max_change 1 2024-09-17T07:30:40.131 INFO:teuthology.orchestra.run.smithi045.stdout:max_change_osds 4 2024-09-17T07:30:40.131 INFO:teuthology.orchestra.run.smithi045.stdout:average_utilization 0.0006 2024-09-17T07:30:40.131 INFO:teuthology.orchestra.run.smithi045.stdout:overload_utilization 0.0007 2024-09-17T07:30:40.132 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 weight 1.0000 -> 0.6866 2024-09-17T07:30:40.132 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 weight 0.8526 -> 0.7258 2024-09-17T07:30:40.135 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-17T07:30:40.156 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-17T07:30:40.157 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-17T07:30:40.186 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:40.186+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:40.200 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:40.201 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:40.201 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:40.201 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:40.201 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:40.210 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:40.287 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:40.287+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:40.365 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:40.365 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:40.365 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:40.365 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:40.365 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.365 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.365 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:40.365 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.230065467, 2024-09-17T07:30:40.365 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:30:40.365 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.366 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.366 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.366 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.366 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.366 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:30:40.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.366 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.366 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.366 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:40.366 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.366 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.366 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.367 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:40.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.367 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.367 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:30:40.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:30:40.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.367 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.367 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.367 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:30:40.367 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:30:40.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:30:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:30:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:30:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:30:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.368 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:30:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:30:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:30:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:30:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:30:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:30:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:30:40.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:30:40.370 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:30:40.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.370 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:30:40.370 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:30:40.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.370 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.370 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.371 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:30:40.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:30:40.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.371 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.371 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:30:40.371 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:30:40.371 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.371 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.371 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.371 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.371 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.372 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:40.372 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.924098763000003, 2024-09-17T07:30:40.372 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:30:40.372 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.372 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.372 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.372 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.372 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.372 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.373 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:30:40.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.373 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.373 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.373 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:40.373 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.373 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.373 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:40.373 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:30:40.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:30:40.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:30:40.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:30:40.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.374 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.374 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:30:40.374 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:30:40.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:30:40.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:30:40.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:30:40.375 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:30:40.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.375 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.375 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.375 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:30:40.376 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:30:40.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.376 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.376 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:30:40.376 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:30:40.376 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.376 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.376 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.376 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.376 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:40.377 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.924076647999996, 2024-09-17T07:30:40.377 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:30:40.377 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.377 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.377 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.377 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.377 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:30:40.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.377 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.377 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.377 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.377 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:40.378 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.378 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.378 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:40.378 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.378 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.378 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:30:40.378 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:40.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.378 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.378 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:30:40.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:30:40.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:30:40.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:30:40.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.379 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.379 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:30:40.379 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:30:40.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.379 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.380 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.380 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:30:40.380 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:30:40.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.380 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.380 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:30:40.380 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:30:40.380 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.380 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.380 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.380 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.381 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:40.381 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.924052544000006, 2024-09-17T07:30:40.381 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:30:40.381 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.381 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.381 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.381 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.381 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.381 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:30:40.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.381 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.381 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.381 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.382 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:40.382 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.382 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.382 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:40.382 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.382 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.382 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:30:40.382 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:40.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.382 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.383 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.383 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:30:40.383 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:30:40.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.383 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.383 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:30:40.383 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:30:40.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.383 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.383 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.383 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:30:40.383 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:30:40.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.384 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.384 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.384 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:30:40.384 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:30:40.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.384 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.384 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.384 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:30:40.384 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:30:40.384 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.384 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.384 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.384 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.385 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.385 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:40.385 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.924028454999998, 2024-09-17T07:30:40.385 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:30:40.385 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.385 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.385 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.385 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.385 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.385 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.385 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:30:40.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.385 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.386 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.386 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:40.386 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.386 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.386 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:40.386 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.386 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.386 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.386 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:40.386 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:30:40.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.387 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.387 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:30:40.387 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:30:40.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.387 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.387 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:30:40.387 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:30:40.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.387 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.387 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.388 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:30:40.388 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:30:40.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.388 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.388 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:30:40.388 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:30:40.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.388 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.388 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:30:40.388 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:30:40.388 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.389 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.389 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.389 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.389 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.389 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:40.389 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.924004035999999, 2024-09-17T07:30:40.389 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:30:40.389 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.389 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.389 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.390 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.390 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:30:40.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.390 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.390 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.390 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:40.390 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.390 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.390 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.390 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:40.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.391 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.391 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:40.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:30:40.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.391 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.391 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:30:40.391 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:30:40.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.391 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:30:40.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:30:40.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:30:40.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:40.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.392 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.392 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:30:40.392 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:30:40.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.393 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.393 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:30:40.393 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:30:40.393 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.393 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.393 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.393 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.393 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.393 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:40.393 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.923982335999995, 2024-09-17T07:30:40.394 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:30:40.394 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.394 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.394 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.394 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.394 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.394 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.394 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:30:40.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.394 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.394 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.394 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.394 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:40.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.395 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:40.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.395 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:40.395 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:30:40.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.395 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:30:40.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:30:40.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.396 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:30:40.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:30:40.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.396 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.396 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.396 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:30:40.396 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:30:40.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.397 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.397 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:30:40.397 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:30:40.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.397 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.397 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:30:40.397 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:30:40.397 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.397 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.397 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.398 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.398 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:40.398 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.923937604000002, 2024-09-17T07:30:40.398 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:30:40.398 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.398 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.398 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.398 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.398 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:30:40.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.398 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:40.399 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:40.399 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.399 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.399 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.399 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:30:40.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:30:40.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:30:40.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:30:40.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.400 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.400 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:30:40.400 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:30:40.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.400 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:30:40.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:30:40.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:30:40.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:30:40.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.401 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.401 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:30:40.401 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:30:40.401 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.402 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.402 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.402 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:40.402 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.923909137999999, 2024-09-17T07:30:40.402 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:30:40.402 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.402 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.402 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.402 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.402 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:30:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.404 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:40.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:30:40.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.404 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:30:40.404 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:30:40.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.404 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.404 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.404 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:30:40.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:30:40.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:30:40.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:30:40.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.405 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.405 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:30:40.405 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:30:40.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.405 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.406 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.406 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:30:40.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:30:40.406 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.406 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.406 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.406 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.406 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.406 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:40.406 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.923791886000004, 2024-09-17T07:30:40.406 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:30:40.406 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.407 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.407 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:30:40.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.407 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.407 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.407 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:40.407 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.408 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.408 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:40.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.408 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.408 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:30:40.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:30:40.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.408 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.408 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.408 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:30:40.408 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:30:40.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.409 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:30:40.409 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:30:40.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.409 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:30:40.409 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:30:40.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.409 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:30:40.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:30:40.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.410 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.410 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:30:40.410 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:30:40.410 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.410 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.410 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.410 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.410 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.411 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:40.411 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.923365783999998, 2024-09-17T07:30:40.411 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:30:40.411 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.411 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.411 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.411 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.411 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.411 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:30:40.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.411 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.411 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:40.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:40.412 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.412 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.412 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.412 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:30:40.413 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:40.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.413 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.413 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:30:40.413 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:30:40.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.413 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.413 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:30:40.413 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:30:40.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.413 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.413 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:30:40.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:30:40.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.414 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:30:40.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:30:40.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.414 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.414 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.414 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:30:40.414 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:30:40.414 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.414 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.415 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.415 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.415 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.415 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:40.415 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.923346112999994, 2024-09-17T07:30:40.415 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:30:40.415 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.415 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.415 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.415 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.415 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:30:40.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.416 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.416 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:40.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.416 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:40.416 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.416 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.416 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.416 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:40.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:30:40.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:30:40.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:30:40.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.417 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:30:40.417 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:30:40.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.417 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.417 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:30:40.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:30:40.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:30:40.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:30:40.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.418 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.418 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:30:40.418 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:30:40.418 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.418 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.419 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.419 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.419 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.419 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:40.419 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.923323249000006, 2024-09-17T07:30:40.419 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:30:40.419 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.419 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.419 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.419 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.419 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.419 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:30:40.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.420 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:40.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:40.420 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.420 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.420 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:40.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:30:40.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.421 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:30:40.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:30:40.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.421 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.421 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:30:40.421 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:30:40.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.421 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:30:40.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:30:40.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:30:40.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:30:40.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.422 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.422 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.422 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:30:40.422 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:30:40.422 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.422 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.923301948000002, 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.423 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:40.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:40.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.424 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.424 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:30:40.424 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:30:40.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:30:40.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:30:40.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:30:40.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:30:40.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.425 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.425 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:30:40.425 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:30:40.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.426 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.426 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:30:40.426 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:30:40.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.426 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.426 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.426 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:30:40.426 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:30:40.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.426 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.426 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.923254159999999, 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.427 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:40.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:40.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.428 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.428 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:30:40.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:30:40.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:30:40.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:30:40.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:30:40.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:30:40.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:30:40.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:30:40.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.430 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:40.431 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.923226937999999, 2024-09-17T07:30:40.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:30:40.431 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.431 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.431 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.431 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.431 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.431 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:30:40.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.431 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.431 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.431 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:40.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:40.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:40.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:30:40.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:30:40.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:30:40.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:30:40.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:30:40.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:30:40.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:30:40.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:30:40.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:30:40.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:30:40.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:30:40.434 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.434 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.434 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.435 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.435 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:40.435 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.923206551000007, 2024-09-17T07:30:40.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:30:40.435 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.435 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.435 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.435 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.435 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.435 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:30:40.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.435 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:40.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:40.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:30:40.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:30:40.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:30:40.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:30:40.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:30:40.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:30:40.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:30:40.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:30:40.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:30:40.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:30:40.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:30:40.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:30:40.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.438 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.439 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.439 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.439 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:40.439 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.923186845999993, 2024-09-17T07:30:40.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:30:40.439 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.439 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.439 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.439 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.439 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:30:40.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.440 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:40.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:40.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:30:40.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:30:40.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:30:40.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:30:40.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:30:40.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:30:40.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:30:40.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:30:40.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:30:40.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:30:40.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:30:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:30:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.923163767999995, 2024-09-17T07:30:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:30:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.443 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.444 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.444 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:30:40.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.444 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.444 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:40.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.444 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:40.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:30:40.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:30:40.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:30:40.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:30:40.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:30:40.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:30:40.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:30:40.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:30:40.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:30:40.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:30:40.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:30:40.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:30:40.447 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.447 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.447 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.447 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:40.447 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:40.447 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.923143206000006, 2024-09-17T07:30:40.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:30:40.447 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.447 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:40.447 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.447 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.448 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.448 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:30:40.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.448 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:40.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:40.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:40.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:30:40.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:30:40.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:30:40.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:30:40.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:30:40.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:30:40.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:30:40.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:30:40.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:30:40.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:30:40.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:30:40.450 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.450 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.451 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.451 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.451 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.451 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:40.452 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:40.453 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:40.452+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:40.502 INFO:teuthology.orchestra.run.smithi045.stdout:moved 0 / 82 (0%) 2024-09-17T07:30:40.502 INFO:teuthology.orchestra.run.smithi045.stdout:avg 10.25 2024-09-17T07:30:40.502 INFO:teuthology.orchestra.run.smithi045.stdout:stddev 3.30719 -> 3.30719 (expected baseline 2.99479) 2024-09-17T07:30:40.502 INFO:teuthology.orchestra.run.smithi045.stdout:min osd.2 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-17T07:30:40.503 INFO:teuthology.orchestra.run.smithi045.stdout:max osd.0 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-17T07:30:40.503 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:30:40.503 INFO:teuthology.orchestra.run.smithi045.stdout:oload 1000 2024-09-17T07:30:40.503 INFO:teuthology.orchestra.run.smithi045.stdout:max_change 0.05 2024-09-17T07:30:40.503 INFO:teuthology.orchestra.run.smithi045.stdout:max_change_osds 4 2024-09-17T07:30:40.503 INFO:teuthology.orchestra.run.smithi045.stdout:average_utilization 0.0006 2024-09-17T07:30:40.503 INFO:teuthology.orchestra.run.smithi045.stdout:overload_utilization 0.0063 2024-09-17T07:30:40.503 INFO:teuthology.orchestra.run.smithi045.stderr:no change 2024-09-17T07:30:40.516 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-17T07:30:40.516 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-17T07:30:40.553 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:40.552+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:40.561 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:40.561 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:40.561 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:40.561 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:40.571 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:40.654 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:40.654+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:40.680 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:40.680 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:40.681 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:40.681 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:40.681 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:40.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:40.755 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:40.754+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:40.800 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.800 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.665293096, 2024-09-17T07:30:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:30:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:30:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:30:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:30:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:30:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:30:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:30:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:30:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:30:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:30:40.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:30:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:30:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:30:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:30:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:30:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:30:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:30:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:30:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:40.806 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.665256199, 2024-09-17T07:30:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:30:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:30:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:40.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:30:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:30:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:30:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:30:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:30:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:30:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:30:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:30:40.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:30:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:30:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:30:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:30:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:30:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:30:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:30:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:30:40.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:30:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:30:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:30:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:30:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:30:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.812 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.665230232, 2024-09-17T07:30:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:30:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.813 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:30:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:30:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:30:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:30:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:30:40.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:30:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:30:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:30:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:30:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:30:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:30:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:30:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:30:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:30:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:30:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:30:40.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:30:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:30:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:30:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:30:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:30:40.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:30:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.665205637, 2024-09-17T07:30:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:30:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.819 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:30:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:40.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:30:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:30:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:30:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:30:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:30:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:30:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:30:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:30:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:30:40.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:30:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:30:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:30:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:30:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:30:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:30:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:30:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:30:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:30:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:30:40.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:30:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:30:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.66518215000001, 2024-09-17T07:30:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:30:40.825 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:30:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:30:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:30:40.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:30:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:30:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:30:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:30:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:30:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:30:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:30:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:30:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:30:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:30:40.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:30:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:30:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:30:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:30:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:30:40.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:30:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:30:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:30:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.665168412, 2024-09-17T07:30:40.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:30:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:30:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.832 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:30:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:30:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:30:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:30:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:30:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:30:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:30:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:30:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:30:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:30:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:30:40.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:30:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:30:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:30:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:30:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:30:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:30:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:30:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:30:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.837 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.665138223, 2024-09-17T07:30:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:30:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:30:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:30:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:30:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:30:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:30:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:30:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:30:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:30:40.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:30:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:30:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:30:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:30:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:30:40.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:30:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:30:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:30:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:30:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:30:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:30:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:30:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:30:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.843 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.665101718, 2024-09-17T07:30:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:30:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:30:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:30:40.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:30:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:30:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:30:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:30:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:30:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:30:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:30:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:30:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:30:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:30:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:30:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:30:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:30:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:30:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:30:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:30:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:30:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:30:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:30:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.665073959, 2024-09-17T07:30:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:30:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:30:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:30:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:30:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:30:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:30:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:30:40.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:30:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:30:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:30:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:30:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:30:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:30:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:30:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:30:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:30:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:30:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:30:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:30:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:30:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:30:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:30:40.855 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.665051799, 2024-09-17T07:30:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:30:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.856 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:30:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:30:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:30:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:30:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:30:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:30:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:30:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:30:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:30:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:30:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:30:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:30:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:30:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:30:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:30:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:30:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:30:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:30:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:30:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:30:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:30:40.861 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.665038638, 2024-09-17T07:30:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:30:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.862 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:30:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:40.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:30:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:30:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:30:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.864 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:30:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:30:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:30:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:30:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:30:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:30:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:30:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:30:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:30:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:30:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:30:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:30:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:30:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:30:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:30:40.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:30:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:30:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.66502005, 2024-09-17T07:30:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:30:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:30:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.869 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:30:40.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:30:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:30:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:30:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:30:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:30:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:30:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:30:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:30:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:30:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:30:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:30:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:30:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:30:40.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:30:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:30:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:30:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:30:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.664987239, 2024-09-17T07:30:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:30:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:30:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:40.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:30:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:30:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:30:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:30:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:30:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:30:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:30:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:30:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:30:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:30:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:30:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:30:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:30:40.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:30:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:30:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:30:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:30:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:30:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.664968182, 2024-09-17T07:30:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:30:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:30:40.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.882 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:30:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:30:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:30:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:30:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:30:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:30:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:30:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:30:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:30:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:30:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:30:40.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:30:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:30:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:30:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:30:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:30:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:30:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:30:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:30:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:30:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:30:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:30:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.886 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.664946073, 2024-09-17T07:30:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:30:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:30:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:30:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:30:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:30:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:30:40.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:30:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:30:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:30:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:30:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:30:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:30:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:30:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:30:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:30:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:30:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:30:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:30:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:30:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:30:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:30:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:30:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.892 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.664919966, 2024-09-17T07:30:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:30:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:30:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:30:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:30:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:30:40.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:30:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:30:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:30:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:30:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:30:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:30:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:30:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:30:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:30:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:30:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:30:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:30:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:30:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:30:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:30:40.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:30:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:30:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.664890821, 2024-09-17T07:30:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:30:40.899 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:30:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:30:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:30:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:40.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:30:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:30:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:30:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:30:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:30:40.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:30:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:30:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:30:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:30:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:30:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:30:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:30:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:30:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:30:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:30:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:30:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:30:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:30:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.664859018, 2024-09-17T07:30:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:30:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:30:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:30:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:30:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:40.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:30:40.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.908 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.908 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:30:40.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:30:40.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.908 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.908 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:30:40.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:30:40.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:30:40.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:30:40.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:30:40.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:30:40.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:30:40.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:30:40.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:30:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:30:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:30:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:30:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:30:40.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:30:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:30:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:30:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:40.911 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.66483207100001, 2024-09-17T07:30:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:30:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:30:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:40.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:30:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:30:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.913 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:30:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:30:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:30:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:30:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:30:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:30:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:30:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:30:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:30:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:30:40.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:30:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:30:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:30:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:30:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.916 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:30:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:30:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:30:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:30:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:40.917 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.664791122, 2024-09-17T07:30:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:30:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:30:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:40.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:30:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:30:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:30:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:30:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:30:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:30:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:30:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:30:40.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:30:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:30:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:30:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:30:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.921 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:30:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:30:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:30:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:30:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:30:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:30:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:30:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:30:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:30:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:30:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:40.923 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:40.923 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:40.925 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:40.926 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-17T07:30:40.820+0000 7f9bb64ca640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-17T07:30:40.926 INFO:teuthology.orchestra.run.smithi045.stdout:moved 5 / 82 (6.09756%) 2024-09-17T07:30:40.926 INFO:teuthology.orchestra.run.smithi045.stdout:avg 10.25 2024-09-17T07:30:40.926 INFO:teuthology.orchestra.run.smithi045.stdout:stddev 3.30719 -> 4.4088 (expected baseline 2.99479) 2024-09-17T07:30:40.926 INFO:teuthology.orchestra.run.smithi045.stdout:min osd.2 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-17T07:30:40.926 INFO:teuthology.orchestra.run.smithi045.stdout:max osd.0 with 15 -> 16 pgs (1.46341 -> 1.56098 * mean) 2024-09-17T07:30:40.926 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:30:40.926 INFO:teuthology.orchestra.run.smithi045.stdout:oload 110 2024-09-17T07:30:40.926 INFO:teuthology.orchestra.run.smithi045.stdout:max_change 1 2024-09-17T07:30:40.927 INFO:teuthology.orchestra.run.smithi045.stdout:max_change_osds 4 2024-09-17T07:30:40.927 INFO:teuthology.orchestra.run.smithi045.stdout:average_utilization 0.0006 2024-09-17T07:30:40.927 INFO:teuthology.orchestra.run.smithi045.stdout:overload_utilization 0.0007 2024-09-17T07:30:40.927 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 weight 0.6866 -> 0.4710 2024-09-17T07:30:40.927 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 weight 0.7258 -> 0.6210 2024-09-17T07:30:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:no change 2024-09-17T07:30:40.928 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-17T07:30:40.928 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-17T07:30:40.928 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:40.924+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:41.026 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:41.026+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:41.076 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:41.076 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:41.076 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:41.076 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:41.086 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:41.127 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:41.127+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:41.228 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:41.228+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:41.262 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:41.262 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:41.263 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:41.263 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:41.274 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:41.307 INFO:teuthology.orchestra.run.smithi045.stdout:moved 0 / 82 (0%) 2024-09-17T07:30:41.307 INFO:teuthology.orchestra.run.smithi045.stdout:avg 10.25 2024-09-17T07:30:41.308 INFO:teuthology.orchestra.run.smithi045.stdout:stddev 3.30719 -> 3.30719 (expected baseline 2.99479) 2024-09-17T07:30:41.308 INFO:teuthology.orchestra.run.smithi045.stdout:min osd.2 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-17T07:30:41.308 INFO:teuthology.orchestra.run.smithi045.stdout:max osd.0 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-17T07:30:41.308 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:30:41.308 INFO:teuthology.orchestra.run.smithi045.stdout:oload 1000 2024-09-17T07:30:41.308 INFO:teuthology.orchestra.run.smithi045.stdout:max_change 0.05 2024-09-17T07:30:41.308 INFO:teuthology.orchestra.run.smithi045.stdout:max_change_osds 4 2024-09-17T07:30:41.308 INFO:teuthology.orchestra.run.smithi045.stdout:average_utilization 0.0006 2024-09-17T07:30:41.308 INFO:teuthology.orchestra.run.smithi045.stdout:overload_utilization 0.0063 2024-09-17T07:30:41.308 INFO:teuthology.orchestra.run.smithi045.stderr:no change 2024-09-17T07:30:41.329 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:41.328+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:41.428 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:41.428 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17069 3.0 e41/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 41'110, mlcod=40'108)", 2024-09-17T07:30:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 98.686880082000002, 2024-09-17T07:30:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011408908000000001, 2024-09-17T07:30:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739753+0000", 2024-09-17T07:30:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:30:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743532+0000", 2024-09-17T07:30:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037794510000000001 2024-09-17T07:30:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743557+0000", 2024-09-17T07:30:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5253e-05 2024-09-17T07:30:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743569+0000", 2024-09-17T07:30:41.431 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1161999999999999e-05 2024-09-17T07:30:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743650+0000", 2024-09-17T07:30:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1087999999999993e-05 2024-09-17T07:30:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743674+0000", 2024-09-17T07:30:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4562e-05 2024-09-17T07:30:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.432 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751095+0000", 2024-09-17T07:30:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074211140000000004 2024-09-17T07:30:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751161+0000", 2024-09-17T07:30:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5707e-05 2024-09-17T07:30:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.433 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17240 3.0 e42/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 42'113, mlcod=41'110)", 2024-09-17T07:30:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 98.201221598000004, 2024-09-17T07:30:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064392440000000002, 2024-09-17T07:30:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:30:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225418+0000", 2024-09-17T07:30:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8360000000000003e-06 2024-09-17T07:30:41.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225426+0000", 2024-09-17T07:30:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2670000000000006e-06 2024-09-17T07:30:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225434+0000", 2024-09-17T07:30:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0560000000000005e-06 2024-09-17T07:30:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.436 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225514+0000", 2024-09-17T07:30:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9979000000000001e-05 2024-09-17T07:30:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225535+0000", 2024-09-17T07:30:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0965e-05 2024-09-17T07:30:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231797+0000", 2024-09-17T07:30:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062621129999999997 2024-09-17T07:30:41.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231850+0000", 2024-09-17T07:30:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2611999999999997e-05 2024-09-17T07:30:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:32543 3.0 e55/23 3:03bfd97d:::rbd_data.11839a97b753.0000000000000007:head v 55'150, mlcod=54'148)", 2024-09-17T07:30:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 83.971995344999996, 2024-09-17T07:30:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031813480000000001, 2024-09-17T07:30:41.438 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5199999999999998e-07 2024-09-17T07:30:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455083+0000", 2024-09-17T07:30:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044629199999999999 2024-09-17T07:30:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455098+0000", 2024-09-17T07:30:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4183e-05 2024-09-17T07:30:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455106+0000", 2024-09-17T07:30:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:30:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455173+0000", 2024-09-17T07:30:41.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6346000000000005e-05 2024-09-17T07:30:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455200+0000", 2024-09-17T07:30:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7319000000000001e-05 2024-09-17T07:30:41.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457774+0000", 2024-09-17T07:30:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025742650000000001 2024-09-17T07:30:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457818+0000", 2024-09-17T07:30:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3891999999999999e-05 2024-09-17T07:30:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33735 3.1f e55/52 3:fea54097:::rbd_data.11839a97b753.0000000000000000:head v 55'260, mlcod=55'258)", 2024-09-17T07:30:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 83.602062716999995, 2024-09-17T07:30:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035215200000000002, 2024-09-17T07:30:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824571+0000", 2024-09-17T07:30:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.164e-06 2024-09-17T07:30:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825118+0000", 2024-09-17T07:30:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054696400000000004 2024-09-17T07:30:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825134+0000", 2024-09-17T07:30:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6342999999999999e-05 2024-09-17T07:30:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825142+0000", 2024-09-17T07:30:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5270000000000001e-06 2024-09-17T07:30:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825200+0000", 2024-09-17T07:30:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.893e-05 2024-09-17T07:30:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825218+0000", 2024-09-17T07:30:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7935999999999999e-05 2024-09-17T07:30:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828037+0000", 2024-09-17T07:30:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002818961 2024-09-17T07:30:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828091+0000", 2024-09-17T07:30:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3695e-05 2024-09-17T07:30:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.448 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33869 3.9 e55/23 3:907b81b5:::rbd_data.11839a97b753.0000000000000001:head v 55'171, mlcod=55'169)", 2024-09-17T07:30:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 83.585006894000003, 2024-09-17T07:30:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033990859999999999, 2024-09-17T07:30:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841626+0000", 2024-09-17T07:30:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.023e-06 2024-09-17T07:30:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842067+0000", 2024-09-17T07:30:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000440612 2024-09-17T07:30:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842080+0000", 2024-09-17T07:30:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.29e-05 2024-09-17T07:30:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842086+0000", 2024-09-17T07:30:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2090000000000002e-06 2024-09-17T07:30:41.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842144+0000", 2024-09-17T07:30:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7627999999999998e-05 2024-09-17T07:30:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842171+0000", 2024-09-17T07:30:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7214000000000001e-05 2024-09-17T07:30:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.844973+0000", 2024-09-17T07:30:41.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002802232 2024-09-17T07:30:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.845024+0000", 2024-09-17T07:30:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1267999999999998e-05 2024-09-17T07:30:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51432 3.0 e59/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 59'177, mlcod=58'174)", 2024-09-17T07:30:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:41.453 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 78.605409877, 2024-09-17T07:30:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010790086000000001, 2024-09-17T07:30:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:41.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821224+0000", 2024-09-17T07:30:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.511e-06 2024-09-17T07:30:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825174+0000", 2024-09-17T07:30:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0039497949999999999 2024-09-17T07:30:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825196+0000", 2024-09-17T07:30:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2643999999999999e-05 2024-09-17T07:30:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825206+0000", 2024-09-17T07:30:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4320000000000005e-06 2024-09-17T07:30:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825272+0000", 2024-09-17T07:30:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5829999999999998e-05 2024-09-17T07:30:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825299+0000", 2024-09-17T07:30:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7107e-05 2024-09-17T07:30:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.831939+0000", 2024-09-17T07:30:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0066398510000000004 2024-09-17T07:30:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.832012+0000", 2024-09-17T07:30:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3916000000000002e-05 2024-09-17T07:30:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51969 3.0 e59/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 59'179, mlcod=59'177)", 2024-09-17T07:30:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 78.496541499000003, 2024-09-17T07:30:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060297249999999997, 2024-09-17T07:30:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.458 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930093+0000", 2024-09-17T07:30:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.764e-06 2024-09-17T07:30:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931779+0000", 2024-09-17T07:30:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001686379 2024-09-17T07:30:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931802+0000", 2024-09-17T07:30:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2682000000000001e-05 2024-09-17T07:30:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931811+0000", 2024-09-17T07:30:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5640000000000002e-06 2024-09-17T07:30:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931877+0000", 2024-09-17T07:30:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6155999999999995e-05 2024-09-17T07:30:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931896+0000", 2024-09-17T07:30:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9111e-05 2024-09-17T07:30:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936063+0000", 2024-09-17T07:30:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004166711 2024-09-17T07:30:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936121+0000", 2024-09-17T07:30:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7358000000000002e-05 2024-09-17T07:30:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54417 3.0 e60/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 60'182, mlcod=59'179)", 2024-09-17T07:30:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 77.989024508, 2024-09-17T07:30:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0059885219999999996, 2024-09-17T07:30:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.463 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437609+0000", 2024-09-17T07:30:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5829999999999999e-06 2024-09-17T07:30:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437623+0000", 2024-09-17T07:30:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3348e-05 2024-09-17T07:30:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437643+0000", 2024-09-17T07:30:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0463999999999999e-05 2024-09-17T07:30:41.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437653+0000", 2024-09-17T07:30:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6290000000000005e-06 2024-09-17T07:30:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437719+0000", 2024-09-17T07:30:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6428999999999997e-05 2024-09-17T07:30:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437745+0000", 2024-09-17T07:30:41.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5690999999999998e-05 2024-09-17T07:30:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443549+0000", 2024-09-17T07:30:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0058038259999999998 2024-09-17T07:30:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443596+0000", 2024-09-17T07:30:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7552000000000001e-05 2024-09-17T07:30:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.467 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54419 3.0 e60/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 60'184, mlcod=59'179)", 2024-09-17T07:30:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 77.982672151000003, 2024-09-17T07:30:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004706844, 2024-09-17T07:30:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443961+0000", 2024-09-17T07:30:41.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:30:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443967+0000", 2024-09-17T07:30:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.268e-06 2024-09-17T07:30:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443983+0000", 2024-09-17T07:30:41.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5248999999999999e-05 2024-09-17T07:30:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443990+0000", 2024-09-17T07:30:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5120000000000002e-06 2024-09-17T07:30:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444056+0000", 2024-09-17T07:30:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6197000000000003e-05 2024-09-17T07:30:41.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444081+0000", 2024-09-17T07:30:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4791e-05 2024-09-17T07:30:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448616+0000", 2024-09-17T07:30:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045346129999999998 2024-09-17T07:30:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448667+0000", 2024-09-17T07:30:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-17T07:30:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:80794 3.9 e72/23 3:90311a61:::rbd_data.1183cb746bb8.0000000000000007:head v 72'222, mlcod=71'220)", 2024-09-17T07:30:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 63.994683266999999, 2024-09-17T07:30:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004371733, 2024-09-17T07:30:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.474 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0800000000000001e-07 2024-09-17T07:30:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434108+0000", 2024-09-17T07:30:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021587569999999999 2024-09-17T07:30:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434120+0000", 2024-09-17T07:30:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1821000000000001e-05 2024-09-17T07:30:41.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434128+0000", 2024-09-17T07:30:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7579999999999992e-06 2024-09-17T07:30:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434191+0000", 2024-09-17T07:30:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3202999999999998e-05 2024-09-17T07:30:41.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434218+0000", 2024-09-17T07:30:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6771000000000002e-05 2024-09-17T07:30:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436289+0000", 2024-09-17T07:30:41.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020717700000000001 2024-09-17T07:30:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436321+0000", 2024-09-17T07:30:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1445000000000001e-05 2024-09-17T07:30:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 75/74 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head, version: 36'126, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head@36'126, size: 9431460, copy_subset: [], clone_subset: {3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9: [(0, 9431460)]}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:41.479 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 59.685775673999999, 2024-09-17T07:30:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041288289999999997, 2024-09-17T07:30:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740858+0000", 2024-09-17T07:30:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1599999999999999e-06 2024-09-17T07:30:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740885+0000", 2024-09-17T07:30:41.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.707e-05 2024-09-17T07:30:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740925+0000", 2024-09-17T07:30:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0649000000000003e-05 2024-09-17T07:30:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740957+0000", 2024-09-17T07:30:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1585999999999999e-05 2024-09-17T07:30:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741185+0000", 2024-09-17T07:30:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022759700000000001 2024-09-17T07:30:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741228+0000", 2024-09-17T07:30:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3705999999999998e-05 2024-09-17T07:30:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.744985+0000", 2024-09-17T07:30:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037570609999999999 2024-09-17T07:30:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.483 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85061 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'257, mlcod=79'255)", 2024-09-17T07:30:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 56.277944718000001, 2024-09-17T07:30:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033850550000000001, 2024-09-17T07:30:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:41.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9699999999999995e-07 2024-09-17T07:30:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148835+0000", 2024-09-17T07:30:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000146852 2024-09-17T07:30:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148845+0000", 2024-09-17T07:30:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0232e-05 2024-09-17T07:30:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148852+0000", 2024-09-17T07:30:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6490000000000002e-06 2024-09-17T07:30:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148913+0000", 2024-09-17T07:30:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1099999999999994e-05 2024-09-17T07:30:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148931+0000", 2024-09-17T07:30:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7938e-05 2024-09-17T07:30:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152025+0000", 2024-09-17T07:30:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0030942729999999998 2024-09-17T07:30:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152073+0000", 2024-09-17T07:30:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7213999999999999e-05 2024-09-17T07:30:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85063 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'259, mlcod=79'255)", 2024-09-17T07:30:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 56.276797764000001, 2024-09-17T07:30:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037722659999999998, 2024-09-17T07:30:41.488 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5799999999999999e-07 2024-09-17T07:30:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149841+0000", 2024-09-17T07:30:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4840000000000003e-06 2024-09-17T07:30:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149849+0000", 2024-09-17T07:30:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4670000000000004e-06 2024-09-17T07:30:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149854+0000", 2024-09-17T07:30:41.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4779999999999999e-06 2024-09-17T07:30:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149920+0000", 2024-09-17T07:30:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5760999999999996e-05 2024-09-17T07:30:41.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149949+0000", 2024-09-17T07:30:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8838000000000001e-05 2024-09-17T07:30:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153558+0000", 2024-09-17T07:30:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036092939999999999 2024-09-17T07:30:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.494 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153607+0000", 2024-09-17T07:30:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8386000000000001e-05 2024-09-17T07:30:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:94996 3.0 e82/23 3:074b4f56:::rbd_data.118311e0d6f8.0000000000000001:head v 82'715, mlcod=80'713)", 2024-09-17T07:30:41.495 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.810892170999999, 2024-09-17T07:30:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031499179999999998, 2024-09-17T07:30:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.496 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615741+0000", 2024-09-17T07:30:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999996e-07 2024-09-17T07:30:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616917+0000", 2024-09-17T07:30:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011763310000000001 2024-09-17T07:30:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616936+0000", 2024-09-17T07:30:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8624e-05 2024-09-17T07:30:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616948+0000", 2024-09-17T07:30:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.238e-05 2024-09-17T07:30:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617054+0000", 2024-09-17T07:30:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000106556 2024-09-17T07:30:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617081+0000", 2024-09-17T07:30:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.622e-05 2024-09-17T07:30:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618836+0000", 2024-09-17T07:30:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001755412 2024-09-17T07:30:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618890+0000", 2024-09-17T07:30:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3915000000000001e-05 2024-09-17T07:30:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:95007 3.5 e82/76 3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head v 82'422, mlcod=82'420)", 2024-09-17T07:30:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 51.802969470000001, 2024-09-17T07:30:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003939901, 2024-09-17T07:30:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.502 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623664+0000", 2024-09-17T07:30:41.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3799999999999999e-06 2024-09-17T07:30:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626104+0000", 2024-09-17T07:30:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002440149 2024-09-17T07:30:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626120+0000", 2024-09-17T07:30:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5962000000000001e-05 2024-09-17T07:30:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626127+0000", 2024-09-17T07:30:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8340000000000001e-06 2024-09-17T07:30:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626186+0000", 2024-09-17T07:30:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9234999999999998e-05 2024-09-17T07:30:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626208+0000", 2024-09-17T07:30:41.505 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1719e-05 2024-09-17T07:30:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627567+0000", 2024-09-17T07:30:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013587919999999999 2024-09-17T07:30:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627603+0000", 2024-09-17T07:30:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5830000000000001e-05 2024-09-17T07:30:41.506 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:104928 3.18 e87/76 3:1ef990b1:::rbd_data.1183dd51323.0000000000000006:head v 87'562, mlcod=86'559)", 2024-09-17T07:30:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 45.884213758999998, 2024-09-17T07:30:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0061095869999999997, 2024-09-17T07:30:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542421+0000", 2024-09-17T07:30:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.069e-06 2024-09-17T07:30:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544190+0000", 2024-09-17T07:30:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017693539999999999 2024-09-17T07:30:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.509 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544215+0000", 2024-09-17T07:30:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5398999999999999e-05 2024-09-17T07:30:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544229+0000", 2024-09-17T07:30:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3865e-05 2024-09-17T07:30:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544322+0000", 2024-09-17T07:30:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2540000000000005e-05 2024-09-17T07:30:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544348+0000", 2024-09-17T07:30:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6197999999999999e-05 2024-09-17T07:30:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548475+0000", 2024-09-17T07:30:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041266000000000002 2024-09-17T07:30:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548528+0000", 2024-09-17T07:30:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3562e-05 2024-09-17T07:30:41.511 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115898 3.9 3:910101b0:::rbd_data.118340e4fbf.0000000000000000:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:30:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.235699654000001, 2024-09-17T07:30:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0042466279999999997, 2024-09-17T07:30:41.512 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115898 2024-09-17T07:30:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:30:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:30:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0030000000000001e-06 2024-09-17T07:30:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.514 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190940+0000", 2024-09-17T07:30:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2349999999999999e-06 2024-09-17T07:30:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190953+0000", 2024-09-17T07:30:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3304e-05 2024-09-17T07:30:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.515 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190995+0000", 2024-09-17T07:30:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2317000000000002e-05 2024-09-17T07:30:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191592+0000", 2024-09-17T07:30:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00059607799999999999 2024-09-17T07:30:41.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191888+0000", 2024-09-17T07:30:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00029619999999999999 2024-09-17T07:30:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192069+0000", 2024-09-17T07:30:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00018125899999999999 2024-09-17T07:30:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:41.517 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193692+0000", 2024-09-17T07:30:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001623364 2024-09-17T07:30:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194968+0000", 2024-09-17T07:30:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012755189999999999 2024-09-17T07:30:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.518 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.195037+0000", 2024-09-17T07:30:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9450999999999996e-05 2024-09-17T07:30:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.195179+0000", 2024-09-17T07:30:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000141898 2024-09-17T07:30:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.519 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115936 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [watch watch cookie 140227662410368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.202780701999998, 2024-09-17T07:30:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043930569999999997, 2024-09-17T07:30:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115936 2024-09-17T07:30:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7300000000000005e-07 2024-09-17T07:30:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223856+0000", 2024-09-17T07:30:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9169999999999999e-06 2024-09-17T07:30:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223870+0000", 2024-09-17T07:30:41.522 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3926e-05 2024-09-17T07:30:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223878+0000", 2024-09-17T07:30:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3699999999999997e-06 2024-09-17T07:30:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223934+0000", 2024-09-17T07:30:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6929000000000003e-05 2024-09-17T07:30:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.523 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224053+0000", 2024-09-17T07:30:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000118952 2024-09-17T07:30:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224271+0000", 2024-09-17T07:30:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021763300000000001 2024-09-17T07:30:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224992+0000", 2024-09-17T07:30:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072142099999999998 2024-09-17T07:30:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228099+0000", 2024-09-17T07:30:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031060850000000002 2024-09-17T07:30:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228143+0000", 2024-09-17T07:30:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4614999999999999e-05 2024-09-17T07:30:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228245+0000", 2024-09-17T07:30:41.525 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000101436 2024-09-17T07:30:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115955 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [call rbd.snapshot_trash_add in=21b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:30:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.183793948999998, 2024-09-17T07:30:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004259054, 2024-09-17T07:30:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:41.526 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115955 2024-09-17T07:30:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:30:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:30:41.527 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242839+0000", 2024-09-17T07:30:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6899999999999998e-07 2024-09-17T07:30:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242842+0000", 2024-09-17T07:30:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1350000000000001e-06 2024-09-17T07:30:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.528 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242849+0000", 2024-09-17T07:30:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0400000000000004e-06 2024-09-17T07:30:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242854+0000", 2024-09-17T07:30:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4940000000000003e-06 2024-09-17T07:30:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243137+0000", 2024-09-17T07:30:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00028217100000000001 2024-09-17T07:30:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243196+0000", 2024-09-17T07:30:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9608999999999998e-05 2024-09-17T07:30:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243448+0000", 2024-09-17T07:30:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00025185000000000002 2024-09-17T07:30:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243990+0000", 2024-09-17T07:30:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054153199999999995 2024-09-17T07:30:41.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.246867+0000", 2024-09-17T07:30:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028771880000000001 2024-09-17T07:30:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.246943+0000", 2024-09-17T07:30:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5689e-05 2024-09-17T07:30:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.531 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247097+0000", 2024-09-17T07:30:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015487699999999999 2024-09-17T07:30:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115956 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:30:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 19.183497365000001, 2024-09-17T07:30:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043741479999999996, 2024-09-17T07:30:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:41.532 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115956 2024-09-17T07:30:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:30:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:30:41.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243136+0000", 2024-09-17T07:30:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7100000000000001e-07 2024-09-17T07:30:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243141+0000", 2024-09-17T07:30:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.716e-06 2024-09-17T07:30:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243153+0000", 2024-09-17T07:30:41.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1198e-05 2024-09-17T07:30:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243160+0000", 2024-09-17T07:30:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6229999999999997e-06 2024-09-17T07:30:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243797+0000", 2024-09-17T07:30:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063716399999999996 2024-09-17T07:30:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.535 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for rw locks", 2024-09-17T07:30:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243852+0000", 2024-09-17T07:30:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4147999999999997e-05 2024-09-17T07:30:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247147+0000", 2024-09-17T07:30:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0032955940000000002 2024-09-17T07:30:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247271+0000", 2024-09-17T07:30:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012397800000000001 2024-09-17T07:30:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:41.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247509+0000", 2024-09-17T07:30:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000237956 2024-09-17T07:30:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:41.537 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:41.537 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:41.539 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:41.539 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:41.538+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:41.632 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:30:41.639 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:41.639+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:41.646 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:41.647 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:41.647 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:41.647 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:41.656 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:41.741 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:41.740+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:41.765 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:41.765 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:41.782 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:41.783 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:41.799 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:41.799 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:41.800 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:41.800 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:41.800 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:41.800 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:41.800 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:41.808 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:41.815 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:41.815 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:41.832 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:41.832 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:41.841 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:41.841+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:41.850 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:41.850 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:41.866 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:41.867 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:41.884 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:41.913 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:41.913 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:41.921 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:41.942 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:41.942+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:42.024 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:42.024 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:42.033 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:42.076 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:42.076+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:42.144 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:42.143+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:42.150 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:42.150 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:42.153 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T07:30:42.160 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:42.245 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:42.245+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:42.316 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:42.316 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:42.326 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:42.369 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:42.369+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:42.428 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:42.428 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:42.437 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:42.478 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:42.477+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:42.549 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:42.548+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:42.580 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:42.580 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:42.589 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:42.673 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:42.672+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:42.731 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:42.731 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:42.739 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:42.751 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:42.750+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:42.852 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:42.851+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:42.883 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:42.883 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:42.891 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:42.953 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:42.952+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:42.994 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:42.995 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:43.004 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:43.054 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:43.054+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:43.088 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-17T07:30:43.152 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:43.153 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:43.155 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:43.154+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:43.163 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:43.255 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:43.255+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:43.277 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:43.277 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:43.286 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:43.370 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:43.370+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:43.435 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:43.435 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:43.444 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:43.484 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:43.484+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:43.558 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:43.558+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:43.591 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:43.591 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:43.600 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:43.659 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:43.659+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:43.707 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:43.707 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:43.715 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:43.799 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:43.799+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:43.861 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:43.860+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:43.863 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:43.863 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:43.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:43.894 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:30:43.962 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:43.962+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:44.025 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:44.025 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:44.034 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:44.040 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:44.040 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:44.059 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:44.059 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:44.063 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:44.063+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:44.075 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:44.076 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:44.097 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:44.098 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:44.116 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:44.135 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:44.136 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:44.154 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:44.154 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:44.154 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:44.154 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:44.155 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:44.155 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:44.164 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:44.163+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:44.165 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:44.176 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:44.177 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:44.265 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:44.265+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:44.288 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:44.288 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:44.288 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:44.288 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:44.288 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:44.296 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:44.366 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:44.366+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:44.446 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:44.446 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:44.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:44.446 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:44.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.446 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.446 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:44.446 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.311230041, 2024-09-17T07:30:44.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:30:44.447 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.447 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.447 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.447 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.447 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.447 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:30:44.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.447 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:44.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:44.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:30:44.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:30:44.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:30:44.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:30:44.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:30:44.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:30:44.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.449 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.449 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.449 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:30:44.449 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:30:44.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:30:44.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:30:44.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:30:44.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:30:44.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:30:44.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:30:44.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:30:44.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:30:44.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:30:44.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:30:44.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:30:44.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:30:44.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:30:44.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:30:44.452 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.452 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.452 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.453 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.453 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:44.453 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.311193144, 2024-09-17T07:30:44.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:30:44.453 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.453 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.453 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.453 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.456 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.456 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.456 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:30:44.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.456 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:44.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:44.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.457 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.457 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:30:44.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:30:44.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:30:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:30:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:30:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:30:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:30:44.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:30:44.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:30:44.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:30:44.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:44.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:30:44.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:30:44.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:30:44.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:30:44.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:30:44.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:30:44.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:30:44.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:30:44.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:30:44.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:30:44.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:30:44.461 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.461 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.461 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.461 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.462 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:44.462 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.311167177, 2024-09-17T07:30:44.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:30:44.462 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.462 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.462 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.462 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.462 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.462 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.462 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:30:44.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.462 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:44.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:44.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:30:44.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:44.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:30:44.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:30:44.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:30:44.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:30:44.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:30:44.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:30:44.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:30:44.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:30:44.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:30:44.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:30:44.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:30:44.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:30:44.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:30:44.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:30:44.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:30:44.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:30:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:30:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:30:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:30:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:30:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.467 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.468 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.468 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:44.468 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.311142582, 2024-09-17T07:30:44.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:30:44.468 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.468 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.468 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.468 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.468 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.469 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:30:44.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.469 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:44.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:44.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:44.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:30:44.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:30:44.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:30:44.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.470 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.470 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:30:44.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:30:44.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:30:44.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:30:44.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:30:44.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:30:44.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:30:44.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:30:44.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:30:44.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:30:44.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:30:44.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:30:44.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:30:44.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:30:44.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:30:44.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:30:44.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.474 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:30:44.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:30:44.474 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.474 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.474 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.474 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.474 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:44.474 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.311119095, 2024-09-17T07:30:44.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:30:44.474 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.475 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.475 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.475 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.475 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.475 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.475 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:30:44.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.475 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:44.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:44.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:30:44.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:44.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:30:44.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:30:44.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:30:44.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:30:44.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:30:44.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:30:44.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:30:44.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:30:44.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:44.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:30:44.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:30:44.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:30:44.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:30:44.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:30:44.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:30:44.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:30:44.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:30:44.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:30:44.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:30:44.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:30:44.481 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.481 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.481 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.481 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.481 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:44.481 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.311105357, 2024-09-17T07:30:44.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:30:44.481 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.482 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.482 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.482 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.482 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:30:44.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.482 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:44.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:44.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:44.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:30:44.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:30:44.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:30:44.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:30:44.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:30:44.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:30:44.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:30:44.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:44.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:30:44.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:44.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:30:44.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:30:44.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:30:44.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:30:44.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:30:44.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:30:44.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:30:44.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:30:44.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:30:44.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.487 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.487 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:30:44.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:30:44.487 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.487 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.487 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.487 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.487 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:44.487 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.311075168, 2024-09-17T07:30:44.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:30:44.488 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.488 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.488 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.488 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.488 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.488 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.488 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:30:44.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.488 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.488 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.488 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:44.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:44.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:30:44.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:30:44.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:30:44.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:30:44.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:44.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:30:44.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:30:44.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:30:44.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.491 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.491 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:30:44.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:30:44.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.491 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.491 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:30:44.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:30:44.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.491 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.491 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:30:44.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:30:44.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.492 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.492 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:30:44.492 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:44.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.492 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.492 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:30:44.492 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:30:44.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.492 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.492 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:30:44.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:30:44.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:30:44.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:30:44.493 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.493 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.493 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.493 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.494 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:44.494 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.31103866300001, 2024-09-17T07:30:44.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:30:44.494 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.494 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.494 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.494 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.494 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.494 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.494 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:30:44.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.494 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.495 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:44.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.495 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.495 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:44.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.495 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.495 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:44.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:30:44.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.496 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.496 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:30:44.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:30:44.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.496 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.496 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:30:44.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:30:44.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.496 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.496 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:30:44.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:30:44.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:30:44.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:30:44.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:44.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:30:44.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:30:44.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:30:44.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:30:44.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:30:44.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:30:44.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:30:44.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:30:44.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:30:44.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:30:44.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:30:44.499 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.499 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.499 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.499 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.499 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:44.499 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.311010904, 2024-09-17T07:30:44.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:30:44.500 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.500 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.500 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.500 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.500 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.500 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.500 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:30:44.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.500 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:44.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:44.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:30:44.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:30:44.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:30:44.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:30:44.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.502 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.502 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:30:44.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:30:44.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.502 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.502 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:30:44.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:30:44.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.502 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:44.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:30:44.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:30:44.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:30:44.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:30:44.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:30:44.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.504 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:30:44.504 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:44.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.504 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:30:44.504 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:30:44.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.505 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:30:44.505 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:30:44.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.505 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.505 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:30:44.505 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:30:44.505 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.505 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.505 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.505 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.505 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:44.506 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.310988744, 2024-09-17T07:30:44.506 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:30:44.506 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.506 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.506 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.506 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.506 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.506 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:30:44.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.506 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.506 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.506 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.506 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:44.507 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.507 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.507 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:44.507 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.507 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.507 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:44.507 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:30:44.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.508 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.508 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:30:44.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:30:44.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.508 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.508 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:30:44.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:44.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.508 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.508 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:30:44.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:30:44.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.509 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.509 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:30:44.509 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:30:44.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.509 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.509 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:30:44.509 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:30:44.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.510 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.510 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:30:44.510 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:30:44.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.510 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.510 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:30:44.510 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:30:44.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.510 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.510 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:30:44.510 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:30:44.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.511 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.511 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:30:44.511 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:30:44.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.511 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.511 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:30:44.511 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:30:44.511 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.511 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.511 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.512 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.512 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:44.512 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.310975583, 2024-09-17T07:30:44.512 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:30:44.512 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.512 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.512 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.512 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.512 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.512 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:30:44.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.512 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.513 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.513 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:44.513 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.513 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.513 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:44.513 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.513 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.513 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:30:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:30:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:30:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:30:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:30:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:30:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:30:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:30:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:30:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:30:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:30:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:30:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:30:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:30:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:30:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:30:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:30:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:30:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:30:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:30:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.310956995, 2024-09-17T07:30:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:30:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:30:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:30:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:30:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:30:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:30:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:30:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:30:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:30:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:30:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:30:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:30:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:30:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:30:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:30:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:30:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:30:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:30:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:30:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:30:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.310924184, 2024-09-17T07:30:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:30:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:30:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.527 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.527 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:44.527 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:30:44.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.527 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.527 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:30:44.527 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:30:44.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.527 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.527 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:30:44.527 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:30:44.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.528 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.528 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:30:44.528 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:30:44.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.528 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.528 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:30:44.528 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:30:44.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.528 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:44.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:44.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.529 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:30:44.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:30:44.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.529 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:30:44.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:44.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.530 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.530 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:30:44.530 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:30:44.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.530 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.530 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:30:44.530 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:30:44.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.530 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.530 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:30:44.531 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:30:44.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.531 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.531 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.531 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:44.531 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.310905127, 2024-09-17T07:30:44.531 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:30:44.531 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.532 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.532 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.532 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.532 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:30:44.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.532 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.532 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.532 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:44.532 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:44.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:30:44.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:30:44.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:30:44.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:30:44.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:30:44.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:30:44.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:30:44.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:30:44.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:30:44.535 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:30:44.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.535 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.535 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:30:44.535 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:30:44.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.535 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.535 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:30:44.535 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:30:44.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.535 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.536 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:30:44.536 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:30:44.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.536 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.536 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:30:44.536 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:30:44.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.536 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.536 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:30:44.536 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:30:44.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.537 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.537 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:30:44.537 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:30:44.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.537 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.537 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.537 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:44.538 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.310883018, 2024-09-17T07:30:44.538 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:30:44.538 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.538 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.538 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.538 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.538 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.538 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.538 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:30:44.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.538 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.538 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.538 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.538 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:44.539 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.539 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.539 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:44.539 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.539 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.539 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:30:44.539 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:30:44.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.539 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.540 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:44.540 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:30:44.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.540 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.540 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:30:44.540 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:30:44.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.540 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.540 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:30:44.540 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:44.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.541 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.541 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:30:44.541 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:30:44.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.541 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.541 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:30:44.541 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:30:44.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.541 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.541 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:30:44.541 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:30:44.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.542 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.542 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:30:44.542 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:30:44.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.542 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.542 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:30:44.542 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:30:44.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.542 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.543 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:30:44.543 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:30:44.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.543 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.543 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:30:44.543 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:30:44.543 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.543 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.543 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.543 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.543 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:44.544 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.310856911, 2024-09-17T07:30:44.544 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:30:44.544 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.544 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.544 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.544 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.544 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.544 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:30:44.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.544 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.544 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:44.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.545 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:44.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.545 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:30:44.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:30:44.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.546 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.546 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:44.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:30:44.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.546 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.546 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:30:44.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:44.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.546 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.546 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:30:44.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:30:44.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:30:44.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:30:44.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:30:44.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:30:44.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:30:44.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:30:44.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.548 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.548 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:30:44.548 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:30:44.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.548 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.548 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:30:44.548 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:30:44.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.548 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.548 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:30:44.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:30:44.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.549 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.549 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:30:44.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:30:44.549 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.549 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.549 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.549 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.549 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:44.549 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.310827766, 2024-09-17T07:30:44.550 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:30:44.550 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.550 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.550 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.550 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.550 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.550 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:30:44.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.550 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.550 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.550 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:44.550 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.551 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.551 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:44.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.551 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.551 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:44.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:30:44.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.551 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:30:44.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:44.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:30:44.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:30:44.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.552 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.552 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:30:44.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:30:44.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:30:44.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:30:44.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:30:44.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:30:44.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:30:44.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:30:44.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:30:44.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:30:44.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:30:44.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:30:44.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.555 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:30:44.555 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:30:44.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.555 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.555 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:30:44.555 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:30:44.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.555 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.555 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.555 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:44.556 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.310795963, 2024-09-17T07:30:44.556 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:30:44.556 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.556 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.556 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.556 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.556 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:30:44.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.556 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.556 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.556 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:44.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:44.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:30:44.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:30:44.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.558 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:44.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:30:44.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.558 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:30:44.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:30:44.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.558 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:30:44.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:30:44.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:30:44.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:30:44.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:30:44.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:30:44.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:30:44.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:30:44.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.560 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.560 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:30:44.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:30:44.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.560 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.560 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:30:44.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:30:44.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.560 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:30:44.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:30:44.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.561 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:30:44.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:30:44.561 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.561 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.561 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.561 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.562 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:44.562 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.31076901599999, 2024-09-17T07:30:44.562 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:30:44.562 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.562 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.562 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.562 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.562 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.562 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.562 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:30:44.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.562 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:44.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:44.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:30:44.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:30:44.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:30:44.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:30:44.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:44.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:30:44.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:44.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:30:44.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:30:44.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:30:44.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:30:44.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:30:44.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:30:44.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:30:44.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:30:44.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:30:44.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:30:44.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:30:44.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.567 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.567 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:30:44.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:30:44.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.567 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.567 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:30:44.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:30:44.567 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.567 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.567 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.568 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:44.568 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:44.568 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.310728067, 2024-09-17T07:30:44.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:30:44.568 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.568 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:44.568 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:44.568 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:44.568 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:44.568 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:30:44.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.568 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:44.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:44.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:30:44.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:30:44.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:30:44.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:30:44.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:30:44.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:30:44.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.571 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:30:44.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:30:44.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.571 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:30:44.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:30:44.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:44.571 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:30:44.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:30:44.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:30:44.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:30:44.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:44.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:30:44.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:30:44.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:30:44.573 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:30:44.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.573 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.573 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:30:44.573 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:30:44.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.573 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.573 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:30:44.573 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:30:44.573 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.573 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.573 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.574 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.574 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.574 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:44.575 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:44.576 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:44.574+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:44.676 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:44.675+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:44.683 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:44.684 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:44.684 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:44.684 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:44.692 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:44.777 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:44.777+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:44.800 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:44.801 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:44.801 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:44.801 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:44.801 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:44.809 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:44.878 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:44.878+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:44.959 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:44.960 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:44.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:44.960 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:44.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.961 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17069 3.0 e41/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 41'110, mlcod=40'108)", 2024-09-17T07:30:44.961 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:44.961 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 102.217776234, 2024-09-17T07:30:44.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011408908000000001, 2024-09-17T07:30:44.961 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.962 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.962 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:44.962 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:44.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:44.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739753+0000", 2024-09-17T07:30:44.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:30:44.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.965 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743532+0000", 2024-09-17T07:30:44.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037794510000000001 2024-09-17T07:30:44.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.966 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.966 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743557+0000", 2024-09-17T07:30:44.966 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5253e-05 2024-09-17T07:30:44.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.967 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743569+0000", 2024-09-17T07:30:44.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1161999999999999e-05 2024-09-17T07:30:44.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.967 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.967 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743650+0000", 2024-09-17T07:30:44.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1087999999999993e-05 2024-09-17T07:30:44.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743674+0000", 2024-09-17T07:30:44.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4562e-05 2024-09-17T07:30:44.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:44.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751095+0000", 2024-09-17T07:30:44.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074211140000000004 2024-09-17T07:30:44.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751161+0000", 2024-09-17T07:30:44.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5707e-05 2024-09-17T07:30:44.969 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.969 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.969 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.970 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17240 3.0 e42/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 42'113, mlcod=41'110)", 2024-09-17T07:30:44.970 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:44.970 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 101.73211775, 2024-09-17T07:30:44.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064392440000000002, 2024-09-17T07:30:44.970 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.970 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.970 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:44.970 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:44.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.971 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:44.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.971 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:44.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:30:44.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.972 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225418+0000", 2024-09-17T07:30:44.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8360000000000003e-06 2024-09-17T07:30:44.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.972 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225426+0000", 2024-09-17T07:30:44.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2670000000000006e-06 2024-09-17T07:30:44.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.973 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.973 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225434+0000", 2024-09-17T07:30:44.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0560000000000005e-06 2024-09-17T07:30:44.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.973 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.973 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225514+0000", 2024-09-17T07:30:44.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9979000000000001e-05 2024-09-17T07:30:44.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.973 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.973 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225535+0000", 2024-09-17T07:30:44.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0965e-05 2024-09-17T07:30:44.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:44.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231797+0000", 2024-09-17T07:30:44.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062621129999999997 2024-09-17T07:30:44.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.974 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.974 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231850+0000", 2024-09-17T07:30:44.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2611999999999997e-05 2024-09-17T07:30:44.974 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.975 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.975 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.975 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:32543 3.0 e55/23 3:03bfd97d:::rbd_data.11839a97b753.0000000000000007:head v 55'150, mlcod=54'148)", 2024-09-17T07:30:44.975 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:44.975 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.502891496999993, 2024-09-17T07:30:44.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031813480000000001, 2024-09-17T07:30:44.975 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:44.975 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.975 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:44.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:44.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:44.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5199999999999998e-07 2024-09-17T07:30:44.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455083+0000", 2024-09-17T07:30:44.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044629199999999999 2024-09-17T07:30:44.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455098+0000", 2024-09-17T07:30:44.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4183e-05 2024-09-17T07:30:44.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455106+0000", 2024-09-17T07:30:44.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:30:44.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455173+0000", 2024-09-17T07:30:44.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6346000000000005e-05 2024-09-17T07:30:44.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455200+0000", 2024-09-17T07:30:44.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7319000000000001e-05 2024-09-17T07:30:44.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:44.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457774+0000", 2024-09-17T07:30:44.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025742650000000001 2024-09-17T07:30:44.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457818+0000", 2024-09-17T07:30:44.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3891999999999999e-05 2024-09-17T07:30:44.979 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.979 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.979 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.979 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33735 3.1f e55/52 3:fea54097:::rbd_data.11839a97b753.0000000000000000:head v 55'260, mlcod=55'258)", 2024-09-17T07:30:44.979 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:44.980 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.132958869000007, 2024-09-17T07:30:44.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035215200000000002, 2024-09-17T07:30:44.980 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.980 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.980 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:44.980 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:44.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:44.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824571+0000", 2024-09-17T07:30:44.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.164e-06 2024-09-17T07:30:44.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825118+0000", 2024-09-17T07:30:44.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054696400000000004 2024-09-17T07:30:44.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825134+0000", 2024-09-17T07:30:44.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6342999999999999e-05 2024-09-17T07:30:44.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825142+0000", 2024-09-17T07:30:44.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5270000000000001e-06 2024-09-17T07:30:44.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825200+0000", 2024-09-17T07:30:44.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.893e-05 2024-09-17T07:30:44.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825218+0000", 2024-09-17T07:30:44.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7935999999999999e-05 2024-09-17T07:30:44.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.984 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:44.984 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828037+0000", 2024-09-17T07:30:44.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002818961 2024-09-17T07:30:44.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.984 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.984 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828091+0000", 2024-09-17T07:30:44.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3695e-05 2024-09-17T07:30:44.985 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.985 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.985 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.985 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.985 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33869 3.9 e55/23 3:907b81b5:::rbd_data.11839a97b753.0000000000000001:head v 55'171, mlcod=55'169)", 2024-09-17T07:30:44.985 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:44.985 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 87.115903046, 2024-09-17T07:30:44.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033990859999999999, 2024-09-17T07:30:44.985 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.986 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:44.986 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:44.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:44.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841626+0000", 2024-09-17T07:30:44.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.023e-06 2024-09-17T07:30:44.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842067+0000", 2024-09-17T07:30:44.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000440612 2024-09-17T07:30:44.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842080+0000", 2024-09-17T07:30:44.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.29e-05 2024-09-17T07:30:44.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842086+0000", 2024-09-17T07:30:44.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2090000000000002e-06 2024-09-17T07:30:44.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842144+0000", 2024-09-17T07:30:44.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7627999999999998e-05 2024-09-17T07:30:44.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842171+0000", 2024-09-17T07:30:44.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7214000000000001e-05 2024-09-17T07:30:44.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:44.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.844973+0000", 2024-09-17T07:30:44.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002802232 2024-09-17T07:30:44.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.845024+0000", 2024-09-17T07:30:44.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1267999999999998e-05 2024-09-17T07:30:44.991 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.991 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.991 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.991 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51432 3.0 e59/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 59'177, mlcod=58'174)", 2024-09-17T07:30:44.991 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:44.991 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 82.136306028999996, 2024-09-17T07:30:44.991 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010790086000000001, 2024-09-17T07:30:44.991 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.991 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.991 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:44.991 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:44.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.992 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:44.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.992 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.993 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821224+0000", 2024-09-17T07:30:44.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.511e-06 2024-09-17T07:30:44.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.993 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825174+0000", 2024-09-17T07:30:44.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0039497949999999999 2024-09-17T07:30:44.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.993 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825196+0000", 2024-09-17T07:30:44.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2643999999999999e-05 2024-09-17T07:30:44.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825206+0000", 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4320000000000005e-06 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825272+0000", 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5829999999999998e-05 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825299+0000", 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7107e-05 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.831939+0000", 2024-09-17T07:30:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0066398510000000004 2024-09-17T07:30:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.832012+0000", 2024-09-17T07:30:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3916000000000002e-05 2024-09-17T07:30:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51969 3.0 e59/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 59'179, mlcod=59'177)", 2024-09-17T07:30:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 82.027437651, 2024-09-17T07:30:44.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060297249999999997, 2024-09-17T07:30:44.996 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:44.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:44.996 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:44.996 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:44.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:44.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:44.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:44.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:44.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:44.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.997 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:44.997 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930093+0000", 2024-09-17T07:30:44.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.764e-06 2024-09-17T07:30:44.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.997 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:44.997 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931779+0000", 2024-09-17T07:30:44.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001686379 2024-09-17T07:30:44.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.997 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.997 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:44.997 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931802+0000", 2024-09-17T07:30:44.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2682000000000001e-05 2024-09-17T07:30:44.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:44.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931811+0000", 2024-09-17T07:30:44.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5640000000000002e-06 2024-09-17T07:30:44.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:44.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931877+0000", 2024-09-17T07:30:44.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6155999999999995e-05 2024-09-17T07:30:44.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:44.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931896+0000", 2024-09-17T07:30:44.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9111e-05 2024-09-17T07:30:44.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:44.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936063+0000", 2024-09-17T07:30:44.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004166711 2024-09-17T07:30:44.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:44.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:44.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:44.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936121+0000", 2024-09-17T07:30:44.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7358000000000002e-05 2024-09-17T07:30:44.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:44.999 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:44.999 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.000 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54417 3.0 e60/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 60'182, mlcod=59'179)", 2024-09-17T07:30:45.000 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:45.000 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 81.519920659999997, 2024-09-17T07:30:45.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0059885219999999996, 2024-09-17T07:30:45.000 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:45.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:45.000 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:45.000 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:45.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:45.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:45.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:45.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:45.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:45.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437609+0000", 2024-09-17T07:30:45.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5829999999999999e-06 2024-09-17T07:30:45.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:45.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437623+0000", 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3348e-05 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437643+0000", 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0463999999999999e-05 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437653+0000", 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6290000000000005e-06 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437719+0000", 2024-09-17T07:30:45.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6428999999999997e-05 2024-09-17T07:30:45.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.003 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:45.003 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437745+0000", 2024-09-17T07:30:45.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5690999999999998e-05 2024-09-17T07:30:45.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.003 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:45.003 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443549+0000", 2024-09-17T07:30:45.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0058038259999999998 2024-09-17T07:30:45.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.003 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:45.003 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443596+0000", 2024-09-17T07:30:45.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7552000000000001e-05 2024-09-17T07:30:45.004 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.004 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:45.004 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.004 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54419 3.0 e60/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 60'184, mlcod=59'179)", 2024-09-17T07:30:45.004 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:45.004 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 81.513568303, 2024-09-17T07:30:45.004 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004706844, 2024-09-17T07:30:45.004 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:45.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:45.004 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:45.004 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:45.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.004 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:45.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:45.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:45.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:45.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:45.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443961+0000", 2024-09-17T07:30:45.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:30:45.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:45.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443967+0000", 2024-09-17T07:30:45.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.268e-06 2024-09-17T07:30:45.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:45.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443983+0000", 2024-09-17T07:30:45.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5248999999999999e-05 2024-09-17T07:30:45.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443990+0000", 2024-09-17T07:30:45.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5120000000000002e-06 2024-09-17T07:30:45.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:45.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444056+0000", 2024-09-17T07:30:45.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6197000000000003e-05 2024-09-17T07:30:45.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:45.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444081+0000", 2024-09-17T07:30:45.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4791e-05 2024-09-17T07:30:45.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:45.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448616+0000", 2024-09-17T07:30:45.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045346129999999998 2024-09-17T07:30:45.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:45.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448667+0000", 2024-09-17T07:30:45.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-17T07:30:45.008 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.008 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:45.008 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.008 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:80794 3.9 e72/23 3:90311a61:::rbd_data.1183cb746bb8.0000000000000007:head v 72'222, mlcod=71'220)", 2024-09-17T07:30:45.008 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:45.008 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 67.525579418999996, 2024-09-17T07:30:45.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004371733, 2024-09-17T07:30:45.008 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:45.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:45.008 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:45.009 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:45.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:45.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:45.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:45.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:45.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.010 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:45.010 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:45.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0800000000000001e-07 2024-09-17T07:30:45.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.010 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:45.010 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434108+0000", 2024-09-17T07:30:45.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021587569999999999 2024-09-17T07:30:45.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.010 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:45.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434120+0000", 2024-09-17T07:30:45.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1821000000000001e-05 2024-09-17T07:30:45.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434128+0000", 2024-09-17T07:30:45.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7579999999999992e-06 2024-09-17T07:30:45.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:45.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434191+0000", 2024-09-17T07:30:45.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3202999999999998e-05 2024-09-17T07:30:45.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434218+0000", 2024-09-17T07:30:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6771000000000002e-05 2024-09-17T07:30:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436289+0000", 2024-09-17T07:30:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020717700000000001 2024-09-17T07:30:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436321+0000", 2024-09-17T07:30:45.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1445000000000001e-05 2024-09-17T07:30:45.013 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.013 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:45.013 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.013 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 75/74 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head, version: 36'126, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head@36'126, size: 9431460, copy_subset: [], clone_subset: {3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9: [(0, 9431460)]}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:45.013 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:45.013 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 63.216671826000002, 2024-09-17T07:30:45.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041288289999999997, 2024-09-17T07:30:45.013 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:45.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:45.013 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:45.013 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:45.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:45.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:45.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:45.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:45.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:45.015 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740858+0000", 2024-09-17T07:30:45.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1599999999999999e-06 2024-09-17T07:30:45.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.015 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:45.015 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740885+0000", 2024-09-17T07:30:45.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.707e-05 2024-09-17T07:30:45.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.015 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:45.015 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740925+0000", 2024-09-17T07:30:45.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0649000000000003e-05 2024-09-17T07:30:45.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740957+0000", 2024-09-17T07:30:45.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1585999999999999e-05 2024-09-17T07:30:45.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:45.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741185+0000", 2024-09-17T07:30:45.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022759700000000001 2024-09-17T07:30:45.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:45.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741228+0000", 2024-09-17T07:30:45.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3705999999999998e-05 2024-09-17T07:30:45.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:45.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.744985+0000", 2024-09-17T07:30:45.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037570609999999999 2024-09-17T07:30:45.017 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.017 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:45.017 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.017 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85061 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'257, mlcod=79'255)", 2024-09-17T07:30:45.017 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:45.017 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 59.808840869999997, 2024-09-17T07:30:45.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033850550000000001, 2024-09-17T07:30:45.018 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:45.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:45.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:45.018 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:45.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:45.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:45.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:45.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:45.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:45.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:45.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9699999999999995e-07 2024-09-17T07:30:45.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:45.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148835+0000", 2024-09-17T07:30:45.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000146852 2024-09-17T07:30:45.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:45.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148845+0000", 2024-09-17T07:30:45.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0232e-05 2024-09-17T07:30:45.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148852+0000", 2024-09-17T07:30:45.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6490000000000002e-06 2024-09-17T07:30:45.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.021 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:45.021 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148913+0000", 2024-09-17T07:30:45.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1099999999999994e-05 2024-09-17T07:30:45.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:45.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148931+0000", 2024-09-17T07:30:45.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7938e-05 2024-09-17T07:30:45.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:45.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152025+0000", 2024-09-17T07:30:45.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0030942729999999998 2024-09-17T07:30:45.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:45.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152073+0000", 2024-09-17T07:30:45.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7213999999999999e-05 2024-09-17T07:30:45.023 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.023 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:45.023 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.023 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85063 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'259, mlcod=79'255)", 2024-09-17T07:30:45.023 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:45.023 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 59.807693915999998, 2024-09-17T07:30:45.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037722659999999998, 2024-09-17T07:30:45.024 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:45.024 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:45.024 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:45.024 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:45.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:45.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:45.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:45.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:45.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:45.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:45.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5799999999999999e-07 2024-09-17T07:30:45.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:45.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149841+0000", 2024-09-17T07:30:45.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4840000000000003e-06 2024-09-17T07:30:45.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.026 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:45.026 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149849+0000", 2024-09-17T07:30:45.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4670000000000004e-06 2024-09-17T07:30:45.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.026 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.026 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149854+0000", 2024-09-17T07:30:45.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4779999999999999e-06 2024-09-17T07:30:45.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:45.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149920+0000", 2024-09-17T07:30:45.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5760999999999996e-05 2024-09-17T07:30:45.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:45.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149949+0000", 2024-09-17T07:30:45.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8838000000000001e-05 2024-09-17T07:30:45.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:45.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153558+0000", 2024-09-17T07:30:45.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036092939999999999 2024-09-17T07:30:45.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:45.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153607+0000", 2024-09-17T07:30:45.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8386000000000001e-05 2024-09-17T07:30:45.028 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.028 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:45.028 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.029 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:94996 3.0 e82/23 3:074b4f56:::rbd_data.118311e0d6f8.0000000000000001:head v 82'715, mlcod=80'713)", 2024-09-17T07:30:45.029 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:45.029 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.341788323000003, 2024-09-17T07:30:45.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031499179999999998, 2024-09-17T07:30:45.029 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:45.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:45.029 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:45.029 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:45.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.030 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:45.030 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:45.030 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.030 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:45.030 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:45.030 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.030 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:45.030 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615741+0000", 2024-09-17T07:30:45.030 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999996e-07 2024-09-17T07:30:45.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.031 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:45.031 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616917+0000", 2024-09-17T07:30:45.031 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011763310000000001 2024-09-17T07:30:45.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.031 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:45.031 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616936+0000", 2024-09-17T07:30:45.031 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8624e-05 2024-09-17T07:30:45.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.032 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.032 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616948+0000", 2024-09-17T07:30:45.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.238e-05 2024-09-17T07:30:45.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:45.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617054+0000", 2024-09-17T07:30:45.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000106556 2024-09-17T07:30:45.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.033 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:45.033 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617081+0000", 2024-09-17T07:30:45.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.622e-05 2024-09-17T07:30:45.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:45.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618836+0000", 2024-09-17T07:30:45.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001755412 2024-09-17T07:30:45.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:45.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618890+0000", 2024-09-17T07:30:45.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3915000000000001e-05 2024-09-17T07:30:45.034 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.034 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:45.034 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.035 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:95007 3.5 e82/76 3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head v 82'422, mlcod=82'420)", 2024-09-17T07:30:45.035 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:45.035 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 55.333865621999998, 2024-09-17T07:30:45.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003939901, 2024-09-17T07:30:45.035 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:45.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:45.035 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:45.035 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:45.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.035 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:45.035 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:45.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:45.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:45.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:45.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623664+0000", 2024-09-17T07:30:45.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3799999999999999e-06 2024-09-17T07:30:45.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:45.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626104+0000", 2024-09-17T07:30:45.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002440149 2024-09-17T07:30:45.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:45.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626120+0000", 2024-09-17T07:30:45.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5962000000000001e-05 2024-09-17T07:30:45.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626127+0000", 2024-09-17T07:30:45.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8340000000000001e-06 2024-09-17T07:30:45.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:45.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626186+0000", 2024-09-17T07:30:45.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9234999999999998e-05 2024-09-17T07:30:45.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:45.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626208+0000", 2024-09-17T07:30:45.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1719e-05 2024-09-17T07:30:45.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:45.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627567+0000", 2024-09-17T07:30:45.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013587919999999999 2024-09-17T07:30:45.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:45.039 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627603+0000", 2024-09-17T07:30:45.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5830000000000001e-05 2024-09-17T07:30:45.039 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.039 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:45.039 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.039 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:104928 3.18 e87/76 3:1ef990b1:::rbd_data.1183dd51323.0000000000000006:head v 87'562, mlcod=86'559)", 2024-09-17T07:30:45.039 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:45.039 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 49.415109911000002, 2024-09-17T07:30:45.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0061095869999999997, 2024-09-17T07:30:45.040 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:45.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:45.040 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:45.040 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:45.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.041 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:45.041 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:45.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.041 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:45.041 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:45.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:45.042 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542421+0000", 2024-09-17T07:30:45.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.069e-06 2024-09-17T07:30:45.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544190+0000", 2024-09-17T07:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017693539999999999 2024-09-17T07:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:45.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544215+0000", 2024-09-17T07:30:45.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5398999999999999e-05 2024-09-17T07:30:45.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.044 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544229+0000", 2024-09-17T07:30:45.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3865e-05 2024-09-17T07:30:45.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.045 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:45.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544322+0000", 2024-09-17T07:30:45.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2540000000000005e-05 2024-09-17T07:30:45.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:45.046 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544348+0000", 2024-09-17T07:30:45.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6197999999999999e-05 2024-09-17T07:30:45.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:45.047 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548475+0000", 2024-09-17T07:30:45.047 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041266000000000002 2024-09-17T07:30:45.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.047 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.047 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:45.047 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548528+0000", 2024-09-17T07:30:45.047 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3562e-05 2024-09-17T07:30:45.048 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.048 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:45.048 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.048 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115898 3.9 3:910101b0:::rbd_data.118340e4fbf.0000000000000000:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:45.048 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:30:45.048 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.766595806000002, 2024-09-17T07:30:45.048 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0042466279999999997, 2024-09-17T07:30:45.048 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:45.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:45.048 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:45.049 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:45.049 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:45.049 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.049 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115898 2024-09-17T07:30:45.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.049 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:45.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.049 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:45.049 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:30:45.049 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.049 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:45.049 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:30:45.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:45.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:45.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0030000000000001e-06 2024-09-17T07:30:45.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:45.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190940+0000", 2024-09-17T07:30:45.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2349999999999999e-06 2024-09-17T07:30:45.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:45.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190953+0000", 2024-09-17T07:30:45.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3304e-05 2024-09-17T07:30:45.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190995+0000", 2024-09-17T07:30:45.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2317000000000002e-05 2024-09-17T07:30:45.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:45.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191592+0000", 2024-09-17T07:30:45.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00059607799999999999 2024-09-17T07:30:45.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:45.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191888+0000", 2024-09-17T07:30:45.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00029619999999999999 2024-09-17T07:30:45.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:45.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192069+0000", 2024-09-17T07:30:45.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00018125899999999999 2024-09-17T07:30:45.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:45.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193692+0000", 2024-09-17T07:30:45.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001623364 2024-09-17T07:30:45.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.053 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:45.053 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194968+0000", 2024-09-17T07:30:45.053 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012755189999999999 2024-09-17T07:30:45.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.053 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:45.053 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.195037+0000", 2024-09-17T07:30:45.053 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9450999999999996e-05 2024-09-17T07:30:45.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.054 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:45.054 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.195179+0000", 2024-09-17T07:30:45.054 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000141898 2024-09-17T07:30:45.054 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.054 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:45.054 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.054 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115936 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [watch watch cookie 140227662410368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:45.054 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:45.054 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.733676853999999, 2024-09-17T07:30:45.054 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043930569999999997, 2024-09-17T07:30:45.054 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:45.055 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:45.055 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:45.055 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:45.055 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:45.055 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.055 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115936 2024-09-17T07:30:45.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.055 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:45.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.055 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:45.055 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:45.055 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:45.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:45.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:45.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:45.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7300000000000005e-07 2024-09-17T07:30:45.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:45.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223856+0000", 2024-09-17T07:30:45.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9169999999999999e-06 2024-09-17T07:30:45.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:45.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223870+0000", 2024-09-17T07:30:45.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3926e-05 2024-09-17T07:30:45.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223878+0000", 2024-09-17T07:30:45.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3699999999999997e-06 2024-09-17T07:30:45.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:45.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223934+0000", 2024-09-17T07:30:45.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6929000000000003e-05 2024-09-17T07:30:45.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:45.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224053+0000", 2024-09-17T07:30:45.058 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000118952 2024-09-17T07:30:45.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:45.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224271+0000", 2024-09-17T07:30:45.058 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021763300000000001 2024-09-17T07:30:45.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.058 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:45.058 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224992+0000", 2024-09-17T07:30:45.058 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072142099999999998 2024-09-17T07:30:45.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:45.059 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228099+0000", 2024-09-17T07:30:45.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031060850000000002 2024-09-17T07:30:45.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:45.059 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228143+0000", 2024-09-17T07:30:45.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4614999999999999e-05 2024-09-17T07:30:45.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:45.059 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228245+0000", 2024-09-17T07:30:45.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000101436 2024-09-17T07:30:45.060 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.060 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:45.060 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.060 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115955 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [call rbd.snapshot_trash_add in=21b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:45.060 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:30:45.060 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.714690100999999, 2024-09-17T07:30:45.060 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004259054, 2024-09-17T07:30:45.060 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:45.060 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:45.060 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:45.060 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:45.061 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:45.061 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.061 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115955 2024-09-17T07:30:45.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.061 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:45.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.061 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:45.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:30:45.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.061 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:45.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:30:45.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:45.062 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242839+0000", 2024-09-17T07:30:45.062 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6899999999999998e-07 2024-09-17T07:30:45.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:45.062 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242842+0000", 2024-09-17T07:30:45.062 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1350000000000001e-06 2024-09-17T07:30:45.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:45.062 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242849+0000", 2024-09-17T07:30:45.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0400000000000004e-06 2024-09-17T07:30:45.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.063 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242854+0000", 2024-09-17T07:30:45.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4940000000000003e-06 2024-09-17T07:30:45.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.063 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:45.063 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243137+0000", 2024-09-17T07:30:45.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00028217100000000001 2024-09-17T07:30:45.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.063 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:45.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243196+0000", 2024-09-17T07:30:45.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9608999999999998e-05 2024-09-17T07:30:45.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:45.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243448+0000", 2024-09-17T07:30:45.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00025185000000000002 2024-09-17T07:30:45.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:45.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243990+0000", 2024-09-17T07:30:45.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054153199999999995 2024-09-17T07:30:45.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.065 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:45.065 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.246867+0000", 2024-09-17T07:30:45.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028771880000000001 2024-09-17T07:30:45.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.065 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:45.065 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.246943+0000", 2024-09-17T07:30:45.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5689e-05 2024-09-17T07:30:45.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.065 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:45.065 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247097+0000", 2024-09-17T07:30:45.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015487699999999999 2024-09-17T07:30:45.066 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.066 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:45.066 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.066 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115956 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:45.066 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:30:45.066 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.714393517000001, 2024-09-17T07:30:45.066 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043741479999999996, 2024-09-17T07:30:45.066 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:45.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:45.066 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:45.066 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:45.066 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:45.067 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.067 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115956 2024-09-17T07:30:45.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.067 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:45.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.067 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:45.067 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:30:45.068 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.068 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:45.068 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:30:45.068 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:45.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.069 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:45.069 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243136+0000", 2024-09-17T07:30:45.069 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7100000000000001e-07 2024-09-17T07:30:45.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.069 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:45.069 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243141+0000", 2024-09-17T07:30:45.069 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.716e-06 2024-09-17T07:30:45.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.069 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:45.070 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243153+0000", 2024-09-17T07:30:45.070 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1198e-05 2024-09-17T07:30:45.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.070 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.070 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243160+0000", 2024-09-17T07:30:45.070 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6229999999999997e-06 2024-09-17T07:30:45.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.070 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:45.070 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243797+0000", 2024-09-17T07:30:45.070 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063716399999999996 2024-09-17T07:30:45.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.071 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for rw locks", 2024-09-17T07:30:45.071 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243852+0000", 2024-09-17T07:30:45.071 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4147999999999997e-05 2024-09-17T07:30:45.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.071 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:45.071 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247147+0000", 2024-09-17T07:30:45.071 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0032955940000000002 2024-09-17T07:30:45.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.071 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:45.071 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247271+0000", 2024-09-17T07:30:45.072 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012397800000000001 2024-09-17T07:30:45.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:45.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:45.072 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:45.072 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247509+0000", 2024-09-17T07:30:45.072 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000237956 2024-09-17T07:30:45.072 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.072 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:45.072 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.072 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:45.072 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:45.073 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:45.077 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:45.077 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:45.073+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:45.108 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T07:30:45.174 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:45.174+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:45.202 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:45.202 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:45.202 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:45.212 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:45.275 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:45.274+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:45.357 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:45.358 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:45.358 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:45.366 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:45.376 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:45.376+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:45.471 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:30525 3.1b 3:def7c8f0:::rbd_data.118381ffbbab.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T07:30:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.946553617999996, 2024-09-17T07:30:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55515850300000003, 2024-09-17T07:30:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30525 2024-09-17T07:30:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522300+0000", 2024-09-17T07:30:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:30:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522305+0000", 2024-09-17T07:30:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-17T07:30:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522318+0000", 2024-09-17T07:30:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3030999999999999e-05 2024-09-17T07:30:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522326+0000", 2024-09-17T07:30:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4200000000000001e-06 2024-09-17T07:30:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522385+0000", 2024-09-17T07:30:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9005000000000002e-05 2024-09-17T07:30:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522393+0000", 2024-09-17T07:30:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-06 2024-09-17T07:30:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075347+0000", 2024-09-17T07:30:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55295417800000002 2024-09-17T07:30:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075502+0000", 2024-09-17T07:30:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154272 2024-09-17T07:30:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075630+0000", 2024-09-17T07:30:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012816300000000001 2024-09-17T07:30:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077221+0000", 2024-09-17T07:30:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015912929999999999 2024-09-17T07:30:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077353+0000", 2024-09-17T07:30:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013215699999999999 2024-09-17T07:30:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077413+0000", 2024-09-17T07:30:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0034999999999997e-05 2024-09-17T07:30:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077457+0000", 2024-09-17T07:30:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4017999999999998e-05 2024-09-17T07:30:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44181 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13922304~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.831374969999999, 2024-09-17T07:30:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046010502000000002, 2024-09-17T07:30:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44181 2024-09-17T07:30:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637478+0000", 2024-09-17T07:30:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:30:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637480+0000", 2024-09-17T07:30:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.843e-06 2024-09-17T07:30:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637487+0000", 2024-09-17T07:30:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6159999999999996e-06 2024-09-17T07:30:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637492+0000", 2024-09-17T07:30:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5360000000000003e-06 2024-09-17T07:30:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683351+0000", 2024-09-17T07:30:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045858758999999999 2024-09-17T07:30:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683422+0000", 2024-09-17T07:30:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1705999999999994e-05 2024-09-17T07:30:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683488+0000", 2024-09-17T07:30:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5612999999999997e-05 2024-09-17T07:30:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44182 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13938688~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.831354298999997, 2024-09-17T07:30:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046153118999999999, 2024-09-17T07:30:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44182 2024-09-17T07:30:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:30:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637506+0000", 2024-09-17T07:30:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1520000000000003e-06 2024-09-17T07:30:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637512+0000", 2024-09-17T07:30:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-09-17T07:30:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637517+0000", 2024-09-17T07:30:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-09-17T07:30:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683514+0000", 2024-09-17T07:30:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045997616999999998 2024-09-17T07:30:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683584+0000", 2024-09-17T07:30:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9942000000000002e-05 2024-09-17T07:30:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683651+0000", 2024-09-17T07:30:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6951000000000003e-05 2024-09-17T07:30:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44183 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13955072~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.831329066999999, 2024-09-17T07:30:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046339793999999997, 2024-09-17T07:30:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44183 2024-09-17T07:30:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639583+0000", 2024-09-17T07:30:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002059412 2024-09-17T07:30:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639597+0000", 2024-09-17T07:30:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3832000000000001e-05 2024-09-17T07:30:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639607+0000", 2024-09-17T07:30:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485e-05 2024-09-17T07:30:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639614+0000", 2024-09-17T07:30:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0849999999999999e-06 2024-09-17T07:30:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683705+0000", 2024-09-17T07:30:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044091261 2024-09-17T07:30:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683792+0000", 2024-09-17T07:30:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6197000000000002e-05 2024-09-17T07:30:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683863+0000", 2024-09-17T07:30:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1521999999999997e-05 2024-09-17T07:30:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44199 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14217216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.827811514999993, 2024-09-17T07:30:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045995623999999999, 2024-09-17T07:30:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44199 2024-09-17T07:30:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:30:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641043+0000", 2024-09-17T07:30:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.624e-06 2024-09-17T07:30:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641048+0000", 2024-09-17T07:30:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4029999999999998e-06 2024-09-17T07:30:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641053+0000", 2024-09-17T07:30:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6670000000000004e-06 2024-09-17T07:30:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686918+0000", 2024-09-17T07:30:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045865131000000003 2024-09-17T07:30:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686986+0000", 2024-09-17T07:30:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8299000000000005e-05 2024-09-17T07:30:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687036+0000", 2024-09-17T07:30:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0164999999999998e-05 2024-09-17T07:30:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44200 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14233600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.827793686000007, 2024-09-17T07:30:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046183984999999997, 2024-09-17T07:30:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44200 2024-09-17T07:30:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641061+0000", 2024-09-17T07:30:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.767e-06 2024-09-17T07:30:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641067+0000", 2024-09-17T07:30:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7379999999999999e-06 2024-09-17T07:30:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641071+0000", 2024-09-17T07:30:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-09-17T07:30:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687078+0000", 2024-09-17T07:30:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046007095999999997 2024-09-17T07:30:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687158+0000", 2024-09-17T07:30:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0374e-05 2024-09-17T07:30:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687243+0000", 2024-09-17T07:30:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4221999999999994e-05 2024-09-17T07:30:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44201 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14249984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.827776040000003, 2024-09-17T07:30:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046303182999999998, 2024-09-17T07:30:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44201 2024-09-17T07:30:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641120+0000", 2024-09-17T07:30:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3933000000000001e-05 2024-09-17T07:30:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641123+0000", 2024-09-17T07:30:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.191e-06 2024-09-17T07:30:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641133+0000", 2024-09-17T07:30:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4020000000000008e-06 2024-09-17T07:30:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641138+0000", 2024-09-17T07:30:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6559999999999997e-06 2024-09-17T07:30:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687279+0000", 2024-09-17T07:30:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046140278999999999 2024-09-17T07:30:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687338+0000", 2024-09-17T07:30:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9268999999999998e-05 2024-09-17T07:30:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687379+0000", 2024-09-17T07:30:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1452999999999998e-05 2024-09-17T07:30:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44202 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14266368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.827705273999996, 2024-09-17T07:30:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046484299, 2024-09-17T07:30:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44202 2024-09-17T07:30:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642810+0000", 2024-09-17T07:30:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016627980000000001 2024-09-17T07:30:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642815+0000", 2024-09-17T07:30:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.091e-06 2024-09-17T07:30:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642829+0000", 2024-09-17T07:30:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4047e-05 2024-09-17T07:30:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642844+0000", 2024-09-17T07:30:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4515000000000001e-05 2024-09-17T07:30:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687426+0000", 2024-09-17T07:30:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044582201000000002 2024-09-17T07:30:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687529+0000", 2024-09-17T07:30:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103078 2024-09-17T07:30:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687631+0000", 2024-09-17T07:30:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102569 2024-09-17T07:30:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44209 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14381056~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.825807122000001, 2024-09-17T07:30:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045944444000000001, 2024-09-17T07:30:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44209 2024-09-17T07:30:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643046+0000", 2024-09-17T07:30:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:30:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643048+0000", 2024-09-17T07:30:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:30:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643054+0000", 2024-09-17T07:30:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.032e-06 2024-09-17T07:30:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643065+0000", 2024-09-17T07:30:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1861999999999999e-05 2024-09-17T07:30:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688806+0000", 2024-09-17T07:30:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045740617999999997 2024-09-17T07:30:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688927+0000", 2024-09-17T07:30:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000121327 2024-09-17T07:30:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688990+0000", 2024-09-17T07:30:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2293999999999997e-05 2024-09-17T07:30:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44210 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14397440~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.825780195999997, 2024-09-17T07:30:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046104635999999997, 2024-09-17T07:30:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44210 2024-09-17T07:30:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643073+0000", 2024-09-17T07:30:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.77e-07 2024-09-17T07:30:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643075+0000", 2024-09-17T07:30:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-09-17T07:30:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643082+0000", 2024-09-17T07:30:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7789999999999996e-06 2024-09-17T07:30:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643087+0000", 2024-09-17T07:30:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.967e-06 2024-09-17T07:30:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689022+0000", 2024-09-17T07:30:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045934349999999999 2024-09-17T07:30:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689112+0000", 2024-09-17T07:30:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0039999999999999e-05 2024-09-17T07:30:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689177+0000", 2024-09-17T07:30:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5041000000000003e-05 2024-09-17T07:30:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44211 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14413824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.825759063000007, 2024-09-17T07:30:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046309003000000001, 2024-09-17T07:30:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44211 2024-09-17T07:30:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643094+0000", 2024-09-17T07:30:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:30:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643095+0000", 2024-09-17T07:30:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-17T07:30:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643101+0000", 2024-09-17T07:30:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8730000000000004e-06 2024-09-17T07:30:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643106+0000", 2024-09-17T07:30:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6859999999999997e-06 2024-09-17T07:30:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689220+0000", 2024-09-17T07:30:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046113945000000003 2024-09-17T07:30:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689309+0000", 2024-09-17T07:30:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9560000000000003e-05 2024-09-17T07:30:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689402+0000", 2024-09-17T07:30:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2786000000000002e-05 2024-09-17T07:30:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44212 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14430208~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.825714500999993, 2024-09-17T07:30:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046448912000000002, 2024-09-17T07:30:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44212 2024-09-17T07:30:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2200000000000004e-07 2024-09-17T07:30:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643141+0000", 2024-09-17T07:30:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5210000000000001e-06 2024-09-17T07:30:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643149+0000", 2024-09-17T07:30:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6490000000000008e-06 2024-09-17T07:30:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643161+0000", 2024-09-17T07:30:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2241e-05 2024-09-17T07:30:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689423+0000", 2024-09-17T07:30:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046262323000000001 2024-09-17T07:30:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689505+0000", 2024-09-17T07:30:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2103999999999999e-05 2024-09-17T07:30:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689587+0000", 2024-09-17T07:30:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1452000000000006e-05 2024-09-17T07:30:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44213 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14446592~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.825683002000005, 2024-09-17T07:30:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046601780000000002, 2024-09-17T07:30:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44213 2024-09-17T07:30:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643170+0000", 2024-09-17T07:30:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-17T07:30:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643172+0000", 2024-09-17T07:30:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1660000000000001e-06 2024-09-17T07:30:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643180+0000", 2024-09-17T07:30:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-09-17T07:30:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643185+0000", 2024-09-17T07:30:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-09-17T07:30:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689622+0000", 2024-09-17T07:30:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046436742000000003 2024-09-17T07:30:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689697+0000", 2024-09-17T07:30:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5179999999999995e-05 2024-09-17T07:30:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689771+0000", 2024-09-17T07:30:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3928000000000001e-05 2024-09-17T07:30:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44214 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14462976~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.825659942000001, 2024-09-17T07:30:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046736038000000001, 2024-09-17T07:30:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44214 2024-09-17T07:30:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643193+0000", 2024-09-17T07:30:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:30:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643195+0000", 2024-09-17T07:30:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0640000000000001e-06 2024-09-17T07:30:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643201+0000", 2024-09-17T07:30:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2859999999999996e-06 2024-09-17T07:30:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643206+0000", 2024-09-17T07:30:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-17T07:30:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689814+0000", 2024-09-17T07:30:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046607742000000001 2024-09-17T07:30:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689886+0000", 2024-09-17T07:30:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2020999999999994e-05 2024-09-17T07:30:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689928+0000", 2024-09-17T07:30:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2771000000000001e-05 2024-09-17T07:30:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44215 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14479360~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.825616045000004, 2024-09-17T07:30:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046868543999999998, 2024-09-17T07:30:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44215 2024-09-17T07:30:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643237+0000", 2024-09-17T07:30:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:30:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643245+0000", 2024-09-17T07:30:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3270000000000001e-06 2024-09-17T07:30:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643254+0000", 2024-09-17T07:30:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6049999999999998e-06 2024-09-17T07:30:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643260+0000", 2024-09-17T07:30:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7849999999999997e-06 2024-09-17T07:30:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689961+0000", 2024-09-17T07:30:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046701196 2024-09-17T07:30:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690036+0000", 2024-09-17T07:30:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5153000000000004e-05 2024-09-17T07:30:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690105+0000", 2024-09-17T07:30:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8874000000000005e-05 2024-09-17T07:30:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44216 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14495744~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.825584332000005, 2024-09-17T07:30:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047003047999999999, 2024-09-17T07:30:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44216 2024-09-17T07:30:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643269+0000", 2024-09-17T07:30:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:30:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643271+0000", 2024-09-17T07:30:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-09-17T07:30:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643278+0000", 2024-09-17T07:30:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8850000000000001e-06 2024-09-17T07:30:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643282+0000", 2024-09-17T07:30:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5979999999999999e-06 2024-09-17T07:30:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690133+0000", 2024-09-17T07:30:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850470999999998 2024-09-17T07:30:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690205+0000", 2024-09-17T07:30:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2442999999999997e-05 2024-09-17T07:30:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690271+0000", 2024-09-17T07:30:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5874e-05 2024-09-17T07:30:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44217 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14512128~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.825563320000001, 2024-09-17T07:30:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047166485000000001, 2024-09-17T07:30:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44217 2024-09-17T07:30:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-17T07:30:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643291+0000", 2024-09-17T07:30:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.962e-06 2024-09-17T07:30:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643297+0000", 2024-09-17T07:30:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0700000000000003e-06 2024-09-17T07:30:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643302+0000", 2024-09-17T07:30:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:30:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690308+0000", 2024-09-17T07:30:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047005354999999999 2024-09-17T07:30:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690385+0000", 2024-09-17T07:30:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7541999999999997e-05 2024-09-17T07:30:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690455+0000", 2024-09-17T07:30:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0328999999999997e-05 2024-09-17T07:30:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44218 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14528512~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.825543643000003, 2024-09-17T07:30:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047321015000000001, 2024-09-17T07:30:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44218 2024-09-17T07:30:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-17T07:30:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643311+0000", 2024-09-17T07:30:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.933e-06 2024-09-17T07:30:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643318+0000", 2024-09-17T07:30:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3980000000000004e-06 2024-09-17T07:30:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643322+0000", 2024-09-17T07:30:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7609999999999998e-06 2024-09-17T07:30:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690484+0000", 2024-09-17T07:30:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047161428999999998 2024-09-17T07:30:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690564+0000", 2024-09-17T07:30:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9845000000000005e-05 2024-09-17T07:30:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690630+0000", 2024-09-17T07:30:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6021999999999999e-05 2024-09-17T07:30:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44219 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14544896~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.825523275999998, 2024-09-17T07:30:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048128708999999999, 2024-09-17T07:30:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44219 2024-09-17T07:30:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:30:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643337+0000", 2024-09-17T07:30:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.109e-06 2024-09-17T07:30:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643350+0000", 2024-09-17T07:30:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3046e-05 2024-09-17T07:30:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643356+0000", 2024-09-17T07:30:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8930000000000002e-06 2024-09-17T07:30:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690660+0000", 2024-09-17T07:30:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047304061000000001 2024-09-17T07:30:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691372+0000", 2024-09-17T07:30:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00071281299999999997 2024-09-17T07:30:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691458+0000", 2024-09-17T07:30:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5352000000000005e-05 2024-09-17T07:30:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44220 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14561280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.825489227999995, 2024-09-17T07:30:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048309849000000002, 2024-09-17T07:30:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44220 2024-09-17T07:30:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646869+0000", 2024-09-17T07:30:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003506267 2024-09-17T07:30:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646874+0000", 2024-09-17T07:30:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7500000000000003e-06 2024-09-17T07:30:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646887+0000", 2024-09-17T07:30:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3115999999999999e-05 2024-09-17T07:30:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646894+0000", 2024-09-17T07:30:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5289999999999997e-06 2024-09-17T07:30:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691547+0000", 2024-09-17T07:30:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044652743000000002 2024-09-17T07:30:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691622+0000", 2024-09-17T07:30:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5037999999999995e-05 2024-09-17T07:30:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691673+0000", 2024-09-17T07:30:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-17T07:30:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.564 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:45.565 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:45.566 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:45.565+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:45.666 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:45.666+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:45.709 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:45.710 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:45.710 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:45.710 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:45.718 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:45.767 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:45.767+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:45.821 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:45.821 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:45.821 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:45.821 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:45.821 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:45.829 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:45.876 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:45.875+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:45.936 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:45.936 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:45.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:45.936 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19985 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5275648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.681604341, 2024-09-17T07:30:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071819998999999995, 2024-09-17T07:30:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.937 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19985 2024-09-17T07:30:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.938 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252493+0000", 2024-09-17T07:30:45.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6300000000000004e-07 2024-09-17T07:30:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252496+0000", 2024-09-17T07:30:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-17T07:30:45.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252499+0000", 2024-09-17T07:30:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:30:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324036+0000", 2024-09-17T07:30:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071536587999999998 2024-09-17T07:30:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324118+0000", 2024-09-17T07:30:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2336999999999995e-05 2024-09-17T07:30:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324312+0000", 2024-09-17T07:30:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000193709 2024-09-17T07:30:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.944 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19986 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5505024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.681593189, 2024-09-17T07:30:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072083761999999996, 2024-09-17T07:30:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.945 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19986 2024-09-17T07:30:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:30:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252508+0000", 2024-09-17T07:30:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:30:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252511+0000", 2024-09-17T07:30:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.145e-06 2024-09-17T07:30:45.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252514+0000", 2024-09-17T07:30:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5569999999999998e-06 2024-09-17T07:30:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324326+0000", 2024-09-17T07:30:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071812681000000003 2024-09-17T07:30:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324372+0000", 2024-09-17T07:30:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5893000000000003e-05 2024-09-17T07:30:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324587+0000", 2024-09-17T07:30:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214608 2024-09-17T07:30:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19987 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5734400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:45.949 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.681579571, 2024-09-17T07:30:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072276676999999998, 2024-09-17T07:30:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19987 2024-09-17T07:30:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252518+0000", 2024-09-17T07:30:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-17T07:30:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252522+0000", 2024-09-17T07:30:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2440000000000004e-06 2024-09-17T07:30:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252525+0000", 2024-09-17T07:30:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5919999999999999e-06 2024-09-17T07:30:45.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324621+0000", 2024-09-17T07:30:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072096442999999996 2024-09-17T07:30:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324699+0000", 2024-09-17T07:30:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8061999999999999e-05 2024-09-17T07:30:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324793+0000", 2024-09-17T07:30:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4207000000000001e-05 2024-09-17T07:30:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19988 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.68156860000001, 2024-09-17T07:30:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072585798000000007, 2024-09-17T07:30:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.954 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19988 2024-09-17T07:30:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252529+0000", 2024-09-17T07:30:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:30:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252532+0000", 2024-09-17T07:30:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9579999999999999e-06 2024-09-17T07:30:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252534+0000", 2024-09-17T07:30:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5730000000000002e-06 2024-09-17T07:30:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324825+0000", 2024-09-17T07:30:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072290757999999997 2024-09-17T07:30:45.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324874+0000", 2024-09-17T07:30:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9360000000000002e-05 2024-09-17T07:30:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325113+0000", 2024-09-17T07:30:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000239025 2024-09-17T07:30:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19989 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.681558979, 2024-09-17T07:30:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072963494000000004, 2024-09-17T07:30:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19989 2024-09-17T07:30:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:30:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:30:45.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:30:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2900000000000002e-07 2024-09-17T07:30:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252542+0000", 2024-09-17T07:30:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8600000000000003e-06 2024-09-17T07:30:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252545+0000", 2024-09-17T07:30:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.605e-06 2024-09-17T07:30:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325144+0000", 2024-09-17T07:30:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072599460000000005 2024-09-17T07:30:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325223+0000", 2024-09-17T07:30:45.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8473999999999994e-05 2024-09-17T07:30:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325501+0000", 2024-09-17T07:30:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000278063 2024-09-17T07:30:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19990 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6422528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.68154850800001, 2024-09-17T07:30:45.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073241127000000003, 2024-09-17T07:30:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19990 2024-09-17T07:30:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:45.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252552+0000", 2024-09-17T07:30:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4719999999999997e-06 2024-09-17T07:30:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252557+0000", 2024-09-17T07:30:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3039999999999998e-06 2024-09-17T07:30:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252560+0000", 2024-09-17T07:30:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8830000000000002e-06 2024-09-17T07:30:45.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325529+0000", 2024-09-17T07:30:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072968880999999999 2024-09-17T07:30:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325574+0000", 2024-09-17T07:30:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466000000000001e-05 2024-09-17T07:30:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325789+0000", 2024-09-17T07:30:45.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021490999999999999 2024-09-17T07:30:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19991 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6651904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.68152920599999, 2024-09-17T07:30:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073720552999999994, 2024-09-17T07:30:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19991 2024-09-17T07:30:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4699999999999998e-07 2024-09-17T07:30:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252568+0000", 2024-09-17T07:30:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5300000000000002e-07 2024-09-17T07:30:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252572+0000", 2024-09-17T07:30:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9160000000000003e-06 2024-09-17T07:30:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252575+0000", 2024-09-17T07:30:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:30:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325820+0000", 2024-09-17T07:30:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073245597999999995 2024-09-17T07:30:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325901+0000", 2024-09-17T07:30:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0913000000000003e-05 2024-09-17T07:30:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326288+0000", 2024-09-17T07:30:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038646900000000002 2024-09-17T07:30:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19992 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6881280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.681518853, 2024-09-17T07:30:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074054683999999996, 2024-09-17T07:30:45.973 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19992 2024-09-17T07:30:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:45.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:45.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252578+0000", 2024-09-17T07:30:45.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:30:45.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252579+0000", 2024-09-17T07:30:45.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2500000000000004e-07 2024-09-17T07:30:45.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252582+0000", 2024-09-17T07:30:45.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-17T07:30:45.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252585+0000", 2024-09-17T07:30:45.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.542e-06 2024-09-17T07:30:45.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326315+0000", 2024-09-17T07:30:45.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073729921000000004 2024-09-17T07:30:45.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326359+0000", 2024-09-17T07:30:45.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4095999999999999e-05 2024-09-17T07:30:45.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326632+0000", 2024-09-17T07:30:45.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027314499999999997 2024-09-17T07:30:45.978 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.978 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.978 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.978 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19993 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7110656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:45.979 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:45.979 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.681508549, 2024-09-17T07:30:45.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074425142999999999, 2024-09-17T07:30:45.979 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.979 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.979 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.979 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.979 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.979 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.979 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19993 2024-09-17T07:30:45.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.979 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:45.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:45.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:45.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:30:45.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252589+0000", 2024-09-17T07:30:45.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.33e-07 2024-09-17T07:30:45.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252593+0000", 2024-09-17T07:30:45.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:30:45.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252595+0000", 2024-09-17T07:30:45.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.621e-06 2024-09-17T07:30:45.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326666+0000", 2024-09-17T07:30:45.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074070787999999999 2024-09-17T07:30:45.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326752+0000", 2024-09-17T07:30:45.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5748000000000006e-05 2024-09-17T07:30:45.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327013+0000", 2024-09-17T07:30:45.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026098400000000002 2024-09-17T07:30:45.983 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.983 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.983 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.983 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19994 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7340032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:45.983 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:45.983 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.68149809000001, 2024-09-17T07:30:45.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074765920999999999, 2024-09-17T07:30:45.983 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.983 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.984 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.984 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.984 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.984 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.984 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19994 2024-09-17T07:30:45.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.984 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:45.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:45.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:45.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:30:45.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252599+0000", 2024-09-17T07:30:45.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2099999999999995e-07 2024-09-17T07:30:45.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252603+0000", 2024-09-17T07:30:45.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:30:45.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252606+0000", 2024-09-17T07:30:45.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-09-17T07:30:45.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327047+0000", 2024-09-17T07:30:45.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074441050999999994 2024-09-17T07:30:45.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327121+0000", 2024-09-17T07:30:45.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4283999999999996e-05 2024-09-17T07:30:45.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327364+0000", 2024-09-17T07:30:45.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000243155 2024-09-17T07:30:45.988 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.988 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.988 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.989 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19995 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7569408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:45.989 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:45.989 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.681487393, 2024-09-17T07:30:45.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075172694999999998, 2024-09-17T07:30:45.989 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.989 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.989 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.989 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19995 2024-09-17T07:30:45.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.990 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:45.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:45.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:45.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:30:45.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252610+0000", 2024-09-17T07:30:45.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6799999999999999e-07 2024-09-17T07:30:45.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252614+0000", 2024-09-17T07:30:45.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000002e-06 2024-09-17T07:30:45.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252616+0000", 2024-09-17T07:30:45.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.419e-06 2024-09-17T07:30:45.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327407+0000", 2024-09-17T07:30:45.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074790617000000004 2024-09-17T07:30:45.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327498+0000", 2024-09-17T07:30:45.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0758000000000001e-05 2024-09-17T07:30:45.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327782+0000", 2024-09-17T07:30:45.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028395599999999998 2024-09-17T07:30:45.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.993 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.993 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7798784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:45.994 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:45.994 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.681477077, 2024-09-17T07:30:45.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075605832999999997, 2024-09-17T07:30:45.994 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.994 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.994 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.994 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:30:45.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.994 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:45.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:45.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:45.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:45.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:45.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:45.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:45.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:30:45.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:45.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:30:45.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:30:45.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:45.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:30:45.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:30:45.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:45.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:30:45.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:30:45.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:45.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327844+0000", 2024-09-17T07:30:45.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075217709999999993 2024-09-17T07:30:45.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:45.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327927+0000", 2024-09-17T07:30:45.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2628999999999998e-05 2024-09-17T07:30:45.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:45.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328225+0000", 2024-09-17T07:30:45.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029800600000000002 2024-09-17T07:30:45.998 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.998 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:45.998 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:45.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.998 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8028160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:45.999 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:45.999 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.681466724, 2024-09-17T07:30:45.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076061237000000004, 2024-09-17T07:30:45.999 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:45.999 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:45.999 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:45.999 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:45.999 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:45.999 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:45.999 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:30:45.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:45.999 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:45.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:45.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:46.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:46.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:46.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:46.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:30:46.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:30:46.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:30:46.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:30:46.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:30:46.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:30:46.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328268+0000", 2024-09-17T07:30:46.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075626323999999995 2024-09-17T07:30:46.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328352+0000", 2024-09-17T07:30:46.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3578999999999994e-05 2024-09-17T07:30:46.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328691+0000", 2024-09-17T07:30:46.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033915099999999999 2024-09-17T07:30:46.003 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.003 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.003 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.003 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8257536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:46.003 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:46.003 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.68145127299999, 2024-09-17T07:30:46.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076441903000000005, 2024-09-17T07:30:46.003 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.003 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.003 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.004 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.004 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:30:46.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.004 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:46.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:46.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:46.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:30:46.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:30:46.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:30:46.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:30:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:30:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:30:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:30:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328747+0000", 2024-09-17T07:30:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076094192000000005 2024-09-17T07:30:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328824+0000", 2024-09-17T07:30:46.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7096999999999997e-05 2024-09-17T07:30:46.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.329087+0000", 2024-09-17T07:30:46.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026249999999999998 2024-09-17T07:30:46.007 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.007 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.007 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.008 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7356416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:46.008 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:46.008 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.671335114, 2024-09-17T07:30:46.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072172616999999994, 2024-09-17T07:30:46.008 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.008 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.008 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.008 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.008 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.008 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.008 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:30:46.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.009 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:46.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:46.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:30:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:30:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:30:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:30:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:30:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:30:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:30:46.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:30:46.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334558+0000", 2024-09-17T07:30:46.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071780790999999997 2024-09-17T07:30:46.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334647+0000", 2024-09-17T07:30:46.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9448000000000001e-05 2024-09-17T07:30:46.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334934+0000", 2024-09-17T07:30:46.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028654599999999999 2024-09-17T07:30:46.012 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.012 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.012 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.012 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7585792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:46.012 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:46.012 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.671313402, 2024-09-17T07:30:46.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072606135000000002, 2024-09-17T07:30:46.012 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.013 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.013 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.013 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:30:46.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.013 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:46.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:46.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:46.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:30:46.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:30:46.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:30:46.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:30:46.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:30:46.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:30:46.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:30:46.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335116+0000", 2024-09-17T07:30:46.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072323897999999998 2024-09-17T07:30:46.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335166+0000", 2024-09-17T07:30:46.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9882000000000001e-05 2024-09-17T07:30:46.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335389+0000", 2024-09-17T07:30:46.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022334300000000001 2024-09-17T07:30:46.016 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.016 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.016 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.017 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7815168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:46.017 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:46.017 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.671301203, 2024-09-17T07:30:46.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073051230999999994, 2024-09-17T07:30:46.017 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.017 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.017 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.017 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.017 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.017 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.017 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:30:46.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.017 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:46.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:46.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:46.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:46.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:30:46.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:30:46.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:30:46.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:30:46.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:30:46.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:30:46.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335431+0000", 2024-09-17T07:30:46.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072627852000000007 2024-09-17T07:30:46.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335525+0000", 2024-09-17T07:30:46.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3708000000000005e-05 2024-09-17T07:30:46.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335846+0000", 2024-09-17T07:30:46.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032182000000000001 2024-09-17T07:30:46.021 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.021 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.021 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.021 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:46.021 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:46.021 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.671290345, 2024-09-17T07:30:46.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073284199999999994, 2024-09-17T07:30:46.022 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.022 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.022 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.022 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.022 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.022 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.022 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:30:46.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.022 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:46.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:46.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:46.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:30:46.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:30:46.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:30:46.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:30:46.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:30:46.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:30:46.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:30:46.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335863+0000", 2024-09-17T07:30:46.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073048921000000003 2024-09-17T07:30:46.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335904+0000", 2024-09-17T07:30:46.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1545000000000003e-05 2024-09-17T07:30:46.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336090+0000", 2024-09-17T07:30:46.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185787 2024-09-17T07:30:46.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.025 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.026 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8273920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:46.026 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:46.026 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 100.671276085, 2024-09-17T07:30:46.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073670478999999997, 2024-09-17T07:30:46.026 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.026 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.026 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.026 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:30:46.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.026 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:46.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:46.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:46.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:30:46.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:30:46.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:30:46.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:30:46.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:30:46.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:30:46.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:30:46.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336126+0000", 2024-09-17T07:30:46.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073297254000000006 2024-09-17T07:30:46.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336211+0000", 2024-09-17T07:30:46.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4728000000000006e-05 2024-09-17T07:30:46.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336491+0000", 2024-09-17T07:30:46.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027979399999999998 2024-09-17T07:30:46.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.032 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.032 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.032 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:116163 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-17T07:30:46.032 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:30:46.032 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.668941217, 2024-09-17T07:30:46.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8549867870000001, 2024-09-17T07:30:46.032 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.032 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.033 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.033 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.033 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.033 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.033 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116163 2024-09-17T07:30:46.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.033 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:30:46.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:30:46.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265157+0000", 2024-09-17T07:30:46.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-06 2024-09-17T07:30:46.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265168+0000", 2024-09-17T07:30:46.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0827e-05 2024-09-17T07:30:46.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265198+0000", 2024-09-17T07:30:46.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0383e-05 2024-09-17T07:30:46.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265212+0000", 2024-09-17T07:30:46.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3799e-05 2024-09-17T07:30:46.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265338+0000", 2024-09-17T07:30:46.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012576800000000001 2024-09-17T07:30:46.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:46.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265357+0000", 2024-09-17T07:30:46.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9417000000000001e-05 2024-09-17T07:30:46.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:27.083102+0000", 2024-09-17T07:30:46.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.81774456299999998 2024-09-17T07:30:46.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:46.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:27.083113+0000", 2024-09-17T07:30:46.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1154e-05 2024-09-17T07:30:46.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:28.119793+0000", 2024-09-17T07:30:46.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.036680365 2024-09-17T07:30:46.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:28.119943+0000", 2024-09-17T07:30:46.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014940999999999999 2024-09-17T07:30:46.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:28.120142+0000", 2024-09-17T07:30:46.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00019897099999999999 2024-09-17T07:30:46.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.038 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.038 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.038 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:46.040 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:46.041 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:46.039+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:46.141 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:46.140+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-17T07:30:46.189 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:30:46.190 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:46.190 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:46.190 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:46.200 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:46.244 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:46.244+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:46.311 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:46.311 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:46.311 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:46.311 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:46.311 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:46.319 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:46.323 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 5, 6, 7, 3, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-17T07:30:46.323 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:30:46.323 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-17T07:30:46.323 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T07:30:46.323 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T07:30:46.324 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-17T07:30:46.361 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:46.360+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:46.365 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:46.366 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:46.383 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:46.399 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:46.400 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:46.416 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:46.417 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:46.433 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:46.434 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:46.434 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "" 2024-09-17T07:30:46.434 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:46.443 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:46.442+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:46.450 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:46.450 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:46.464 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.464 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.55780993400001, 2024-09-17T07:30:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:30:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:30:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:30:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:30:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:30:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:30:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:30:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:30:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:30:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:30:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:30:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:30:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:30:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:30:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:30:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.468 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.55779125700001, 2024-09-17T07:30:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:30:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:30:46.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:30:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:30:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:30:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:30:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:30:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:30:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:30:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:30:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:30:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:30:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:30:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:30:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:30:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.473 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.557772904, 2024-09-17T07:30:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:30:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:30:46.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:30:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:30:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:30:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:30:46.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:30:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:30:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:30:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:30:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:30:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:30:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:30:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:30:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:30:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:46.478 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.557753306, 2024-09-17T07:30:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:30:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:30:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:46.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:30:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:30:46.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:30:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:30:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:30:46.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:30:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:30:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:30:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:30:46.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:30:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:30:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:30:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:30:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:30:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.485 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.557733351, 2024-09-17T07:30:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:30:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:30:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:46.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:46.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:30:46.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:30:46.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:30:46.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:30:46.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:30:46.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:30:46.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:30:46.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:30:46.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:30:46.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:30:46.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:30:46.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:30:46.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:30:46.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.493 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.493 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:46.493 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:46.493 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.55771469600001, 2024-09-17T07:30:46.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:30:46.494 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.494 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.494 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.494 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:30:46.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.494 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:46.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:46.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:46.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:30:46.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:30:46.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:30:46.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:30:46.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:30:46.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:30:46.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:30:46.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:30:46.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:30:46.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:30:46.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:30:46.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:30:46.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:30:46.498 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.498 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.498 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.499 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:46.499 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:46.499 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.55769622, 2024-09-17T07:30:46.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:30:46.499 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.499 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.499 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.499 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.499 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.500 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:30:46.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.500 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:46.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:46.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:30:46.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:30:46.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:30:46.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:30:46.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:30:46.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:30:46.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:30:46.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:30:46.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:30:46.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:30:46.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:30:46.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:30:46.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:30:46.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:30:46.506 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.506 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.506 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.506 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:46.506 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:46.506 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.557676621, 2024-09-17T07:30:46.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:30:46.507 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.507 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.507 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.507 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.507 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.507 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.507 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:30:46.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.508 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:46.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:46.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:46.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:30:46.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:30:46.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:30:46.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:30:46.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:30:46.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:30:46.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:30:46.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:30:46.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:30:46.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:30:46.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:30:46.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:30:46.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:30:46.513 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.513 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.514 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:46.514 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:46.514 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.557658046, 2024-09-17T07:30:46.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:30:46.514 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.515 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.515 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.515 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:30:46.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.515 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:46.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:46.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:30:46.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:30:46.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:30:46.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:30:46.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:30:46.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:30:46.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:30:46.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:30:46.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:30:46.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:30:46.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:30:46.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:30:46.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:30:46.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:30:46.519 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.519 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.519 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.519 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:46.520 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:46.520 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.557635619, 2024-09-17T07:30:46.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:30:46.520 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.520 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.520 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.520 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:30:46.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.520 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:46.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:46.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:46.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:46.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:30:46.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:30:46.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:30:46.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:30:46.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:30:46.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:30:46.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:30:46.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:30:46.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:30:46.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:30:46.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:30:46.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:30:46.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.524 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.525 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:46.525 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:46.525 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.55760780200001, 2024-09-17T07:30:46.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:30:46.525 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.525 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.525 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.525 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:30:46.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.526 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:46.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:46.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:46.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:30:46.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:30:46.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:30:46.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:30:46.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:30:46.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:30:46.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:30:46.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:30:46.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:30:46.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:30:46.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:30:46.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:30:46.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:30:46.529 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.529 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.529 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.530 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:46.530 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:46.530 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.557585858, 2024-09-17T07:30:46.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:30:46.530 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.530 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.530 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.530 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:30:46.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.530 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:46.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:46.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:46.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:30:46.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:30:46.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:30:46.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:30:46.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:30:46.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:30:46.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:30:46.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:30:46.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:30:46.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:30:46.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:30:46.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:30:46.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:30:46.535 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.535 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.535 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.535 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:46.535 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:46.535 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.557564754, 2024-09-17T07:30:46.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:30:46.535 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.535 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.535 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.536 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:30:46.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.536 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:46.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:46.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:46.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:30:46.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:30:46.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:30:46.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:30:46.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:46.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:30:46.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:30:46.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:30:46.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:30:46.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:30:46.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:30:46.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:30:46.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:30:46.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.541 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.541 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.542 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:46.542 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:46.542 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.557545151, 2024-09-17T07:30:46.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:30:46.542 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.542 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.542 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.542 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.543 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:30:46.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.543 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:46.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:46.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:30:46.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:30:46.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:30:46.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:30:46.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:30:46.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:30:46.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:30:46.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:30:46.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:30:46.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:30:46.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:30:46.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:30:46.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:30:46.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:30:46.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.548 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.549 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:46.549 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:46.549 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.604604038000005, 2024-09-17T07:30:46.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:30:46.549 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.549 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.549 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.550 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:30:46.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.550 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:46.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:46.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:30:46.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:30:46.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:30:46.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:46.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:30:46.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:30:46.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:30:46.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:30:46.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:30:46.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:30:46.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:30:46.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:30:46.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:30:46.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:30:46.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.554 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.554 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:46.554 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:46.554 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.604590377999997, 2024-09-17T07:30:46.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:30:46.554 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:46.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:46.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:30:46.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:30:46.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:30:46.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:30:46.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:30:46.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:30:46.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:30:46.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:30:46.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:30:46.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:30:46.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:30:46.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:30:46.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:30:46.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.558 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.558 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.604575596000004, 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:46.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:46.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:46.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:30:46.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:30:46.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:30:46.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:30:46.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:46.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:30:46.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:30:46.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:30:46.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:30:46.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:30:46.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:30:46.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:30:46.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:30:46.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.563 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.563 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:46.563 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:46.563 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.604564705000001, 2024-09-17T07:30:46.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:30:46.563 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.563 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.563 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.564 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.564 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.564 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.564 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:30:46.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.564 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:46.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:46.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:46.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:46.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:30:46.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:30:46.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:30:46.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:30:46.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:30:46.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:30:46.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:30:46.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:30:46.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:30:46.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:30:46.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:30:46.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:30:46.567 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.568 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.568 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.568 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:46.568 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:46.568 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.604555145000006, 2024-09-17T07:30:46.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:30:46.568 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.568 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.568 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.569 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:30:46.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.569 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:46.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:46.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:30:46.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:46.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:30:46.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:30:46.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:30:46.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:30:46.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:30:46.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:30:46.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:30:46.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:30:46.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:30:46.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:30:46.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:30:46.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:30:46.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.572 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.573 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:46.573 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:46.573 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.604544584999999, 2024-09-17T07:30:46.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:30:46.573 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:46.573 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:46.574 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:46.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:46.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:46.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:46.574 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:30:46.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.574 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:46.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:46.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:46.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:46.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:30:46.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:30:46.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:46.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:30:46.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:30:46.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:46.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:30:46.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:30:46.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:46.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:30:46.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:30:46.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:46.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:46.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:46.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:30:46.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:30:46.577 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.577 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.577 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.577 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:46.577 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:46.577 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:46.578 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:46.579 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:46.579 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:46.580 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:46.580 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:46.581 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:46.578+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:46.679 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:46.679+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:46.723 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:46.723 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:46.733 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:46.780 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:46.779+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:46.876 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:46.876 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:46.881 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:46.881+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:46.885 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:46.982 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:46.982+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:46.988 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:46.988 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:46.997 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:47.083 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:47.083+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:47.125 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:47.125 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:47.135 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:47.221 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:47.220+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:47.285 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:47.284+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:47.288 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:47.288 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:47.297 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:47.386 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:47.386+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:47.444 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:47.444 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:47.452 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:47.494 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:47.493+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:47.588 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:47.588+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:47.600 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:47.600 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:47.609 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:47.690 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:47.689+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:47.757 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:47.757 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:47.765 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:47.790 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:47.790+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:47.871 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:47.872 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:47.880 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:47.924 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:47.924+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:47.988 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:47.989 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:47.991 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:47.991+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:47.997 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:48.092 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:48.092+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:48.144 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:48.144 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:48.153 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:48.190 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T07:30:48.195 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:48.193+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:48.294 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:48.294+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:48.317 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:48.317 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:48.327 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:48.395 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:48.395+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:48.474 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:48.475 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:48.483 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:48.527 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:48.527+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:48.582 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:30:48.590 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:48.590 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:48.597 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:48.596+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:48.600 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:48.698 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:48.697+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:48.709 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:48.709 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:48.726 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:48.727 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:48.743 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:48.743 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:48.751 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17069 3.0 e41/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 41'110, mlcod=40'108)", 2024-09-17T07:30:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 106.01048438700001, 2024-09-17T07:30:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011408908000000001, 2024-09-17T07:30:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.752 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:30:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.753 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739753+0000", 2024-09-17T07:30:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:30:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743532+0000", 2024-09-17T07:30:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037794510000000001 2024-09-17T07:30:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743557+0000", 2024-09-17T07:30:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5253e-05 2024-09-17T07:30:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743569+0000", 2024-09-17T07:30:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1161999999999999e-05 2024-09-17T07:30:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743650+0000", 2024-09-17T07:30:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1087999999999993e-05 2024-09-17T07:30:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743674+0000", 2024-09-17T07:30:48.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4562e-05 2024-09-17T07:30:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751095+0000", 2024-09-17T07:30:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074211140000000004 2024-09-17T07:30:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751161+0000", 2024-09-17T07:30:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5707e-05 2024-09-17T07:30:48.756 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17240 3.0 e42/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 42'113, mlcod=41'110)", 2024-09-17T07:30:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 105.52482590299999, 2024-09-17T07:30:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064392440000000002, 2024-09-17T07:30:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:30:48.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:30:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225418+0000", 2024-09-17T07:30:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8360000000000003e-06 2024-09-17T07:30:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225426+0000", 2024-09-17T07:30:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2670000000000006e-06 2024-09-17T07:30:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225434+0000", 2024-09-17T07:30:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0560000000000005e-06 2024-09-17T07:30:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225514+0000", 2024-09-17T07:30:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9979000000000001e-05 2024-09-17T07:30:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225535+0000", 2024-09-17T07:30:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0965e-05 2024-09-17T07:30:48.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231797+0000", 2024-09-17T07:30:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062621129999999997 2024-09-17T07:30:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231850+0000", 2024-09-17T07:30:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2611999999999997e-05 2024-09-17T07:30:48.761 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:32543 3.0 e55/23 3:03bfd97d:::rbd_data.11839a97b753.0000000000000007:head v 55'150, mlcod=54'148)", 2024-09-17T07:30:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 91.29559965, 2024-09-17T07:30:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031813480000000001, 2024-09-17T07:30:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.762 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.454637+0000", 2024-09-17T07:30:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5199999999999998e-07 2024-09-17T07:30:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455083+0000", 2024-09-17T07:30:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00044629199999999999 2024-09-17T07:30:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455098+0000", 2024-09-17T07:30:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4183e-05 2024-09-17T07:30:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455106+0000", 2024-09-17T07:30:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:30:48.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455173+0000", 2024-09-17T07:30:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6346000000000005e-05 2024-09-17T07:30:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.455200+0000", 2024-09-17T07:30:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7319000000000001e-05 2024-09-17T07:30:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457774+0000", 2024-09-17T07:30:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025742650000000001 2024-09-17T07:30:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.457818+0000", 2024-09-17T07:30:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3891999999999999e-05 2024-09-17T07:30:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33735 3.1f e55/52 3:fea54097:::rbd_data.11839a97b753.0000000000000000:head v 55'260, mlcod=55'258)", 2024-09-17T07:30:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 90.925667021999999, 2024-09-17T07:30:48.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035215200000000002, 2024-09-17T07:30:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:30:48.767 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824571+0000", 2024-09-17T07:30:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.164e-06 2024-09-17T07:30:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825118+0000", 2024-09-17T07:30:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054696400000000004 2024-09-17T07:30:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.768 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825134+0000", 2024-09-17T07:30:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6342999999999999e-05 2024-09-17T07:30:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825142+0000", 2024-09-17T07:30:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5270000000000001e-06 2024-09-17T07:30:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825200+0000", 2024-09-17T07:30:48.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.893e-05 2024-09-17T07:30:48.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.770 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825218+0000", 2024-09-17T07:30:48.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7935999999999999e-05 2024-09-17T07:30:48.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.770 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.770 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828037+0000", 2024-09-17T07:30:48.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002818961 2024-09-17T07:30:48.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.771 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828091+0000", 2024-09-17T07:30:48.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3695e-05 2024-09-17T07:30:48.771 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.771 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.771 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.771 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33869 3.9 e55/23 3:907b81b5:::rbd_data.11839a97b753.0000000000000001:head v 55'171, mlcod=55'169)", 2024-09-17T07:30:48.771 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:48.772 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 90.908611199000006, 2024-09-17T07:30:48.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033990859999999999, 2024-09-17T07:30:48.772 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.772 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.772 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.772 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.772 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.772 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:48.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.772 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:30:48.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841626+0000", 2024-09-17T07:30:48.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.023e-06 2024-09-17T07:30:48.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842067+0000", 2024-09-17T07:30:48.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000440612 2024-09-17T07:30:48.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.774 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842080+0000", 2024-09-17T07:30:48.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.29e-05 2024-09-17T07:30:48.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.774 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842086+0000", 2024-09-17T07:30:48.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2090000000000002e-06 2024-09-17T07:30:48.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.774 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842144+0000", 2024-09-17T07:30:48.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7627999999999998e-05 2024-09-17T07:30:48.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842171+0000", 2024-09-17T07:30:48.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7214000000000001e-05 2024-09-17T07:30:48.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.844973+0000", 2024-09-17T07:30:48.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002802232 2024-09-17T07:30:48.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.845024+0000", 2024-09-17T07:30:48.776 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1267999999999998e-05 2024-09-17T07:30:48.776 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.776 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.776 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.776 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.776 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51432 3.0 e59/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 59'177, mlcod=58'174)", 2024-09-17T07:30:48.776 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:48.776 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 85.929014182000003, 2024-09-17T07:30:48.776 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010790086000000001, 2024-09-17T07:30:48.776 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.776 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.776 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.777 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.777 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.777 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:48.777 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.777 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.777 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:30:48.777 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.777 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.777 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821224+0000", 2024-09-17T07:30:48.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.511e-06 2024-09-17T07:30:48.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.778 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.778 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825174+0000", 2024-09-17T07:30:48.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0039497949999999999 2024-09-17T07:30:48.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.778 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.778 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825196+0000", 2024-09-17T07:30:48.778 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2643999999999999e-05 2024-09-17T07:30:48.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.778 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.778 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825206+0000", 2024-09-17T07:30:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4320000000000005e-06 2024-09-17T07:30:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825272+0000", 2024-09-17T07:30:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5829999999999998e-05 2024-09-17T07:30:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825299+0000", 2024-09-17T07:30:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7107e-05 2024-09-17T07:30:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.831939+0000", 2024-09-17T07:30:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0066398510000000004 2024-09-17T07:30:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.832012+0000", 2024-09-17T07:30:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3916000000000002e-05 2024-09-17T07:30:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51969 3.0 e59/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 59'179, mlcod=59'177)", 2024-09-17T07:30:48.780 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:48.781 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 85.820145804000006, 2024-09-17T07:30:48.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060297249999999997, 2024-09-17T07:30:48.781 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.781 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.781 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.781 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.781 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:48.781 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.781 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.781 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:30:48.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.782 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.782 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930093+0000", 2024-09-17T07:30:48.782 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.764e-06 2024-09-17T07:30:48.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.782 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.782 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931779+0000", 2024-09-17T07:30:48.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001686379 2024-09-17T07:30:48.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.783 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.783 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931802+0000", 2024-09-17T07:30:48.783 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2682000000000001e-05 2024-09-17T07:30:48.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.783 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.784 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.784 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931811+0000", 2024-09-17T07:30:48.784 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5640000000000002e-06 2024-09-17T07:30:48.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.784 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.784 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931877+0000", 2024-09-17T07:30:48.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6155999999999995e-05 2024-09-17T07:30:48.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931896+0000", 2024-09-17T07:30:48.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9111e-05 2024-09-17T07:30:48.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.785 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.785 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.785 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936063+0000", 2024-09-17T07:30:48.785 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004166711 2024-09-17T07:30:48.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.786 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.786 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.786 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936121+0000", 2024-09-17T07:30:48.786 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7358000000000002e-05 2024-09-17T07:30:48.786 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.787 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.787 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.787 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.787 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54417 3.0 e60/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 60'182, mlcod=59'179)", 2024-09-17T07:30:48.788 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:48.788 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 85.312628813000003, 2024-09-17T07:30:48.788 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0059885219999999996, 2024-09-17T07:30:48.788 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.788 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.789 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.789 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.789 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:48.789 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.789 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:30:48.790 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.790 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.790 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.790 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437609+0000", 2024-09-17T07:30:48.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5829999999999999e-06 2024-09-17T07:30:48.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.791 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.791 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.791 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437623+0000", 2024-09-17T07:30:48.791 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3348e-05 2024-09-17T07:30:48.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.792 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.792 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437643+0000", 2024-09-17T07:30:48.792 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0463999999999999e-05 2024-09-17T07:30:48.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.792 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437653+0000", 2024-09-17T07:30:48.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6290000000000005e-06 2024-09-17T07:30:48.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.793 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.793 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437719+0000", 2024-09-17T07:30:48.793 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6428999999999997e-05 2024-09-17T07:30:48.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437745+0000", 2024-09-17T07:30:48.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5690999999999998e-05 2024-09-17T07:30:48.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.794 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443549+0000", 2024-09-17T07:30:48.794 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0058038259999999998 2024-09-17T07:30:48.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.794 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.794 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.795 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443596+0000", 2024-09-17T07:30:48.795 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7552000000000001e-05 2024-09-17T07:30:48.795 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.795 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.795 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.795 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.795 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54419 3.0 e60/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 60'184, mlcod=59'179)", 2024-09-17T07:30:48.795 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:48.796 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 85.306276456000006, 2024-09-17T07:30:48.796 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004706844, 2024-09-17T07:30:48.796 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.797 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.797 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.797 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.797 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:48.797 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.797 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.797 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:30:48.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443961+0000", 2024-09-17T07:30:48.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:30:48.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.798 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.798 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443967+0000", 2024-09-17T07:30:48.798 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.268e-06 2024-09-17T07:30:48.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.798 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443983+0000", 2024-09-17T07:30:48.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5248999999999999e-05 2024-09-17T07:30:48.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443990+0000", 2024-09-17T07:30:48.799 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5120000000000002e-06 2024-09-17T07:30:48.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.799 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.799 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.799 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444056+0000", 2024-09-17T07:30:48.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6197000000000003e-05 2024-09-17T07:30:48.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.800 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.800 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444081+0000", 2024-09-17T07:30:48.800 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4791e-05 2024-09-17T07:30:48.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.800 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448616+0000", 2024-09-17T07:30:48.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045346129999999998 2024-09-17T07:30:48.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.801 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.801 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448667+0000", 2024-09-17T07:30:48.801 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-17T07:30:48.801 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.801 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.801 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.801 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.802 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:80794 3.9 e72/23 3:90311a61:::rbd_data.1183cb746bb8.0000000000000007:head v 72'222, mlcod=71'220)", 2024-09-17T07:30:48.802 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:48.802 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 71.318287572000003, 2024-09-17T07:30:48.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004371733, 2024-09-17T07:30:48.802 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.802 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.802 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.802 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.802 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.802 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:48.802 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:48.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.803 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.803 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:30:48.803 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0800000000000001e-07 2024-09-17T07:30:48.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.803 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.804 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.804 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434108+0000", 2024-09-17T07:30:48.804 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021587569999999999 2024-09-17T07:30:48.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.804 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434120+0000", 2024-09-17T07:30:48.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1821000000000001e-05 2024-09-17T07:30:48.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.805 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.805 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.805 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434128+0000", 2024-09-17T07:30:48.805 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7579999999999992e-06 2024-09-17T07:30:48.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434191+0000", 2024-09-17T07:30:48.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3202999999999998e-05 2024-09-17T07:30:48.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.806 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434218+0000", 2024-09-17T07:30:48.806 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6771000000000002e-05 2024-09-17T07:30:48.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.806 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.806 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436289+0000", 2024-09-17T07:30:48.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020717700000000001 2024-09-17T07:30:48.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.807 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.807 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.807 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436321+0000", 2024-09-17T07:30:48.807 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1445000000000001e-05 2024-09-17T07:30:48.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.807 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.807 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.808 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 75/74 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head, version: 36'126, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head@36'126, size: 9431460, copy_subset: [], clone_subset: {3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9: [(0, 9431460)]}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:48.808 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:48.808 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 67.009379979000002, 2024-09-17T07:30:48.808 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041288289999999997, 2024-09-17T07:30:48.808 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.808 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:48.808 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.808 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:48.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:30:48.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.809 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.809 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.809 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740858+0000", 2024-09-17T07:30:48.809 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1599999999999999e-06 2024-09-17T07:30:48.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740885+0000", 2024-09-17T07:30:48.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.707e-05 2024-09-17T07:30:48.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740925+0000", 2024-09-17T07:30:48.810 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0649000000000003e-05 2024-09-17T07:30:48.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.810 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.810 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.810 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740957+0000", 2024-09-17T07:30:48.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1585999999999999e-05 2024-09-17T07:30:48.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741185+0000", 2024-09-17T07:30:48.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022759700000000001 2024-09-17T07:30:48.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.811 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.811 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.811 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741228+0000", 2024-09-17T07:30:48.811 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3705999999999998e-05 2024-09-17T07:30:48.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.812 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.812 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.744985+0000", 2024-09-17T07:30:48.812 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037570609999999999 2024-09-17T07:30:48.812 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.812 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.812 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.812 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85061 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'257, mlcod=79'255)", 2024-09-17T07:30:48.812 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:48.813 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 63.601549022999997, 2024-09-17T07:30:48.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033850550000000001, 2024-09-17T07:30:48.813 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.813 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.813 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.813 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:48.813 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.813 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:48.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:30:48.814 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9699999999999995e-07 2024-09-17T07:30:48.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.814 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.814 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.814 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148835+0000", 2024-09-17T07:30:48.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000146852 2024-09-17T07:30:48.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148845+0000", 2024-09-17T07:30:48.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0232e-05 2024-09-17T07:30:48.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.815 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148852+0000", 2024-09-17T07:30:48.815 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6490000000000002e-06 2024-09-17T07:30:48.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.815 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.815 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148913+0000", 2024-09-17T07:30:48.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1099999999999994e-05 2024-09-17T07:30:48.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148931+0000", 2024-09-17T07:30:48.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7938e-05 2024-09-17T07:30:48.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.816 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.816 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.816 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152025+0000", 2024-09-17T07:30:48.816 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0030942729999999998 2024-09-17T07:30:48.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.817 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.817 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152073+0000", 2024-09-17T07:30:48.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7213999999999999e-05 2024-09-17T07:30:48.817 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.817 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.817 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.817 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.817 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85063 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'259, mlcod=79'255)", 2024-09-17T07:30:48.817 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:48.817 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 63.600402068999998, 2024-09-17T07:30:48.817 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037722659999999998, 2024-09-17T07:30:48.818 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.818 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:48.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.818 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.818 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.818 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:48.818 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:30:48.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5799999999999999e-07 2024-09-17T07:30:48.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149841+0000", 2024-09-17T07:30:48.819 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4840000000000003e-06 2024-09-17T07:30:48.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.819 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.819 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.819 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149849+0000", 2024-09-17T07:30:48.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4670000000000004e-06 2024-09-17T07:30:48.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149854+0000", 2024-09-17T07:30:48.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4779999999999999e-06 2024-09-17T07:30:48.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.820 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.820 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149920+0000", 2024-09-17T07:30:48.820 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5760999999999996e-05 2024-09-17T07:30:48.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.820 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149949+0000", 2024-09-17T07:30:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8838000000000001e-05 2024-09-17T07:30:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153558+0000", 2024-09-17T07:30:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036092939999999999 2024-09-17T07:30:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153607+0000", 2024-09-17T07:30:48.821 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8386000000000001e-05 2024-09-17T07:30:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:94996 3.0 e82/23 3:074b4f56:::rbd_data.118311e0d6f8.0000000000000001:head v 82'715, mlcod=80'713)", 2024-09-17T07:30:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 59.134496476000002, 2024-09-17T07:30:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031499179999999998, 2024-09-17T07:30:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.822 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:48.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.823 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615740+0000", 2024-09-17T07:30:48.823 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.823 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.823 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.615741+0000", 2024-09-17T07:30:48.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999996e-07 2024-09-17T07:30:48.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.824 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.824 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.824 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616917+0000", 2024-09-17T07:30:48.824 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0011763310000000001 2024-09-17T07:30:48.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616936+0000", 2024-09-17T07:30:48.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8624e-05 2024-09-17T07:30:48.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.616948+0000", 2024-09-17T07:30:48.825 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.238e-05 2024-09-17T07:30:48.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.825 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.825 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.825 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617054+0000", 2024-09-17T07:30:48.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000106556 2024-09-17T07:30:48.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.617081+0000", 2024-09-17T07:30:48.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.622e-05 2024-09-17T07:30:48.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.826 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618836+0000", 2024-09-17T07:30:48.826 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001755412 2024-09-17T07:30:48.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.826 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.826 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.827 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.618890+0000", 2024-09-17T07:30:48.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3915000000000001e-05 2024-09-17T07:30:48.827 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.827 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.827 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.827 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.827 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:95007 3.5 e82/76 3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head v 82'422, mlcod=82'420)", 2024-09-17T07:30:48.827 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:48.827 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 59.126573774999997, 2024-09-17T07:30:48.827 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003939901, 2024-09-17T07:30:48.827 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.827 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:48.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:30:48.828 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.828 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.828 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.828 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623664+0000", 2024-09-17T07:30:48.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3799999999999999e-06 2024-09-17T07:30:48.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626104+0000", 2024-09-17T07:30:48.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002440149 2024-09-17T07:30:48.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.829 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626120+0000", 2024-09-17T07:30:48.829 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5962000000000001e-05 2024-09-17T07:30:48.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.829 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.829 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626127+0000", 2024-09-17T07:30:48.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8340000000000001e-06 2024-09-17T07:30:48.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626186+0000", 2024-09-17T07:30:48.830 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9234999999999998e-05 2024-09-17T07:30:48.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.830 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.830 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.830 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626208+0000", 2024-09-17T07:30:48.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1719e-05 2024-09-17T07:30:48.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627567+0000", 2024-09-17T07:30:48.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013587919999999999 2024-09-17T07:30:48.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.831 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.831 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.831 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627603+0000", 2024-09-17T07:30:48.831 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5830000000000001e-05 2024-09-17T07:30:48.831 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.831 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.832 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.832 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.832 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:104928 3.18 e87/76 3:1ef990b1:::rbd_data.1183dd51323.0000000000000006:head v 87'562, mlcod=86'559)", 2024-09-17T07:30:48.832 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:48.832 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 53.207818064000001, 2024-09-17T07:30:48.832 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0061095869999999997, 2024-09-17T07:30:48.832 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.832 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.832 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:30:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542421+0000", 2024-09-17T07:30:48.833 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.069e-06 2024-09-17T07:30:48.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544190+0000", 2024-09-17T07:30:48.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017693539999999999 2024-09-17T07:30:48.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.834 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.834 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.834 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544215+0000", 2024-09-17T07:30:48.834 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5398999999999999e-05 2024-09-17T07:30:48.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544229+0000", 2024-09-17T07:30:48.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3865e-05 2024-09-17T07:30:48.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544322+0000", 2024-09-17T07:30:48.835 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2540000000000005e-05 2024-09-17T07:30:48.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.835 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.835 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.835 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544348+0000", 2024-09-17T07:30:48.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6197999999999999e-05 2024-09-17T07:30:48.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548475+0000", 2024-09-17T07:30:48.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041266000000000002 2024-09-17T07:30:48.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.836 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.836 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.836 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548528+0000", 2024-09-17T07:30:48.836 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3562e-05 2024-09-17T07:30:48.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.836 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.837 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.837 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115898 3.9 3:910101b0:::rbd_data.118340e4fbf.0000000000000000:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:48.837 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:30:48.837 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 26.559303959000001, 2024-09-17T07:30:48.837 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0042466279999999997, 2024-09-17T07:30:48.837 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.837 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:48.837 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:48.837 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:48.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115898 2024-09-17T07:30:48.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.838 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:30:48.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.838 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.838 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:30:48.838 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.838 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:48.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0030000000000001e-06 2024-09-17T07:30:48.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.839 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.839 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190940+0000", 2024-09-17T07:30:48.839 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2349999999999999e-06 2024-09-17T07:30:48.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.839 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190953+0000", 2024-09-17T07:30:48.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3304e-05 2024-09-17T07:30:48.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190995+0000", 2024-09-17T07:30:48.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2317000000000002e-05 2024-09-17T07:30:48.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.840 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.840 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.840 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191592+0000", 2024-09-17T07:30:48.840 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00059607799999999999 2024-09-17T07:30:48.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191888+0000", 2024-09-17T07:30:48.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00029619999999999999 2024-09-17T07:30:48.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:48.841 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192069+0000", 2024-09-17T07:30:48.841 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00018125899999999999 2024-09-17T07:30:48.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.841 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.841 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:48.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193692+0000", 2024-09-17T07:30:48.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001623364 2024-09-17T07:30:48.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:48.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194968+0000", 2024-09-17T07:30:48.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012755189999999999 2024-09-17T07:30:48.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.842 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.842 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.195037+0000", 2024-09-17T07:30:48.842 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9450999999999996e-05 2024-09-17T07:30:48.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.842 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.843 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.843 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.195179+0000", 2024-09-17T07:30:48.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000141898 2024-09-17T07:30:48.843 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.843 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.843 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.843 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.843 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115936 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [watch watch cookie 140227662410368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:48.843 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:48.843 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 26.526385007000002, 2024-09-17T07:30:48.843 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043930569999999997, 2024-09-17T07:30:48.843 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.844 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.844 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.844 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:48.844 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:48.844 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:48.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115936 2024-09-17T07:30:48.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.844 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.844 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.844 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:48.844 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.844 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:48.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.845 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:30:48.845 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7300000000000005e-07 2024-09-17T07:30:48.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.845 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.845 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223856+0000", 2024-09-17T07:30:48.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9169999999999999e-06 2024-09-17T07:30:48.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223870+0000", 2024-09-17T07:30:48.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3926e-05 2024-09-17T07:30:48.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.846 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223878+0000", 2024-09-17T07:30:48.846 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3699999999999997e-06 2024-09-17T07:30:48.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.846 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.846 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223934+0000", 2024-09-17T07:30:48.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6929000000000003e-05 2024-09-17T07:30:48.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224053+0000", 2024-09-17T07:30:48.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000118952 2024-09-17T07:30:48.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.847 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:48.847 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224271+0000", 2024-09-17T07:30:48.847 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021763300000000001 2024-09-17T07:30:48.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.847 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:48.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224992+0000", 2024-09-17T07:30:48.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072142099999999998 2024-09-17T07:30:48.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:48.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228099+0000", 2024-09-17T07:30:48.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031060850000000002 2024-09-17T07:30:48.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.848 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.848 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228143+0000", 2024-09-17T07:30:48.848 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4614999999999999e-05 2024-09-17T07:30:48.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.848 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.849 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.849 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228245+0000", 2024-09-17T07:30:48.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000101436 2024-09-17T07:30:48.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.849 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.849 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115955 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [call rbd.snapshot_trash_add in=21b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:48.849 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:30:48.849 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 26.507398254000002, 2024-09-17T07:30:48.849 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004259054, 2024-09-17T07:30:48.849 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.850 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:48.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:48.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:48.850 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:48.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115955 2024-09-17T07:30:48.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.850 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:30:48.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:30:48.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242839+0000", 2024-09-17T07:30:48.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6899999999999998e-07 2024-09-17T07:30:48.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242842+0000", 2024-09-17T07:30:48.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1350000000000001e-06 2024-09-17T07:30:48.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242849+0000", 2024-09-17T07:30:48.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0400000000000004e-06 2024-09-17T07:30:48.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242854+0000", 2024-09-17T07:30:48.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4940000000000003e-06 2024-09-17T07:30:48.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243137+0000", 2024-09-17T07:30:48.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00028217100000000001 2024-09-17T07:30:48.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243196+0000", 2024-09-17T07:30:48.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9608999999999998e-05 2024-09-17T07:30:48.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:48.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243448+0000", 2024-09-17T07:30:48.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00025185000000000002 2024-09-17T07:30:48.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:48.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243990+0000", 2024-09-17T07:30:48.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054153199999999995 2024-09-17T07:30:48.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:48.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.246867+0000", 2024-09-17T07:30:48.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028771880000000001 2024-09-17T07:30:48.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:48.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.246943+0000", 2024-09-17T07:30:48.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5689e-05 2024-09-17T07:30:48.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247097+0000", 2024-09-17T07:30:48.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015487699999999999 2024-09-17T07:30:48.855 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.855 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.855 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.855 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115956 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:48.855 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:30:48.855 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 26.507101670000001, 2024-09-17T07:30:48.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043741479999999996, 2024-09-17T07:30:48.856 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:48.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:48.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:48.856 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:48.856 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:48.856 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:48.856 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115956 2024-09-17T07:30:48.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.857 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:48.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:48.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:30:48.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:48.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:30:48.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:48.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:48.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243136+0000", 2024-09-17T07:30:48.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7100000000000001e-07 2024-09-17T07:30:48.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:48.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243141+0000", 2024-09-17T07:30:48.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.716e-06 2024-09-17T07:30:48.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:48.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243153+0000", 2024-09-17T07:30:48.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1198e-05 2024-09-17T07:30:48.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:48.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243160+0000", 2024-09-17T07:30:48.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6229999999999997e-06 2024-09-17T07:30:48.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243797+0000", 2024-09-17T07:30:48.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063716399999999996 2024-09-17T07:30:48.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for rw locks", 2024-09-17T07:30:48.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243852+0000", 2024-09-17T07:30:48.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4147999999999997e-05 2024-09-17T07:30:48.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247147+0000", 2024-09-17T07:30:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0032955940000000002 2024-09-17T07:30:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247271+0000", 2024-09-17T07:30:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012397800000000001 2024-09-17T07:30:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247509+0000", 2024-09-17T07:30:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000237956 2024-09-17T07:30:48.860 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.861 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.861 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.861 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:48.861 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:48.862 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:48.863 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:48.864 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:48.864 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:48.865 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:48.866 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:48.866 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:48.862+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:48.963 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:48.962+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:48.981 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:48.981 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:48.981 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:48.981 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:48.990 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:49.074 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:49.074+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:49.134 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:49.134 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:49.135 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:49.135 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:49.135 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:49.143 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:49.182 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-17T07:30:49.184 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:49.184+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:49.266 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:49.266+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:49.299 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:49.299 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:30525 3.1b 3:def7c8f0:::rbd_data.118381ffbbab.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T07:30:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.775408106, 2024-09-17T07:30:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55515850300000003, 2024-09-17T07:30:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30525 2024-09-17T07:30:49.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:30:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522300+0000", 2024-09-17T07:30:49.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:30:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522305+0000", 2024-09-17T07:30:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-17T07:30:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522318+0000", 2024-09-17T07:30:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3030999999999999e-05 2024-09-17T07:30:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522326+0000", 2024-09-17T07:30:49.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4200000000000001e-06 2024-09-17T07:30:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522385+0000", 2024-09-17T07:30:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9005000000000002e-05 2024-09-17T07:30:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522393+0000", 2024-09-17T07:30:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-06 2024-09-17T07:30:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075347+0000", 2024-09-17T07:30:49.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55295417800000002 2024-09-17T07:30:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075502+0000", 2024-09-17T07:30:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154272 2024-09-17T07:30:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075630+0000", 2024-09-17T07:30:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012816300000000001 2024-09-17T07:30:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077221+0000", 2024-09-17T07:30:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015912929999999999 2024-09-17T07:30:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077353+0000", 2024-09-17T07:30:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013215699999999999 2024-09-17T07:30:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077413+0000", 2024-09-17T07:30:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0034999999999997e-05 2024-09-17T07:30:49.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077457+0000", 2024-09-17T07:30:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4017999999999998e-05 2024-09-17T07:30:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44181 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13922304~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.660229458000003, 2024-09-17T07:30:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046010502000000002, 2024-09-17T07:30:49.306 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44181 2024-09-17T07:30:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:30:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637478+0000", 2024-09-17T07:30:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:30:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637480+0000", 2024-09-17T07:30:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.843e-06 2024-09-17T07:30:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637487+0000", 2024-09-17T07:30:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6159999999999996e-06 2024-09-17T07:30:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637492+0000", 2024-09-17T07:30:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5360000000000003e-06 2024-09-17T07:30:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683351+0000", 2024-09-17T07:30:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045858758999999999 2024-09-17T07:30:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683422+0000", 2024-09-17T07:30:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1705999999999994e-05 2024-09-17T07:30:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683488+0000", 2024-09-17T07:30:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5612999999999997e-05 2024-09-17T07:30:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.310 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44182 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13938688~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.660208787000002, 2024-09-17T07:30:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046153118999999999, 2024-09-17T07:30:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44182 2024-09-17T07:30:49.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:30:49.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:30:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637506+0000", 2024-09-17T07:30:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1520000000000003e-06 2024-09-17T07:30:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637512+0000", 2024-09-17T07:30:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-09-17T07:30:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637517+0000", 2024-09-17T07:30:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-09-17T07:30:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683514+0000", 2024-09-17T07:30:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045997616999999998 2024-09-17T07:30:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683584+0000", 2024-09-17T07:30:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9942000000000002e-05 2024-09-17T07:30:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683651+0000", 2024-09-17T07:30:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6951000000000003e-05 2024-09-17T07:30:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44183 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13955072~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.660183555000003, 2024-09-17T07:30:49.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046339793999999997, 2024-09-17T07:30:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44183 2024-09-17T07:30:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:30:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639583+0000", 2024-09-17T07:30:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002059412 2024-09-17T07:30:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639597+0000", 2024-09-17T07:30:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3832000000000001e-05 2024-09-17T07:30:49.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639607+0000", 2024-09-17T07:30:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485e-05 2024-09-17T07:30:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639614+0000", 2024-09-17T07:30:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0849999999999999e-06 2024-09-17T07:30:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683705+0000", 2024-09-17T07:30:49.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044091261 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683792+0000", 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6197000000000002e-05 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683863+0000", 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1521999999999997e-05 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.319 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44199 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14217216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.656666002999998, 2024-09-17T07:30:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045995623999999999, 2024-09-17T07:30:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44199 2024-09-17T07:30:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:30:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:30:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641043+0000", 2024-09-17T07:30:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.624e-06 2024-09-17T07:30:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641048+0000", 2024-09-17T07:30:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4029999999999998e-06 2024-09-17T07:30:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641053+0000", 2024-09-17T07:30:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6670000000000004e-06 2024-09-17T07:30:49.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686918+0000", 2024-09-17T07:30:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045865131000000003 2024-09-17T07:30:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686986+0000", 2024-09-17T07:30:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8299000000000005e-05 2024-09-17T07:30:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687036+0000", 2024-09-17T07:30:49.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0164999999999998e-05 2024-09-17T07:30:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44200 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14233600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.656648173999997, 2024-09-17T07:30:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046183984999999997, 2024-09-17T07:30:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.324 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44200 2024-09-17T07:30:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:30:49.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:49.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641061+0000", 2024-09-17T07:30:49.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.767e-06 2024-09-17T07:30:49.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641067+0000", 2024-09-17T07:30:49.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7379999999999999e-06 2024-09-17T07:30:49.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641071+0000", 2024-09-17T07:30:49.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-09-17T07:30:49.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687078+0000", 2024-09-17T07:30:49.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046007095999999997 2024-09-17T07:30:49.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687158+0000", 2024-09-17T07:30:49.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0374e-05 2024-09-17T07:30:49.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687243+0000", 2024-09-17T07:30:49.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4221999999999994e-05 2024-09-17T07:30:49.328 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.328 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.328 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.328 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44201 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14249984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.328 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:49.328 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.656630527999994, 2024-09-17T07:30:49.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046303182999999998, 2024-09-17T07:30:49.329 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.329 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.329 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.329 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44201 2024-09-17T07:30:49.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.329 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:49.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:30:49.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641120+0000", 2024-09-17T07:30:49.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3933000000000001e-05 2024-09-17T07:30:49.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641123+0000", 2024-09-17T07:30:49.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.191e-06 2024-09-17T07:30:49.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641133+0000", 2024-09-17T07:30:49.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4020000000000008e-06 2024-09-17T07:30:49.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641138+0000", 2024-09-17T07:30:49.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6559999999999997e-06 2024-09-17T07:30:49.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687279+0000", 2024-09-17T07:30:49.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046140278999999999 2024-09-17T07:30:49.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687338+0000", 2024-09-17T07:30:49.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9268999999999998e-05 2024-09-17T07:30:49.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687379+0000", 2024-09-17T07:30:49.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1452999999999998e-05 2024-09-17T07:30:49.332 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.332 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.332 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.333 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44202 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14266368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.333 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:49.333 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.656559762000001, 2024-09-17T07:30:49.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046484299, 2024-09-17T07:30:49.333 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.333 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.333 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.333 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44202 2024-09-17T07:30:49.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.334 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:49.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:30:49.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642810+0000", 2024-09-17T07:30:49.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016627980000000001 2024-09-17T07:30:49.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642815+0000", 2024-09-17T07:30:49.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.091e-06 2024-09-17T07:30:49.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642829+0000", 2024-09-17T07:30:49.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4047e-05 2024-09-17T07:30:49.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642844+0000", 2024-09-17T07:30:49.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4515000000000001e-05 2024-09-17T07:30:49.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687426+0000", 2024-09-17T07:30:49.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044582201000000002 2024-09-17T07:30:49.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687529+0000", 2024-09-17T07:30:49.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103078 2024-09-17T07:30:49.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687631+0000", 2024-09-17T07:30:49.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102569 2024-09-17T07:30:49.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.337 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.337 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44209 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14381056~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.337 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:49.337 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.654661610000005, 2024-09-17T07:30:49.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045944444000000001, 2024-09-17T07:30:49.338 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.338 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.338 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.338 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44209 2024-09-17T07:30:49.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.338 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:49.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:30:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643046+0000", 2024-09-17T07:30:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:30:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643048+0000", 2024-09-17T07:30:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:30:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643054+0000", 2024-09-17T07:30:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.032e-06 2024-09-17T07:30:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643065+0000", 2024-09-17T07:30:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1861999999999999e-05 2024-09-17T07:30:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688806+0000", 2024-09-17T07:30:49.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045740617999999997 2024-09-17T07:30:49.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688927+0000", 2024-09-17T07:30:49.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000121327 2024-09-17T07:30:49.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688990+0000", 2024-09-17T07:30:49.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2293999999999997e-05 2024-09-17T07:30:49.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.342 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.342 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.342 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44210 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14397440~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.342 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:49.342 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.654634684000001, 2024-09-17T07:30:49.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046104635999999997, 2024-09-17T07:30:49.342 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.342 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.342 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.342 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44210 2024-09-17T07:30:49.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.343 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:49.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:30:49.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643073+0000", 2024-09-17T07:30:49.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.77e-07 2024-09-17T07:30:49.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643075+0000", 2024-09-17T07:30:49.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-09-17T07:30:49.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643082+0000", 2024-09-17T07:30:49.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7789999999999996e-06 2024-09-17T07:30:49.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643087+0000", 2024-09-17T07:30:49.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.967e-06 2024-09-17T07:30:49.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689022+0000", 2024-09-17T07:30:49.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045934349999999999 2024-09-17T07:30:49.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689112+0000", 2024-09-17T07:30:49.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0039999999999999e-05 2024-09-17T07:30:49.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689177+0000", 2024-09-17T07:30:49.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5041000000000003e-05 2024-09-17T07:30:49.346 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.346 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.346 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.346 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44211 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14413824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.346 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:49.346 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.654613550999997, 2024-09-17T07:30:49.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046309003000000001, 2024-09-17T07:30:49.346 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.346 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.347 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.347 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.347 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.347 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.347 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44211 2024-09-17T07:30:49.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.347 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:49.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:30:49.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643094+0000", 2024-09-17T07:30:49.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:30:49.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643095+0000", 2024-09-17T07:30:49.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-17T07:30:49.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643101+0000", 2024-09-17T07:30:49.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8730000000000004e-06 2024-09-17T07:30:49.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643106+0000", 2024-09-17T07:30:49.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6859999999999997e-06 2024-09-17T07:30:49.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689220+0000", 2024-09-17T07:30:49.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046113945000000003 2024-09-17T07:30:49.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689309+0000", 2024-09-17T07:30:49.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9560000000000003e-05 2024-09-17T07:30:49.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689402+0000", 2024-09-17T07:30:49.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2786000000000002e-05 2024-09-17T07:30:49.350 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.350 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.350 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.351 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44212 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14430208~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.351 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:49.351 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.654568988999998, 2024-09-17T07:30:49.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046448912000000002, 2024-09-17T07:30:49.351 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.351 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.351 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.351 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44212 2024-09-17T07:30:49.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.351 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:49.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:49.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:30:49.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2200000000000004e-07 2024-09-17T07:30:49.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643141+0000", 2024-09-17T07:30:49.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5210000000000001e-06 2024-09-17T07:30:49.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643149+0000", 2024-09-17T07:30:49.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6490000000000008e-06 2024-09-17T07:30:49.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643161+0000", 2024-09-17T07:30:49.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2241e-05 2024-09-17T07:30:49.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689423+0000", 2024-09-17T07:30:49.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046262323000000001 2024-09-17T07:30:49.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689505+0000", 2024-09-17T07:30:49.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2103999999999999e-05 2024-09-17T07:30:49.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689587+0000", 2024-09-17T07:30:49.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1452000000000006e-05 2024-09-17T07:30:49.355 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.355 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.355 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.355 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44213 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14446592~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.355 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:49.355 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.654537489999996, 2024-09-17T07:30:49.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046601780000000002, 2024-09-17T07:30:49.356 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.356 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.356 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.356 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44213 2024-09-17T07:30:49.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.356 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:49.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:30:49.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643170+0000", 2024-09-17T07:30:49.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-17T07:30:49.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643172+0000", 2024-09-17T07:30:49.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1660000000000001e-06 2024-09-17T07:30:49.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643180+0000", 2024-09-17T07:30:49.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-09-17T07:30:49.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643185+0000", 2024-09-17T07:30:49.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-09-17T07:30:49.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689622+0000", 2024-09-17T07:30:49.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046436742000000003 2024-09-17T07:30:49.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689697+0000", 2024-09-17T07:30:49.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5179999999999995e-05 2024-09-17T07:30:49.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689771+0000", 2024-09-17T07:30:49.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3928000000000001e-05 2024-09-17T07:30:49.360 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.360 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.360 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.360 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44214 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14462976~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.360 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:49.360 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.654514430000006, 2024-09-17T07:30:49.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046736038000000001, 2024-09-17T07:30:49.360 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.360 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.360 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.361 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44214 2024-09-17T07:30:49.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.361 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:49.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:30:49.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643193+0000", 2024-09-17T07:30:49.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:30:49.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643195+0000", 2024-09-17T07:30:49.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0640000000000001e-06 2024-09-17T07:30:49.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643201+0000", 2024-09-17T07:30:49.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2859999999999996e-06 2024-09-17T07:30:49.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643206+0000", 2024-09-17T07:30:49.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-17T07:30:49.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689814+0000", 2024-09-17T07:30:49.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046607742000000001 2024-09-17T07:30:49.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689886+0000", 2024-09-17T07:30:49.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2020999999999994e-05 2024-09-17T07:30:49.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689928+0000", 2024-09-17T07:30:49.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2771000000000001e-05 2024-09-17T07:30:49.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.364 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.364 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44215 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14479360~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.364 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:49.364 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.654470532999994, 2024-09-17T07:30:49.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046868543999999998, 2024-09-17T07:30:49.365 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.365 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.365 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.365 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44215 2024-09-17T07:30:49.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.365 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:49.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:30:49.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643237+0000", 2024-09-17T07:30:49.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:30:49.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643245+0000", 2024-09-17T07:30:49.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3270000000000001e-06 2024-09-17T07:30:49.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643254+0000", 2024-09-17T07:30:49.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6049999999999998e-06 2024-09-17T07:30:49.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643260+0000", 2024-09-17T07:30:49.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7849999999999997e-06 2024-09-17T07:30:49.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689961+0000", 2024-09-17T07:30:49.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046701196 2024-09-17T07:30:49.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690036+0000", 2024-09-17T07:30:49.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5153000000000004e-05 2024-09-17T07:30:49.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690105+0000", 2024-09-17T07:30:49.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8874000000000005e-05 2024-09-17T07:30:49.370 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.370 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.370 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.370 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44216 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14495744~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.370 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:49.370 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.654438819999996, 2024-09-17T07:30:49.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047003047999999999, 2024-09-17T07:30:49.371 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.371 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.371 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.371 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.371 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44216 2024-09-17T07:30:49.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.371 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:49.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:30:49.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643269+0000", 2024-09-17T07:30:49.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:30:49.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643271+0000", 2024-09-17T07:30:49.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-09-17T07:30:49.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643278+0000", 2024-09-17T07:30:49.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8850000000000001e-06 2024-09-17T07:30:49.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643282+0000", 2024-09-17T07:30:49.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5979999999999999e-06 2024-09-17T07:30:49.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690133+0000", 2024-09-17T07:30:49.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850470999999998 2024-09-17T07:30:49.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690205+0000", 2024-09-17T07:30:49.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2442999999999997e-05 2024-09-17T07:30:49.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690271+0000", 2024-09-17T07:30:49.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5874e-05 2024-09-17T07:30:49.374 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.374 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.375 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44217 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14512128~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.375 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:49.375 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.654417808000005, 2024-09-17T07:30:49.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047166485000000001, 2024-09-17T07:30:49.375 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.375 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.375 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.375 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.375 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.375 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.375 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44217 2024-09-17T07:30:49.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.376 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:49.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:49.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:30:49.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-17T07:30:49.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643291+0000", 2024-09-17T07:30:49.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.962e-06 2024-09-17T07:30:49.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643297+0000", 2024-09-17T07:30:49.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0700000000000003e-06 2024-09-17T07:30:49.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643302+0000", 2024-09-17T07:30:49.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:30:49.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690308+0000", 2024-09-17T07:30:49.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047005354999999999 2024-09-17T07:30:49.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690385+0000", 2024-09-17T07:30:49.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7541999999999997e-05 2024-09-17T07:30:49.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690455+0000", 2024-09-17T07:30:49.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0328999999999997e-05 2024-09-17T07:30:49.379 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.379 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.379 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.379 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44218 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14528512~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.379 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:49.379 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.654398130999994, 2024-09-17T07:30:49.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047321015000000001, 2024-09-17T07:30:49.380 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.380 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.380 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.380 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.380 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44218 2024-09-17T07:30:49.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.380 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:49.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:49.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:30:49.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-17T07:30:49.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643311+0000", 2024-09-17T07:30:49.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.933e-06 2024-09-17T07:30:49.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643318+0000", 2024-09-17T07:30:49.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3980000000000004e-06 2024-09-17T07:30:49.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643322+0000", 2024-09-17T07:30:49.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7609999999999998e-06 2024-09-17T07:30:49.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690484+0000", 2024-09-17T07:30:49.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047161428999999998 2024-09-17T07:30:49.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690564+0000", 2024-09-17T07:30:49.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9845000000000005e-05 2024-09-17T07:30:49.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690630+0000", 2024-09-17T07:30:49.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6021999999999999e-05 2024-09-17T07:30:49.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.384 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.384 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.384 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44219 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14544896~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.384 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:49.384 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.654377764000003, 2024-09-17T07:30:49.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048128708999999999, 2024-09-17T07:30:49.384 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.384 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.385 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.385 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.385 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44219 2024-09-17T07:30:49.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.385 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:49.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:49.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:30:49.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:30:49.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643337+0000", 2024-09-17T07:30:49.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.109e-06 2024-09-17T07:30:49.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643350+0000", 2024-09-17T07:30:49.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3046e-05 2024-09-17T07:30:49.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643356+0000", 2024-09-17T07:30:49.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8930000000000002e-06 2024-09-17T07:30:49.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690660+0000", 2024-09-17T07:30:49.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047304061000000001 2024-09-17T07:30:49.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691372+0000", 2024-09-17T07:30:49.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00071281299999999997 2024-09-17T07:30:49.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691458+0000", 2024-09-17T07:30:49.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5352000000000005e-05 2024-09-17T07:30:49.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.388 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.389 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44220 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14561280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:49.389 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:49.389 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.654343716, 2024-09-17T07:30:49.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048309849000000002, 2024-09-17T07:30:49.389 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.389 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.389 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.389 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.389 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44220 2024-09-17T07:30:49.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.390 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:49.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:30:49.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646869+0000", 2024-09-17T07:30:49.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003506267 2024-09-17T07:30:49.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646874+0000", 2024-09-17T07:30:49.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7500000000000003e-06 2024-09-17T07:30:49.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646887+0000", 2024-09-17T07:30:49.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3115999999999999e-05 2024-09-17T07:30:49.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646894+0000", 2024-09-17T07:30:49.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5289999999999997e-06 2024-09-17T07:30:49.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691547+0000", 2024-09-17T07:30:49.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044652743000000002 2024-09-17T07:30:49.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691622+0000", 2024-09-17T07:30:49.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5037999999999995e-05 2024-09-17T07:30:49.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691673+0000", 2024-09-17T07:30:49.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-17T07:30:49.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.393 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.393 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.393 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:49.395 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:49.395 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:49.394+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:49.495 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:49.495+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:49.540 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:49.540 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:49.540 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:49.540 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:49.548 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:49.596 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:49.595+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:49.651 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:49.651 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:49.652 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:49.652 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:49.652 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:49.660 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:49.701 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:49.701+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:49.797 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:49.796+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:49.803 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19985 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5275648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.550101263, 2024-09-17T07:30:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071819998999999995, 2024-09-17T07:30:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19985 2024-09-17T07:30:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:30:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252493+0000", 2024-09-17T07:30:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6300000000000004e-07 2024-09-17T07:30:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252496+0000", 2024-09-17T07:30:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-17T07:30:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252499+0000", 2024-09-17T07:30:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:30:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324036+0000", 2024-09-17T07:30:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071536587999999998 2024-09-17T07:30:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324118+0000", 2024-09-17T07:30:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2336999999999995e-05 2024-09-17T07:30:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324312+0000", 2024-09-17T07:30:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000193709 2024-09-17T07:30:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19986 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5505024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.550090111, 2024-09-17T07:30:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072083761999999996, 2024-09-17T07:30:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19986 2024-09-17T07:30:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:30:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:30:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252508+0000", 2024-09-17T07:30:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:30:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252511+0000", 2024-09-17T07:30:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.145e-06 2024-09-17T07:30:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252514+0000", 2024-09-17T07:30:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5569999999999998e-06 2024-09-17T07:30:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324326+0000", 2024-09-17T07:30:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071812681000000003 2024-09-17T07:30:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324372+0000", 2024-09-17T07:30:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5893000000000003e-05 2024-09-17T07:30:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324587+0000", 2024-09-17T07:30:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214608 2024-09-17T07:30:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19987 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5734400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.55007649300001, 2024-09-17T07:30:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072276676999999998, 2024-09-17T07:30:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19987 2024-09-17T07:30:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.814 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:30:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:49.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252518+0000", 2024-09-17T07:30:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-17T07:30:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252522+0000", 2024-09-17T07:30:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2440000000000004e-06 2024-09-17T07:30:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252525+0000", 2024-09-17T07:30:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5919999999999999e-06 2024-09-17T07:30:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324621+0000", 2024-09-17T07:30:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072096442999999996 2024-09-17T07:30:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324699+0000", 2024-09-17T07:30:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8061999999999999e-05 2024-09-17T07:30:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324793+0000", 2024-09-17T07:30:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4207000000000001e-05 2024-09-17T07:30:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19988 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.550065522, 2024-09-17T07:30:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072585798000000007, 2024-09-17T07:30:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19988 2024-09-17T07:30:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:30:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252529+0000", 2024-09-17T07:30:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:30:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252532+0000", 2024-09-17T07:30:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9579999999999999e-06 2024-09-17T07:30:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252534+0000", 2024-09-17T07:30:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5730000000000002e-06 2024-09-17T07:30:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324825+0000", 2024-09-17T07:30:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072290757999999997 2024-09-17T07:30:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324874+0000", 2024-09-17T07:30:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9360000000000002e-05 2024-09-17T07:30:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325113+0000", 2024-09-17T07:30:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000239025 2024-09-17T07:30:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19989 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.55005590099999, 2024-09-17T07:30:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072963494000000004, 2024-09-17T07:30:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19989 2024-09-17T07:30:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:30:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:30:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:30:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:30:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2900000000000002e-07 2024-09-17T07:30:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252542+0000", 2024-09-17T07:30:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8600000000000003e-06 2024-09-17T07:30:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252545+0000", 2024-09-17T07:30:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.605e-06 2024-09-17T07:30:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325144+0000", 2024-09-17T07:30:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072599460000000005 2024-09-17T07:30:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325223+0000", 2024-09-17T07:30:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8473999999999994e-05 2024-09-17T07:30:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325501+0000", 2024-09-17T07:30:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000278063 2024-09-17T07:30:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19990 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6422528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.55004543, 2024-09-17T07:30:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073241127000000003, 2024-09-17T07:30:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19990 2024-09-17T07:30:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:30:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252552+0000", 2024-09-17T07:30:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4719999999999997e-06 2024-09-17T07:30:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252557+0000", 2024-09-17T07:30:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3039999999999998e-06 2024-09-17T07:30:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252560+0000", 2024-09-17T07:30:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8830000000000002e-06 2024-09-17T07:30:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325529+0000", 2024-09-17T07:30:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072968880999999999 2024-09-17T07:30:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325574+0000", 2024-09-17T07:30:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466000000000001e-05 2024-09-17T07:30:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325789+0000", 2024-09-17T07:30:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021490999999999999 2024-09-17T07:30:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19991 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6651904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.550026128, 2024-09-17T07:30:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073720552999999994, 2024-09-17T07:30:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19991 2024-09-17T07:30:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:30:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4699999999999998e-07 2024-09-17T07:30:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252568+0000", 2024-09-17T07:30:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5300000000000002e-07 2024-09-17T07:30:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252572+0000", 2024-09-17T07:30:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9160000000000003e-06 2024-09-17T07:30:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252575+0000", 2024-09-17T07:30:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:30:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325820+0000", 2024-09-17T07:30:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073245597999999995 2024-09-17T07:30:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325901+0000", 2024-09-17T07:30:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0913000000000003e-05 2024-09-17T07:30:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326288+0000", 2024-09-17T07:30:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038646900000000002 2024-09-17T07:30:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19992 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6881280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.55001577500001, 2024-09-17T07:30:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074054683999999996, 2024-09-17T07:30:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19992 2024-09-17T07:30:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:30:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252578+0000", 2024-09-17T07:30:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:30:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252579+0000", 2024-09-17T07:30:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2500000000000004e-07 2024-09-17T07:30:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252582+0000", 2024-09-17T07:30:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-17T07:30:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252585+0000", 2024-09-17T07:30:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.542e-06 2024-09-17T07:30:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326315+0000", 2024-09-17T07:30:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073729921000000004 2024-09-17T07:30:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326359+0000", 2024-09-17T07:30:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4095999999999999e-05 2024-09-17T07:30:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326632+0000", 2024-09-17T07:30:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027314499999999997 2024-09-17T07:30:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19993 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7110656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.55000547100001, 2024-09-17T07:30:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074425142999999999, 2024-09-17T07:30:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19993 2024-09-17T07:30:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:30:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:30:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252589+0000", 2024-09-17T07:30:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.33e-07 2024-09-17T07:30:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252593+0000", 2024-09-17T07:30:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:30:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252595+0000", 2024-09-17T07:30:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.621e-06 2024-09-17T07:30:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326666+0000", 2024-09-17T07:30:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074070787999999999 2024-09-17T07:30:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326752+0000", 2024-09-17T07:30:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5748000000000006e-05 2024-09-17T07:30:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327013+0000", 2024-09-17T07:30:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026098400000000002 2024-09-17T07:30:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19994 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7340032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.549995012, 2024-09-17T07:30:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074765920999999999, 2024-09-17T07:30:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19994 2024-09-17T07:30:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:30:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:30:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252599+0000", 2024-09-17T07:30:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2099999999999995e-07 2024-09-17T07:30:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252603+0000", 2024-09-17T07:30:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:30:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252606+0000", 2024-09-17T07:30:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-09-17T07:30:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327047+0000", 2024-09-17T07:30:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074441050999999994 2024-09-17T07:30:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327121+0000", 2024-09-17T07:30:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4283999999999996e-05 2024-09-17T07:30:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327364+0000", 2024-09-17T07:30:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000243155 2024-09-17T07:30:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19995 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7569408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.549984315, 2024-09-17T07:30:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075172694999999998, 2024-09-17T07:30:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19995 2024-09-17T07:30:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:30:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:30:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252610+0000", 2024-09-17T07:30:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6799999999999999e-07 2024-09-17T07:30:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252614+0000", 2024-09-17T07:30:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000002e-06 2024-09-17T07:30:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252616+0000", 2024-09-17T07:30:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.419e-06 2024-09-17T07:30:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327407+0000", 2024-09-17T07:30:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074790617000000004 2024-09-17T07:30:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327498+0000", 2024-09-17T07:30:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0758000000000001e-05 2024-09-17T07:30:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327782+0000", 2024-09-17T07:30:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028395599999999998 2024-09-17T07:30:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7798784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.549973999, 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075605832999999997, 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:30:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:30:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:30:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:30:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:30:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:30:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:30:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:30:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327844+0000", 2024-09-17T07:30:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075217709999999993 2024-09-17T07:30:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327927+0000", 2024-09-17T07:30:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2628999999999998e-05 2024-09-17T07:30:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328225+0000", 2024-09-17T07:30:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029800600000000002 2024-09-17T07:30:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8028160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.54996364599999, 2024-09-17T07:30:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076061237000000004, 2024-09-17T07:30:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:30:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:30:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:30:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:30:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:30:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:30:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:30:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:30:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328268+0000", 2024-09-17T07:30:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075626323999999995 2024-09-17T07:30:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328352+0000", 2024-09-17T07:30:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3578999999999994e-05 2024-09-17T07:30:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328691+0000", 2024-09-17T07:30:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033915099999999999 2024-09-17T07:30:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8257536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.549948195, 2024-09-17T07:30:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076441903000000005, 2024-09-17T07:30:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:30:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:30:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:30:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:30:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:30:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:30:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:30:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:30:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:30:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328747+0000", 2024-09-17T07:30:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076094192000000005 2024-09-17T07:30:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328824+0000", 2024-09-17T07:30:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7096999999999997e-05 2024-09-17T07:30:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.329087+0000", 2024-09-17T07:30:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026249999999999998 2024-09-17T07:30:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7356416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.53983203600001, 2024-09-17T07:30:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072172616999999994, 2024-09-17T07:30:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:30:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:30:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:30:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:30:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:30:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:30:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:30:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:30:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334558+0000", 2024-09-17T07:30:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071780790999999997 2024-09-17T07:30:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334647+0000", 2024-09-17T07:30:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9448000000000001e-05 2024-09-17T07:30:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334934+0000", 2024-09-17T07:30:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028654599999999999 2024-09-17T07:30:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7585792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.539810324, 2024-09-17T07:30:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072606135000000002, 2024-09-17T07:30:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:30:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:30:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:30:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:30:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:30:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335116+0000", 2024-09-17T07:30:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072323897999999998 2024-09-17T07:30:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335166+0000", 2024-09-17T07:30:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9882000000000001e-05 2024-09-17T07:30:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335389+0000", 2024-09-17T07:30:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022334300000000001 2024-09-17T07:30:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7815168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.539798125, 2024-09-17T07:30:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073051230999999994, 2024-09-17T07:30:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:30:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:30:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:30:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:30:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:30:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:30:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:30:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335431+0000", 2024-09-17T07:30:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072627852000000007 2024-09-17T07:30:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335525+0000", 2024-09-17T07:30:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3708000000000005e-05 2024-09-17T07:30:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335846+0000", 2024-09-17T07:30:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032182000000000001 2024-09-17T07:30:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.53978726699999, 2024-09-17T07:30:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073284199999999994, 2024-09-17T07:30:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:30:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:30:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:30:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:30:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:30:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:30:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:30:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:30:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:30:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335863+0000", 2024-09-17T07:30:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073048921000000003 2024-09-17T07:30:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335904+0000", 2024-09-17T07:30:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1545000000000003e-05 2024-09-17T07:30:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336090+0000", 2024-09-17T07:30:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185787 2024-09-17T07:30:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8273920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:30:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.53977300699999, 2024-09-17T07:30:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073670478999999997, 2024-09-17T07:30:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:30:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:30:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:30:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:30:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:30:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:30:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:30:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:30:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:30:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336126+0000", 2024-09-17T07:30:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073297254000000006 2024-09-17T07:30:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336211+0000", 2024-09-17T07:30:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4728000000000006e-05 2024-09-17T07:30:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336491+0000", 2024-09-17T07:30:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027979399999999998 2024-09-17T07:30:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:116163 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-17T07:30:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:30:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.537438138999999, 2024-09-17T07:30:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8549867870000001, 2024-09-17T07:30:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116163 2024-09-17T07:30:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:30:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:30:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265157+0000", 2024-09-17T07:30:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-06 2024-09-17T07:30:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265168+0000", 2024-09-17T07:30:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0827e-05 2024-09-17T07:30:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265198+0000", 2024-09-17T07:30:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0383e-05 2024-09-17T07:30:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265212+0000", 2024-09-17T07:30:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3799e-05 2024-09-17T07:30:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265338+0000", 2024-09-17T07:30:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012576800000000001 2024-09-17T07:30:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265357+0000", 2024-09-17T07:30:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9417000000000001e-05 2024-09-17T07:30:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:27.083102+0000", 2024-09-17T07:30:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.81774456299999998 2024-09-17T07:30:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:27.083113+0000", 2024-09-17T07:30:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1154e-05 2024-09-17T07:30:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:28.119793+0000", 2024-09-17T07:30:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.036680365 2024-09-17T07:30:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:28.119943+0000", 2024-09-17T07:30:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014940999999999999 2024-09-17T07:30:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:28.120142+0000", 2024-09-17T07:30:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00019897099999999999 2024-09-17T07:30:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:49.896 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:49.897 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:49.898 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:49.898+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:49.999 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:49.999+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:50.041 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:50.041 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:50.041 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:50.041 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:50.049 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:50.100 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:50.099+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:50.152 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:50.152 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:50.152 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:50.152 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:50.152 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:50.160 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:50.201 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:50.200+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:50.278 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:50.278 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.37125927300001, 2024-09-17T07:30:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:30:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:30:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:30:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:30:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:30:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:30:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:30:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:30:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:30:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:30:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:30:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:30:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:30:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:30:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:30:50.282 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.37124059600001, 2024-09-17T07:30:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:30:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:30:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:30:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:30:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:30:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:30:50.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:30:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:30:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:30:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:30:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:30:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:30:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:30:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:30:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:30:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:50.287 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.37122224300001, 2024-09-17T07:30:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:30:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:30:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:50.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:30:50.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:30:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:30:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:30:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:30:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:30:50.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:30:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:30:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:30:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:30:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:30:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:30:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:30:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.371202645, 2024-09-17T07:30:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:30:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.292 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:30:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:30:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:30:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:30:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:30:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:30:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:30:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:30:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:30:50.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:30:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:30:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:30:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:30:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:30:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:30:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.37118269, 2024-09-17T07:30:50.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:30:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:30:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:30:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:30:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:30:50.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:30:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:30:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:30:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:30:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:30:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:30:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:30:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:30:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:30:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:30:50.301 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.37116403500001, 2024-09-17T07:30:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:30:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.302 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:30:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:30:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:30:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:30:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:30:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:30:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:30:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:30:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:30:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:30:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:30:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:30:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:30:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:30:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:50.306 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.371145559, 2024-09-17T07:30:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:30:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:30:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:30:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:30:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:30:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:30:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:30:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:30:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:30:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:30:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:30:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:30:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:30:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:30:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:30:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:30:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.37112596, 2024-09-17T07:30:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:30:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.311 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:30:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:30:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:30:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:30:50.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:30:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:30:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:30:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:30:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:30:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:30:50.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:30:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:30:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:30:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:30:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.315 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.371107385, 2024-09-17T07:30:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:30:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:30:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:30:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:30:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:30:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:30:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:30:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:30:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:30:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:30:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:30:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:30:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:30:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:30:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:30:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:30:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.319 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.371084958, 2024-09-17T07:30:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:30:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:30:50.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:30:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:30:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:30:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:30:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:30:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:30:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:30:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:30:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:30:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:30:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:30:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:30:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.37105714099999, 2024-09-17T07:30:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:30:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.324 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:30:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:30:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:30:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:30:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:30:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:30:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:30:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:30:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:50.328 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.371035197, 2024-09-17T07:30:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:30:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:30:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:30:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:30:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:30:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:30:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:30:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:30:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:30:50.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:30:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:30:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:30:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:30:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:30:50.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:30:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.371014093, 2024-09-17T07:30:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:30:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.333 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:30:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:30:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:30:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:30:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:30:50.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:30:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:30:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:30:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:30:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:30:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:30:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:30:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:30:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.37099449, 2024-09-17T07:30:50.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:30:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:30:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:30:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:30:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:30:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:30:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:30:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:30:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:30:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:30:50.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:30:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:30:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:30:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:30:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.341 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.418053377000007, 2024-09-17T07:30:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:30:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:30:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:30:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:30:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:30:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:30:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:30:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:30:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:30:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:30:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:30:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:30:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:30:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.418039716999999, 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:30:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:30:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:30:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:30:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:30:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:30:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:30:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:30:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:30:50.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:30:50.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:30:50.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:30:50.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:30:50.350 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.350 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.350 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.418024935000005, 2024-09-17T07:30:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:30:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:30:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.351 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:30:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:30:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:30:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:30:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:30:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:30:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:30:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:30:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:30:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:30:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:30:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:30:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.418014044000003, 2024-09-17T07:30:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:30:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:30:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:30:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:30:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:30:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:30:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:30:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:30:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:30:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:30:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:30:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:30:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:30:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:30:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.418004483999994, 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:30:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:30:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:30:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:30:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:30:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:30:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:30:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:30:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:30:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:30:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:30:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:30:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:30:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.417993924000001, 2024-09-17T07:30:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:30:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:30:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:30:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:30:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:30:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:30:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:30:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:30:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:30:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:30:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:30:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:30:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:30:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:30:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:30:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:30:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.368 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:50.370 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:50.370 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:50.369+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:50.470 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:50.470+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:50.514 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:50.514 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:50.514 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:50.514 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:50.522 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:50.572 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:50.572+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:50.665 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:50.665 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:50.665 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:50.665 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:50.665 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:50.673 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:50.672+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:50.674 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:50.773 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:50.773+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:50.777 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:50.777 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:50.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:50.777 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:50.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.777 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:30:50.778 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:50.778 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 144.44153403499999, 2024-09-17T07:30:50.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:30:50.778 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.778 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.778 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:50.778 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.778 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:30:50.778 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:30:50.778 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:30:50.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.778 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:50.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:50.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:30:50.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:30:50.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:30:50.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:30:50.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:30:50.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:30:50.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:30:50.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:30:50.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:30:50.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:30:50.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:50.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:30:50.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:30:50.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:30:50.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:30:50.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:30:50.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:30:50.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:50.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:30:50.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:30:50.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:50.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:30:50.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:30:50.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:50.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:30:50.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:30:50.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:50.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:30:50.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:30:50.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:30:50.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:30:50.784 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.784 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.784 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.784 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.784 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:50.784 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.852481885, 2024-09-17T07:30:50.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:30:50.784 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.784 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.785 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.785 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:30:50.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.785 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:50.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:50.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:30:50.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:30:50.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:30:50.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:30:50.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:30:50.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:30:50.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:30:50.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:30:50.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:30:50.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:30:50.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:30:50.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:30:50.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:30:50.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:30:50.788 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.788 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.788 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.789 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.789 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:50.789 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.847198769, 2024-09-17T07:30:50.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:30:50.789 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.789 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.789 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.789 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:30:50.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.790 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:50.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:50.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:50.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:30:50.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:30:50.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:30:50.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:30:50.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:30:50.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:30:50.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:30:50.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:30:50.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:30:50.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:30:50.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:30:50.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:30:50.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:30:50.793 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.793 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.793 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.793 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.793 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:50.793 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.847167789, 2024-09-17T07:30:50.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:30:50.794 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.794 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.794 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.794 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.794 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.794 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.796 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:30:50.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.796 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:50.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:50.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:30:50.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:30:50.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:30:50.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:30:50.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:30:50.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:30:50.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:30:50.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:30:50.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:30:50.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:30:50.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:30:50.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:30:50.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:30:50.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:30:50.799 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.799 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.799 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.800 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.800 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:50.800 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.84713773999999, 2024-09-17T07:30:50.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:30:50.800 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.800 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.800 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.800 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.800 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:30:50.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.800 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:50.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:50.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:30:50.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:30:50.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:30:50.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:30:50.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:30:50.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:30:50.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:30:50.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:30:50.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:30:50.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:30:50.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:30:50.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:30:50.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:30:50.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:30:50.805 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.805 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.805 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.805 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.805 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:50.805 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.84710933300001, 2024-09-17T07:30:50.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:30:50.805 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.806 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.806 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.806 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.806 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.806 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.806 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:30:50.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.806 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:50.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:50.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:50.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:50.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:30:50.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:30:50.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:30:50.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:30:50.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:30:50.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:30:50.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:30:50.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:30:50.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:30:50.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:30:50.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:30:50.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:30:50.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.810 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.810 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.810 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:50.810 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.847089512, 2024-09-17T07:30:50.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:30:50.811 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.811 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.811 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.811 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:30:50.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.811 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:50.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:50.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:30:50.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:30:50.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:30:50.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:30:50.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:30:50.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:30:50.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:30:50.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:30:50.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:30:50.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:30:50.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:30:50.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:30:50.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:30:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:30:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.84706525999999, 2024-09-17T07:30:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:30:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:30:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:30:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:30:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:30:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:30:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:30:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:30:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:30:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:30:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:30:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:30:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:30:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:30:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:30:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:30:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.84164267200001, 2024-09-17T07:30:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:30:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:30:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:30:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:30:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:30:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:30:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:30:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:30:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:30:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:30:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:30:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:30:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:30:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:30:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:30:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:30:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.841595797, 2024-09-17T07:30:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:30:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:30:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:30:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:30:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:30:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:30:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:30:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:30:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:30:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:30:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:30:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:30:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:30:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:30:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:30:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.841581093, 2024-09-17T07:30:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:30:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:30:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:30:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:30:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:30:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:30:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:30:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:30:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:30:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:30:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:30:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:30:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:30:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:30:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.841557998, 2024-09-17T07:30:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:30:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:30:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:50.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:30:50.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:30:50.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:30:50.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:30:50.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:30:50.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:30:50.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:30:50.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:30:50.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:30:50.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:30:50.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:30:50.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:30:50.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:30:50.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.839 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.839 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.839 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:50.840 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.84152662299999, 2024-09-17T07:30:50.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:30:50.840 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.840 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.840 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.840 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:30:50.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.840 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:50.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:50.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:30:50.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:30:50.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:30:50.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:30:50.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:30:50.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:30:50.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:30:50.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:30:50.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:30:50.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:30:50.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:30:50.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:30:50.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:30:50.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:30:50.843 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.843 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.843 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.844 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.844 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:50.844 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.841511363, 2024-09-17T07:30:50.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:30:50.844 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.844 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.844 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.844 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.845 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.845 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:30:50.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.845 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:50.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:50.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:50.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:30:50.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:30:50.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:30:50.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:30:50.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:30:50.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:30:50.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:30:50.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:30:50.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:30:50.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:30:50.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:30:50.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:30:50.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:30:50.849 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.849 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.849 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.849 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.849 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:50.849 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.841491293, 2024-09-17T07:30:50.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:30:50.849 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.850 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.850 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.850 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:30:50.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.850 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:50.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:50.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:50.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:30:50.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:30:50.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:30:50.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:30:50.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:30:50.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:30:50.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:30:50.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:30:50.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:30:50.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:30:50.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:30:50.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:30:50.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:30:50.856 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.856 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.856 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.857 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.857 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:50.857 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.841479353, 2024-09-17T07:30:50.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:30:50.857 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.857 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.857 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.857 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.857 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.857 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.857 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:30:50.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.857 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:50.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:50.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:50.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:30:50.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:30:50.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:30:50.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:30:50.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:30:50.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:30:50.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:30:50.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:30:50.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:30:50.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:30:50.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:30:50.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:30:50.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:30:50.861 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.862 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.862 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.862 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:50.862 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:50.862 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.84145619500001, 2024-09-17T07:30:50.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:30:50.862 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.863 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.863 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.863 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:30:50.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.863 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:50.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:50.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:30:50.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:30:50.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:30:50.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:30:50.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:30:50.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:30:50.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:30:50.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:30:50.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:30:50.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:30:50.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:30:50.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:30:50.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:30:50.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:30:50.866 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:115644 3.15 3:af37e0f6:::rbd_header.118311e0d6f8:head [watch reconnect cookie 140227662450992 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T07:30:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:30:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.698299300999999, 2024-09-17T07:30:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.013658722, 2024-09-17T07:30:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.867 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.868 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115644 2024-09-17T07:30:50.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.868 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:30:50.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:30:50.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077907+0000", 2024-09-17T07:30:50.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.08e-06 2024-09-17T07:30:50.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077916+0000", 2024-09-17T07:30:50.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-09-17T07:30:50.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077938+0000", 2024-09-17T07:30:50.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1883999999999999e-05 2024-09-17T07:30:50.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077950+0000", 2024-09-17T07:30:50.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2298e-05 2024-09-17T07:30:50.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.078037+0000", 2024-09-17T07:30:50.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6726999999999998e-05 2024-09-17T07:30:50.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-17T07:30:50.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.078046+0000", 2024-09-17T07:30:50.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6010000000000001e-06 2024-09-17T07:30:50.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085705+0000", 2024-09-17T07:30:50.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007659084 2024-09-17T07:30:50.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:50.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085718+0000", 2024-09-17T07:30:50.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2081e-05 2024-09-17T07:30:50.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.089697+0000", 2024-09-17T07:30:50.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.003979373 2024-09-17T07:30:50.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.089875+0000", 2024-09-17T07:30:50.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017760000000000001 2024-09-17T07:30:50.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.091563+0000", 2024-09-17T07:30:50.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001688836 2024-09-17T07:30:50.873 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.873 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.873 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.873 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:115645 3.15 3:af37e0f6:::rbd_header.118311e0d6f8:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T07:30:50.873 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:30:50.873 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 33.695481780999998, 2024-09-17T07:30:50.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.011649171, 2024-09-17T07:30:50.873 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115645 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:30:50.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080724+0000", 2024-09-17T07:30:50.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-17T07:30:50.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080735+0000", 2024-09-17T07:30:50.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1544e-05 2024-09-17T07:30:50.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080751+0000", 2024-09-17T07:30:50.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5244e-05 2024-09-17T07:30:50.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080761+0000", 2024-09-17T07:30:50.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0372e-05 2024-09-17T07:30:50.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080834+0000", 2024-09-17T07:30:50.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3263999999999995e-05 2024-09-17T07:30:50.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:50.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080842+0000", 2024-09-17T07:30:50.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2209999999999997e-06 2024-09-17T07:30:50.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085755+0000", 2024-09-17T07:30:50.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049130729999999996 2024-09-17T07:30:50.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:50.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085766+0000", 2024-09-17T07:30:50.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0288e-05 2024-09-17T07:30:50.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.090746+0000", 2024-09-17T07:30:50.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0049801190000001 2024-09-17T07:30:50.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.090798+0000", 2024-09-17T07:30:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2502e-05 2024-09-17T07:30:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-09-17T07:30:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.091037+0000", 2024-09-17T07:30:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0002385 2024-09-17T07:30:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092119+0000", 2024-09-17T07:30:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010820249999999999 2024-09-17T07:30:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:50.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092276+0000", 2024-09-17T07:30:50.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000156737 2024-09-17T07:30:50.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:50.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092330+0000", 2024-09-17T07:30:50.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4666999999999997e-05 2024-09-17T07:30:50.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092371+0000", 2024-09-17T07:30:50.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1043e-05 2024-09-17T07:30:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:116418 3.13 3:cdf2692d:::rbd_data.1183cb746bb8.0000000000000024:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T07:30:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 19.606065384000001, 2024-09-17T07:30:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92140365899999999, 2024-09-17T07:30:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:50.881 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116418 2024-09-17T07:30:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:50.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170140+0000", 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.398e-06 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170144+0000", 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4399999999999998e-06 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170157+0000", 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2133999999999999e-05 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170164+0000", 2024-09-17T07:30:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7200000000000006e-06 2024-09-17T07:30:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170227+0000", 2024-09-17T07:30:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3035999999999998e-05 2024-09-17T07:30:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170238+0000", 2024-09-17T07:30:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0397000000000001e-05 2024-09-17T07:30:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086367+0000", 2024-09-17T07:30:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.916128889 2024-09-17T07:30:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086523+0000", 2024-09-17T07:30:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015638800000000001 2024-09-17T07:30:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:30:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086652+0000", 2024-09-17T07:30:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000128806 2024-09-17T07:30:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091261+0000", 2024-09-17T07:30:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046091020000000003 2024-09-17T07:30:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091449+0000", 2024-09-17T07:30:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000188486 2024-09-17T07:30:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091503+0000", 2024-09-17T07:30:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3254999999999997e-05 2024-09-17T07:30:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:50.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:50.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:50.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091542+0000", 2024-09-17T07:30:50.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9607999999999998e-05 2024-09-17T07:30:50.887 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.887 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.887 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.887 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:50.887 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:50.887 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:50.887 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:30:50.889 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:50.890 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:50.889+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:50.989 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:50.989+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:51.011 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:51.011 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:51.012 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:51.012 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:51.022 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:51.028 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:51.028 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:51.046 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:51.046 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:51.062 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:51.062 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:51.079 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:51.079 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:51.090 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:51.090+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:51.096 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:51.097 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:51.113 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:51.131 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:51.131 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:51.131 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:51.140 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:51.148 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:51.148 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:51.201 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:51.200+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:51.204 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T07:30:51.265 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:51.265 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:51.278 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:51.292 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:51.292+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:51.393 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:51.393+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:51.394 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:51.395 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:51.403 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:51.443 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 5, 6, 7, 3, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-17T07:30:51.444 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:30:51.444 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-17T07:30:51.444 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-17T07:30:51.494 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:51.493+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:51.511 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:51.512 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:51.521 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:51.595 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:51.594+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:51.630 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:51.630 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:51.639 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:51.695 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:51.695+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:51.749 INFO:teuthology.orchestra.run.smithi045.stdout:moved 0 / 82 (0%) 2024-09-17T07:30:51.749 INFO:teuthology.orchestra.run.smithi045.stdout:avg 10.25 2024-09-17T07:30:51.749 INFO:teuthology.orchestra.run.smithi045.stdout:stddev 3.30719 -> 3.30719 (expected baseline 2.99479) 2024-09-17T07:30:51.749 INFO:teuthology.orchestra.run.smithi045.stdout:min osd.2 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-17T07:30:51.749 INFO:teuthology.orchestra.run.smithi045.stdout:max osd.0 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-17T07:30:51.749 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:30:51.749 INFO:teuthology.orchestra.run.smithi045.stdout:oload 1000 2024-09-17T07:30:51.749 INFO:teuthology.orchestra.run.smithi045.stdout:max_change 3 2024-09-17T07:30:51.749 INFO:teuthology.orchestra.run.smithi045.stdout:max_change_osds 4 2024-09-17T07:30:51.750 INFO:teuthology.orchestra.run.smithi045.stdout:average_utilization 0.0007 2024-09-17T07:30:51.750 INFO:teuthology.orchestra.run.smithi045.stdout:overload_utilization 0.0066 2024-09-17T07:30:51.750 INFO:teuthology.orchestra.run.smithi045.stderr:no change 2024-09-17T07:30:51.762 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T07:30:51.762 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-17T07:30:51.788 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:51.788 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:51.796 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:51.796+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:51.797 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:51.897 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:51.896+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:51.947 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:51.948 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:51.956 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:51.998 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:51.998+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:52.085 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-17T07:30:52.085 INFO:teuthology.orchestra.run.smithi045.stdout:moved 1 / 82 (1.21951%) 2024-09-17T07:30:52.085 INFO:teuthology.orchestra.run.smithi045.stdout:avg 10.25 2024-09-17T07:30:52.085 INFO:teuthology.orchestra.run.smithi045.stdout:stddev 3.30719 -> 3.11247 (expected baseline 2.99479) 2024-09-17T07:30:52.085 INFO:teuthology.orchestra.run.smithi045.stdout:min osd.2 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-17T07:30:52.085 INFO:teuthology.orchestra.run.smithi045.stdout:max osd.0 with 15 -> 14 pgs (1.46341 -> 1.36585 * mean) 2024-09-17T07:30:52.085 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:30:52.085 INFO:teuthology.orchestra.run.smithi045.stdout:oload 110 2024-09-17T07:30:52.086 INFO:teuthology.orchestra.run.smithi045.stdout:max_change 0.05 2024-09-17T07:30:52.086 INFO:teuthology.orchestra.run.smithi045.stdout:max_change_osds 4 2024-09-17T07:30:52.086 INFO:teuthology.orchestra.run.smithi045.stdout:average_utilization 0.0007 2024-09-17T07:30:52.086 INFO:teuthology.orchestra.run.smithi045.stdout:overload_utilization 0.0007 2024-09-17T07:30:52.086 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 weight 0.6866 -> 0.6366 2024-09-17T07:30:52.086 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 weight 1.0000 -> 0.9500 2024-09-17T07:30:52.098 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:52.098+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:52.112 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-17T07:30:52.112 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-17T07:30:52.115 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:52.115 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:52.126 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:52.210 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:52.210+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:52.278 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:52.278 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:52.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:52.329 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:52.328+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:52.402 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:52.401+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:52.431 INFO:teuthology.orchestra.run.smithi045.stdout:moved 0 / 82 (0%) 2024-09-17T07:30:52.431 INFO:teuthology.orchestra.run.smithi045.stdout:avg 10.25 2024-09-17T07:30:52.431 INFO:teuthology.orchestra.run.smithi045.stdout:stddev 3.11247 -> 3.11247 (expected baseline 2.99479) 2024-09-17T07:30:52.431 INFO:teuthology.orchestra.run.smithi045.stdout:min osd.2 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-17T07:30:52.431 INFO:teuthology.orchestra.run.smithi045.stdout:max osd.0 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-17T07:30:52.431 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:30:52.431 INFO:teuthology.orchestra.run.smithi045.stdout:oload 1000 2024-09-17T07:30:52.432 INFO:teuthology.orchestra.run.smithi045.stdout:max_change 0.05 2024-09-17T07:30:52.432 INFO:teuthology.orchestra.run.smithi045.stdout:max_change_osds 4 2024-09-17T07:30:52.432 INFO:teuthology.orchestra.run.smithi045.stdout:average_utilization 0.0007 2024-09-17T07:30:52.432 INFO:teuthology.orchestra.run.smithi045.stdout:overload_utilization 0.0066 2024-09-17T07:30:52.432 INFO:teuthology.orchestra.run.smithi045.stderr:no change 2024-09-17T07:30:52.437 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:52.437 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:52.445 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-17T07:30:52.445 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-17T07:30:52.446 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:52.530 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:52.530+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:52.596 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:52.596 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:52.603 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:52.603+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:52.607 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:52.704 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:52.704+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:52.716 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:52.716 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:52.725 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:52.791 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-09-17T07:30:52.790+0000 7f9bb64ca640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-17T07:30:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:moved 2 / 82 (2.43902%) 2024-09-17T07:30:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:avg 10.25 2024-09-17T07:30:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:stddev 3.11247 -> 2.58602 (expected baseline 2.99479) 2024-09-17T07:30:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:min osd.2 with 6 -> 8 pgs (0.585366 -> 0.780488 * mean) 2024-09-17T07:30:52.791 INFO:teuthology.orchestra.run.smithi045.stdout:max osd.0 with 14 -> 12 pgs (1.36585 -> 1.17073 * mean) 2024-09-17T07:30:52.792 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:30:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:oload 110 2024-09-17T07:30:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:max_change 0.05 2024-09-17T07:30:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:max_change_osds 4 2024-09-17T07:30:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:average_utilization 0.0007 2024-09-17T07:30:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:overload_utilization 0.0007 2024-09-17T07:30:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 weight 0.6366 -> 0.5866 2024-09-17T07:30:52.792 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 weight 0.9500 -> 0.9000 2024-09-17T07:30:52.792 INFO:teuthology.orchestra.run.smithi045.stderr:no change 2024-09-17T07:30:52.805 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-17T07:30:52.805 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-17T07:30:52.805 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:52.804+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:52.868 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:52.868 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:52.877 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:52.906 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:52.905+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:52.979 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:52.980 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:52.988 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:53.006 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:53.006+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:53.094 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:53.095 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:53.103 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:53.106 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:53.106+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:53.149 INFO:teuthology.orchestra.run.smithi045.stdout:moved 0 / 82 (0%) 2024-09-17T07:30:53.149 INFO:teuthology.orchestra.run.smithi045.stdout:avg 10.25 2024-09-17T07:30:53.149 INFO:teuthology.orchestra.run.smithi045.stdout:stddev 3.11247 -> 3.11247 (expected baseline 2.99479) 2024-09-17T07:30:53.149 INFO:teuthology.orchestra.run.smithi045.stdout:min osd.2 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-17T07:30:53.149 INFO:teuthology.orchestra.run.smithi045.stdout:max osd.0 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-17T07:30:53.149 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:30:53.149 INFO:teuthology.orchestra.run.smithi045.stdout:oload 1000 2024-09-17T07:30:53.149 INFO:teuthology.orchestra.run.smithi045.stdout:max_change 1 2024-09-17T07:30:53.149 INFO:teuthology.orchestra.run.smithi045.stdout:max_change_osds 4 2024-09-17T07:30:53.150 INFO:teuthology.orchestra.run.smithi045.stdout:average_utilization 0.0007 2024-09-17T07:30:53.150 INFO:teuthology.orchestra.run.smithi045.stdout:overload_utilization 0.0066 2024-09-17T07:30:53.150 INFO:teuthology.orchestra.run.smithi045.stderr:no change 2024-09-17T07:30:53.159 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:30:53.207 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:53.207 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:53.207 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:53.207+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:53.216 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:53.284 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:53.284 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:53.301 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:53.302 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:53.308 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:53.307+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:53.318 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:53.318 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:53.334 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:53.334 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:53.351 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:53.351 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:53.360 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:53.360 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:53.368 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:53.368 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:53.369 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:53.384 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:53.385 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:53.402 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:53.403 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:53.412 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:53.412+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:53.473 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:53.473 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:53.473 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:53.473 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:53.481 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:53.509 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:53.509+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:53.610 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:53.610+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:53.624 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:53.624 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:53.624 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:53.633 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:53.711 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:53.711+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:53.737 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:53.737 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:53.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:53.737 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:53.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.737 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:53.737 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:53.737 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.829773049, 2024-09-17T07:30:53.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:30:53.737 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.737 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:30:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:30:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:30:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:30:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:30:53.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:30:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:30:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:30:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:30:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:30:53.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:30:53.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:30:53.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:30:53.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:30:53.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:30:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.829754372, 2024-09-17T07:30:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:30:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.742 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:30:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:30:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:30:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:30:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:30:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:30:53.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:30:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:30:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:30:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:30:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:30:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:30:53.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:30:53.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:30:53.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:30:53.746 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.746 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.746 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.82973601899999, 2024-09-17T07:30:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:30:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:30:53.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:30:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:30:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:30:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:30:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:30:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:30:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:30:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:30:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:30:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:30:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:30:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:30:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:30:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:30:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.829716421, 2024-09-17T07:30:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:30:53.751 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:30:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:30:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:30:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:30:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:30:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:30:53.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:30:53.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:30:53.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:30:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:30:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:30:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:30:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:30:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:30:53.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:30:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:30:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.829696466, 2024-09-17T07:30:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:30:53.756 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.757 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.757 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.757 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:30:53.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.757 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:30:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:30:53.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:30:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:30:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:30:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:30:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:30:53.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:30:53.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:30:53.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:30:53.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:30:53.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:30:53.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:30:53.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:30:53.761 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.761 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.761 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.829677811, 2024-09-17T07:30:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:30:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.762 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:30:53.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.763 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:53.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:53.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:30:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:30:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:30:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:30:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:30:53.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:30:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:30:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:30:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:30:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:30:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:30:53.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:30:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:30:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:30:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:53.766 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.829659335, 2024-09-17T07:30:53.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:30:53.767 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.767 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.767 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.767 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.767 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.767 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.767 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:30:53.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.767 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:30:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:30:53.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:30:53.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:30:53.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:30:53.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:30:53.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:30:53.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:30:53.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:30:53.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:30:53.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:30:53.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:30:53.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:30:53.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:30:53.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:30:53.772 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.772 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.773 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.773 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:53.773 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:53.773 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.829639736, 2024-09-17T07:30:53.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:30:53.774 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.774 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.774 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.775 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:30:53.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.775 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:53.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:53.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:30:53.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:30:53.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:30:53.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:30:53.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:30:53.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:30:53.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:30:53.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:30:53.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:30:53.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:30:53.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:30:53.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:30:53.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:30:53.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:30:53.781 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.781 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.781 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.781 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:53.781 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:53.781 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.82962116100001, 2024-09-17T07:30:53.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:30:53.781 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.781 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.781 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.781 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.782 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:30:53.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.782 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:53.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:30:53.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:30:53.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:30:53.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:30:53.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:30:53.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:30:53.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:30:53.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:30:53.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:30:53.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:30:53.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:30:53.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:30:53.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:30:53.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:30:53.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:30:53.785 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.785 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.786 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.786 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:53.786 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:53.786 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.829598734, 2024-09-17T07:30:53.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:30:53.786 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.786 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.786 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.786 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.786 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.786 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.787 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:30:53.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.787 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:53.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:53.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:30:53.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:53.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:30:53.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:30:53.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:30:53.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:30:53.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:30:53.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:30:53.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:30:53.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:30:53.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:30:53.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:30:53.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:30:53.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:30:53.790 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.791 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.791 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.791 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:53.791 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:53.791 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.829570917, 2024-09-17T07:30:53.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:30:53.791 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.791 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.791 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.791 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.791 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.791 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.792 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:30:53.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.792 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:53.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:53.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:30:53.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:30:53.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:30:53.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:30:53.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:30:53.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:30:53.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:30:53.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:30:53.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:30:53.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:30:53.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:30:53.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:30:53.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:30:53.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:30:53.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.795 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.796 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:53.796 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:53.796 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.829548973, 2024-09-17T07:30:53.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:30:53.796 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.796 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.796 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.796 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:30:53.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.797 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:53.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:53.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:30:53.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:30:53.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:30:53.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:30:53.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:30:53.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:30:53.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:30:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:30:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:30:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:30:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:30:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:30:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:30:53.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:30:53.800 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.800 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.800 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.800 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:53.800 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:53.800 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.829527869, 2024-09-17T07:30:53.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:30:53.801 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.801 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.801 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.801 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.801 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.801 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.801 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:30:53.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.801 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:53.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:53.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:30:53.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:30:53.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:30:53.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:30:53.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:30:53.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:53.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:30:53.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:30:53.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:30:53.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:30:53.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:30:53.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:30:53.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:30:53.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:30:53.805 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.805 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.805 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.805 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:30:53.806 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:53.806 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.829508266, 2024-09-17T07:30:53.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:30:53.806 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.806 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.806 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.806 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.806 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.806 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.806 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:30:53.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.806 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:53.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:30:53.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:30:53.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:30:53.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:30:53.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:30:53.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:30:53.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:30:53.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:30:53.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:30:53.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:30:53.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:30:53.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:30:53.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:30:53.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:30:53.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:30:53.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.810 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.811 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:53.811 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:53.811 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.876567152999996, 2024-09-17T07:30:53.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:30:53.811 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.811 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.811 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.811 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:30:53.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.812 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:53.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:30:53.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:30:53.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:30:53.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:30:53.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:53.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:30:53.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:30:53.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:30:53.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:30:53.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:30:53.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:30:53.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:30:53.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:30:53.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:30:53.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:30:53.816 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.816 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.816 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.816 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:53.816 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:53.816 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.876553493000003, 2024-09-17T07:30:53.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:30:53.817 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.817 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.817 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.817 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.817 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.817 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.817 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:30:53.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.817 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:53.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:53.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:30:53.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:30:53.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:30:53.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:30:53.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:30:53.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:30:53.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:30:53.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:30:53.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:30:53.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:30:53.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:30:53.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:30:53.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:30:53.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:30:53.821 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.821 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.821 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.822 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:53.822 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:53.822 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.876538710999995, 2024-09-17T07:30:53.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:30:53.822 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.822 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.822 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.822 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:30:53.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.822 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:53.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:53.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:30:53.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:30:53.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:30:53.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:30:53.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:30:53.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:53.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:30:53.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:30:53.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:30:53.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:30:53.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:30:53.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:30:53.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:30:53.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:30:53.826 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.826 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.826 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.826 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:53.826 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:53.827 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.876527820000007, 2024-09-17T07:30:53.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:30:53.827 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.827 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.827 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.827 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.827 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.827 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.827 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:30:53.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.827 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:53.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:53.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:30:53.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:53.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:30:53.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:30:53.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:30:53.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:30:53.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:30:53.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:30:53.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:30:53.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:30:53.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:30:53.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:30:53.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:30:53.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:30:53.832 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.832 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.832 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.832 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:53.832 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:53.832 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.876518259999997, 2024-09-17T07:30:53.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:30:53.832 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.832 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.832 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.833 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.833 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.833 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:30:53.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.833 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:53.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:30:53.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:30:53.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:30:53.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:30:53.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:30:53.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:30:53.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:30:53.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:30:53.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:30:53.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:30:53.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:30:53.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:30:53.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:30:53.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:30:53.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:30:53.837 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.837 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.837 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.837 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:30:53.837 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:53.837 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.876507700000005, 2024-09-17T07:30:53.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:30:53.837 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:53.837 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:53.837 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:53.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:53.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:53.838 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:53.838 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:30:53.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.838 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:53.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:53.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:53.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:53.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:30:53.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:53.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:53.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:30:53.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:30:53.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:53.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:30:53.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:30:53.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:53.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:30:53.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:30:53.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:53.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:30:53.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:30:53.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:53.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:30:53.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:30:53.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:53.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:30:53.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:30:53.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:53.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:53.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:53.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:30:53.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:30:53.842 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.842 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.842 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.842 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:53.842 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:53.842 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:53.844 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:53.844 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:53.843+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:53.944 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:53.944+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:53.987 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:53.987 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:53.987 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:30:53.987 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:53.995 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:54.079 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:54.079+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:54.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T07:30:54.146 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:54.146+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:54.148 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:54.148 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:54.148 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:54.148 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:54.148 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:54.159 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:54.247 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:54.246+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:54.314 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:54.314 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:54.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:54.315 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:54.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.315 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:30:54.315 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:54.315 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.97815364499999, 2024-09-17T07:30:54.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:30:54.315 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.315 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.315 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.315 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.315 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:30:54.315 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:30:54.315 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:30:54.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.316 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:54.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:54.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:30:54.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:30:54.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:30:54.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:30:54.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:30:54.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:30:54.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:30:54.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:30:54.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:30:54.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:30:54.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:54.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:30:54.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:30:54.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:30:54.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:30:54.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:30:54.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:30:54.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:54.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:30:54.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:30:54.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:54.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:30:54.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:30:54.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:30:54.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:30:54.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:54.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:30:54.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:30:54.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:30:54.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:30:54.322 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.322 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.322 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.322 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.322 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:54.322 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.38910149500001, 2024-09-17T07:30:54.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:30:54.323 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.323 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.323 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.323 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.323 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.323 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.323 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:30:54.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.324 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:54.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:54.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:30:54.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:30:54.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:30:54.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:30:54.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:30:54.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:30:54.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:30:54.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:30:54.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:30:54.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:30:54.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:30:54.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:30:54.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:30:54.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:30:54.328 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.328 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.328 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.329 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.329 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:54.329 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.383818379, 2024-09-17T07:30:54.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:30:54.329 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.329 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.329 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.329 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:30:54.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.329 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:54.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:54.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:54.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:30:54.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:30:54.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:30:54.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:30:54.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:30:54.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:30:54.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:30:54.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:30:54.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:30:54.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:30:54.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:30:54.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:30:54.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:30:54.333 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.333 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.333 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.333 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.333 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:54.333 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.383787399, 2024-09-17T07:30:54.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:30:54.334 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.334 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.334 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.334 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.334 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.334 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.334 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:30:54.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.334 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:54.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:54.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:30:54.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:30:54.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:30:54.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:30:54.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:30:54.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:30:54.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:30:54.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:30:54.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:30:54.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:30:54.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:30:54.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:30:54.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:30:54.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:30:54.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.337 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.338 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.338 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:54.338 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.38375735, 2024-09-17T07:30:54.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:30:54.338 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.338 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.338 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.339 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.339 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:30:54.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.339 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:54.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:54.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:30:54.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:30:54.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:30:54.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:30:54.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:30:54.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:30:54.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:30:54.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:30:54.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:30:54.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:30:54.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:30:54.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:30:54.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:30:54.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:30:54.342 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.342 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.342 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.343 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.343 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:54.343 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.38372894299999, 2024-09-17T07:30:54.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:30:54.343 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.343 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.343 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.343 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.343 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.343 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.343 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:30:54.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.343 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:54.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:54.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:54.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:54.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:30:54.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:30:54.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:30:54.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:30:54.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:30:54.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:30:54.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:30:54.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:30:54.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:30:54.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:30:54.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:30:54.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:30:54.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.347 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.347 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.347 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:54.347 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.383709122, 2024-09-17T07:30:54.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:30:54.347 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.347 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.348 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.348 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:30:54.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.348 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:54.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:54.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:30:54.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:30:54.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:30:54.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:30:54.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:30:54.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:30:54.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:30:54.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:30:54.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:30:54.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:30:54.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:30:54.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:30:54.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:30:54.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:30:54.353 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.353 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.354 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.354 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.354 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:54.354 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.38368487, 2024-09-17T07:30:54.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:30:54.355 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.355 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.355 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.355 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.355 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.355 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.355 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:30:54.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.356 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:54.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:54.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:30:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:30:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:30:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:30:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:30:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:30:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:30:54.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:30:54.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:30:54.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:30:54.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:30:54.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:30:54.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:30:54.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:30:54.361 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.361 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.361 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.361 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.361 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:54.361 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.37826228199999, 2024-09-17T07:30:54.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:30:54.361 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:30:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:54.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:30:54.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:30:54.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:30:54.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:30:54.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:30:54.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:30:54.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:30:54.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:30:54.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:30:54.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:30:54.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:30:54.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:30:54.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:30:54.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:30:54.366 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.366 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.366 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.366 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.366 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:54.367 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.378215407, 2024-09-17T07:30:54.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:30:54.367 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.367 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.367 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.368 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.368 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:30:54.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.368 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:54.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:54.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:54.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:30:54.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:30:54.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:30:54.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:30:54.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:30:54.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:30:54.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:30:54.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:30:54.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:30:54.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:30:54.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:30:54.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:30:54.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:30:54.372 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.372 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.372 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.372 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.373 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:54.373 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.378200703, 2024-09-17T07:30:54.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:30:54.373 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.373 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.373 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.373 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.373 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:30:54.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.373 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:54.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:54.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:54.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:54.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:30:54.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:30:54.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:30:54.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:30:54.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:30:54.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:30:54.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:30:54.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:30:54.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:30:54.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:30:54.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:30:54.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:30:54.378 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.378 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.378 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.378 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.378 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:54.378 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.378177608, 2024-09-17T07:30:54.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:30:54.378 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.378 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.378 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.379 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.379 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.379 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.379 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:30:54.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.379 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:54.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:54.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:54.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:30:54.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:30:54.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:30:54.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:30:54.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:30:54.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:30:54.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:30:54.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:30:54.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:30:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:30:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:30:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:30:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:30:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.383 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.383 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:54.383 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.378146233, 2024-09-17T07:30:54.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:30:54.383 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.383 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.383 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.383 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:30:54.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.383 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:54.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:54.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:30:54.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:30:54.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:30:54.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:30:54.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:30:54.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:30:54.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:30:54.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:30:54.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:30:54.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:30:54.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:30:54.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:30:54.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:30:54.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:30:54.387 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.387 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.387 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.387 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.387 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:54.387 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.378130973, 2024-09-17T07:30:54.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:30:54.387 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.388 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.388 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.388 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.388 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.388 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.388 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:30:54.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.388 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:54.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:54.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:54.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:30:54.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:30:54.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:30:54.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:30:54.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:30:54.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:30:54.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:30:54.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:30:54.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.392 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.392 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:54.392 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.37811090300001, 2024-09-17T07:30:54.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:30:54.392 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.392 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.392 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.392 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.392 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.392 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.392 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:30:54.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.393 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:54.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:54.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.393 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:54.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:30:54.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:30:54.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:30:54.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:30:54.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:30:54.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:30:54.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:30:54.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:30:54.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:30:54.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:30:54.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:30:54.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:30:54.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:30:54.396 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.396 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.396 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.396 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.396 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:54.396 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.378098963, 2024-09-17T07:30:54.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:30:54.396 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.396 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.397 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.397 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.397 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.397 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.397 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:30:54.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.397 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:54.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:54.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:54.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:30:54.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:30:54.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:30:54.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:30:54.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:30:54.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:30:54.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:30:54.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:30:54.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:30:54.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:30:54.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:30:54.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:30:54.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:30:54.400 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.400 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.400 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.401 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:54.401 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:54.401 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.37807580499999, 2024-09-17T07:30:54.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:30:54.401 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.401 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.401 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.401 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.401 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.401 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.401 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:30:54.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.401 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:54.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:54.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:30:54.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:30:54.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:30:54.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:30:54.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:30:54.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:30:54.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:30:54.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:30:54.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:30:54.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:30:54.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:30:54.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:30:54.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:30:54.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:30:54.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.405 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.405 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:115644 3.15 3:af37e0f6:::rbd_header.118311e0d6f8:head [watch reconnect cookie 140227662450992 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T07:30:54.405 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:30:54.405 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.234918911000001, 2024-09-17T07:30:54.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.013658722, 2024-09-17T07:30:54.405 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.405 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.406 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:54.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.406 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115644 2024-09-17T07:30:54.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.406 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:30:54.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:30:54.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077907+0000", 2024-09-17T07:30:54.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.08e-06 2024-09-17T07:30:54.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077916+0000", 2024-09-17T07:30:54.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-09-17T07:30:54.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077938+0000", 2024-09-17T07:30:54.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1883999999999999e-05 2024-09-17T07:30:54.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077950+0000", 2024-09-17T07:30:54.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2298e-05 2024-09-17T07:30:54.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.078037+0000", 2024-09-17T07:30:54.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6726999999999998e-05 2024-09-17T07:30:54.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-17T07:30:54.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.078046+0000", 2024-09-17T07:30:54.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6010000000000001e-06 2024-09-17T07:30:54.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085705+0000", 2024-09-17T07:30:54.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007659084 2024-09-17T07:30:54.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085718+0000", 2024-09-17T07:30:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2081e-05 2024-09-17T07:30:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.089697+0000", 2024-09-17T07:30:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.003979373 2024-09-17T07:30:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.089875+0000", 2024-09-17T07:30:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017760000000000001 2024-09-17T07:30:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.091563+0000", 2024-09-17T07:30:54.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001688836 2024-09-17T07:30:54.411 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.411 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.411 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.411 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:115645 3.15 3:af37e0f6:::rbd_header.118311e0d6f8:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T07:30:54.411 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:30:54.411 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.232101391, 2024-09-17T07:30:54.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.011649171, 2024-09-17T07:30:54.411 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.411 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.412 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.412 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.412 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.412 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.412 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115645 2024-09-17T07:30:54.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.412 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:30:54.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:30:54.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080724+0000", 2024-09-17T07:30:54.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-17T07:30:54.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080735+0000", 2024-09-17T07:30:54.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1544e-05 2024-09-17T07:30:54.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080751+0000", 2024-09-17T07:30:54.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5244e-05 2024-09-17T07:30:54.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080761+0000", 2024-09-17T07:30:54.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0372e-05 2024-09-17T07:30:54.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080834+0000", 2024-09-17T07:30:54.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3263999999999995e-05 2024-09-17T07:30:54.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:54.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080842+0000", 2024-09-17T07:30:54.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2209999999999997e-06 2024-09-17T07:30:54.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085755+0000", 2024-09-17T07:30:54.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049130729999999996 2024-09-17T07:30:54.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:54.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085766+0000", 2024-09-17T07:30:54.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0288e-05 2024-09-17T07:30:54.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.090746+0000", 2024-09-17T07:30:54.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0049801190000001 2024-09-17T07:30:54.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.090798+0000", 2024-09-17T07:30:54.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2502e-05 2024-09-17T07:30:54.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-09-17T07:30:54.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.091037+0000", 2024-09-17T07:30:54.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0002385 2024-09-17T07:30:54.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092119+0000", 2024-09-17T07:30:54.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010820249999999999 2024-09-17T07:30:54.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:54.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092276+0000", 2024-09-17T07:30:54.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000156737 2024-09-17T07:30:54.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:54.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092330+0000", 2024-09-17T07:30:54.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4666999999999997e-05 2024-09-17T07:30:54.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092371+0000", 2024-09-17T07:30:54.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1043e-05 2024-09-17T07:30:54.419 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.419 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.419 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.419 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:116418 3.13 3:cdf2692d:::rbd_data.1183cb746bb8.0000000000000024:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T07:30:54.419 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:54.419 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.142684994, 2024-09-17T07:30:54.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92140365899999999, 2024-09-17T07:30:54.419 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:54.419 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:54.419 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.419 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:54.420 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:54.420 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.420 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116418 2024-09-17T07:30:54.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.420 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:54.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:54.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:54.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:54.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:54.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:54.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:54.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170140+0000", 2024-09-17T07:30:54.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.398e-06 2024-09-17T07:30:54.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:54.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170144+0000", 2024-09-17T07:30:54.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4399999999999998e-06 2024-09-17T07:30:54.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:54.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170157+0000", 2024-09-17T07:30:54.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2133999999999999e-05 2024-09-17T07:30:54.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170164+0000", 2024-09-17T07:30:54.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7200000000000006e-06 2024-09-17T07:30:54.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170227+0000", 2024-09-17T07:30:54.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3035999999999998e-05 2024-09-17T07:30:54.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:54.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170238+0000", 2024-09-17T07:30:54.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0397000000000001e-05 2024-09-17T07:30:54.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:54.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086367+0000", 2024-09-17T07:30:54.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.916128889 2024-09-17T07:30:54.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:54.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086523+0000", 2024-09-17T07:30:54.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015638800000000001 2024-09-17T07:30:54.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:30:54.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086652+0000", 2024-09-17T07:30:54.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000128806 2024-09-17T07:30:54.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:54.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091261+0000", 2024-09-17T07:30:54.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046091020000000003 2024-09-17T07:30:54.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091449+0000", 2024-09-17T07:30:54.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000188486 2024-09-17T07:30:54.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:54.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091503+0000", 2024-09-17T07:30:54.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3254999999999997e-05 2024-09-17T07:30:54.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:54.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:54.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:54.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091542+0000", 2024-09-17T07:30:54.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9607999999999998e-05 2024-09-17T07:30:54.426 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.426 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.426 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.426 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:54.426 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:54.426 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:54.427 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:54.428 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:54.427+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:54.528 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:54.528+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:54.534 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:54.535 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:54.535 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:54.535 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:54.543 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:54.629 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:54.629+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:54.691 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:54.691 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:54.691 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:54.691 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:54.691 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:54.699 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:54.740 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:54.740+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:54.831 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:54.831+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:54.849 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:54.849 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:54.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:54.850 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:54.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.850 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 108/106 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head, version: 36'126, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head@36'126, size: 9431460, copy_subset: [], clone_subset: {3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9: [(0, 9431460)]}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:54.850 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:54.850 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 37.643856174, 2024-09-17T07:30:54.850 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002812681, 2024-09-17T07:30:54.850 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.850 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.850 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:54.850 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.850 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.850 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.850 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203790+0000", 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9100000000000004e-07 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203801+0000", 2024-09-17T07:30:54.851 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0696000000000001e-05 2024-09-17T07:30:54.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203817+0000", 2024-09-17T07:30:54.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6480000000000001e-05 2024-09-17T07:30:54.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203829+0000", 2024-09-17T07:30:54.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1935e-05 2024-09-17T07:30:54.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.852 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.852 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.852 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203891+0000", 2024-09-17T07:30:54.852 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2079e-05 2024-09-17T07:30:54.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.203900+0000", 2024-09-17T07:30:54.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4999999999999999e-06 2024-09-17T07:30:54.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.853 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.853 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.853 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:17.206602+0000", 2024-09-17T07:30:54.853 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027024000000000002 2024-09-17T07:30:54.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.853 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.853 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.18 109/106 [PushOp(3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e, version: 44'134, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e@44'134, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:18.083461+0000", 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 36.764184698999998, 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003057976, 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083461+0000", 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.854 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083461+0000", 2024-09-17T07:30:54.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083462+0000", 2024-09-17T07:30:54.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1500000000000004e-07 2024-09-17T07:30:54.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.855 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083603+0000", 2024-09-17T07:30:54.855 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140863 2024-09-17T07:30:54.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.855 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.855 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083623+0000", 2024-09-17T07:30:54.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.003e-05 2024-09-17T07:30:54.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083642+0000", 2024-09-17T07:30:54.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9734000000000001e-05 2024-09-17T07:30:54.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.856 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.084564+0000", 2024-09-17T07:30:54.856 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00092196200000000002 2024-09-17T07:30:54.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.856 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.856 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.084580+0000", 2024-09-17T07:30:54.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5464e-05 2024-09-17T07:30:54.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.857 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.857 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.086519+0000", 2024-09-17T07:30:54.857 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001939208 2024-09-17T07:30:54.857 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.857 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.857 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.857 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.857 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115899 3.b 3:d406e0ff:::rbd_data.118340e4fbf.0000000000000001:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:54.857 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.656711027, 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0040641219999999999, 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115899 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:54.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190936+0000", 2024-09-17T07:30:54.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.26e-06 2024-09-17T07:30:54.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.859 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.859 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.859 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190941+0000", 2024-09-17T07:30:54.859 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0499999999999999e-06 2024-09-17T07:30:54.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190958+0000", 2024-09-17T07:30:54.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6548000000000001e-05 2024-09-17T07:30:54.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.860 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.860 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190966+0000", 2024-09-17T07:30:54.860 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6799999999999999e-06 2024-09-17T07:30:54.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.860 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191028+0000", 2024-09-17T07:30:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2082999999999995e-05 2024-09-17T07:30:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191161+0000", 2024-09-17T07:30:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000132461 2024-09-17T07:30:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191284+0000", 2024-09-17T07:30:54.861 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012301200000000001 2024-09-17T07:30:54.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192371+0000", 2024-09-17T07:30:54.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00108709 2024-09-17T07:30:54.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.862 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194898+0000", 2024-09-17T07:30:54.862 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025272710000000002 2024-09-17T07:30:54.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.862 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.862 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194957+0000", 2024-09-17T07:30:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8792000000000002e-05 2024-09-17T07:30:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194999+0000", 2024-09-17T07:30:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1875000000000001e-05 2024-09-17T07:30:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115903 3.18 3:1fc9b9ef:::rbd_data.118340e4fbf.0000000000000005:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:54.863 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.655912903999997, 2024-09-17T07:30:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00477466, 2024-09-17T07:30:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115903 2024-09-17T07:30:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.864 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:54.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.865 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:54.865 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.865 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.865 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:54.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9800000000000004e-07 2024-09-17T07:30:54.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191736+0000", 2024-09-17T07:30:54.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6620000000000001e-06 2024-09-17T07:30:54.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.866 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.866 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.866 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191743+0000", 2024-09-17T07:30:54.866 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6939999999999998e-06 2024-09-17T07:30:54.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191748+0000", 2024-09-17T07:30:54.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9749999999999998e-06 2024-09-17T07:30:54.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.867 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191823+0000", 2024-09-17T07:30:54.867 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4954000000000001e-05 2024-09-17T07:30:54.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.867 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.867 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191933+0000", 2024-09-17T07:30:54.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000110708 2024-09-17T07:30:54.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:30:54.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192031+0000", 2024-09-17T07:30:54.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7974999999999996e-05 2024-09-17T07:30:54.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.868 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.868 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193329+0000", 2024-09-17T07:30:54.868 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012976419999999999 2024-09-17T07:30:54.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.868 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196404+0000", 2024-09-17T07:30:54.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003075174 2024-09-17T07:30:54.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196461+0000", 2024-09-17T07:30:54.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6663999999999998e-05 2024-09-17T07:30:54.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.869 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.869 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196507+0000", 2024-09-17T07:30:54.869 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6714e-05 2024-09-17T07:30:54.869 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.870 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.870 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.870 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115905 3.18 3:19f08345:::rbd_data.118340e4fbf.0000000000000009:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:54.870 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:54.870 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 32.655882034000001, 2024-09-17T07:30:54.870 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0049339900000000001, 2024-09-17T07:30:54.870 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.870 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.870 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.870 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.871 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.871 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115905 2024-09-17T07:30:54.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.871 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:54.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.871 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.871 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:54.871 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:54.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0699999999999998e-07 2024-09-17T07:30:54.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191766+0000", 2024-09-17T07:30:54.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.928e-06 2024-09-17T07:30:54.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.872 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.872 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191773+0000", 2024-09-17T07:30:54.872 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2749999999999998e-06 2024-09-17T07:30:54.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191778+0000", 2024-09-17T07:30:54.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7849999999999999e-06 2024-09-17T07:30:54.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192756+0000", 2024-09-17T07:30:54.873 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097834399999999992 2024-09-17T07:30:54.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.873 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.873 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192868+0000", 2024-09-17T07:30:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111575 2024-09-17T07:30:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:30:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192970+0000", 2024-09-17T07:30:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010192300000000001 2024-09-17T07:30:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193865+0000", 2024-09-17T07:30:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089556100000000001 2024-09-17T07:30:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196583+0000", 2024-09-17T07:30:54.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027174180000000001 2024-09-17T07:30:54.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196648+0000", 2024-09-17T07:30:54.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4743e-05 2024-09-17T07:30:54.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.875 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.875 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196698+0000", 2024-09-17T07:30:54.875 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0031000000000003e-05 2024-09-17T07:30:54.875 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.875 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.876 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.876 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:116410 3.2 3:4721cecc:::rbd_data.1183cb746bb8.000000000000000a:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T07:30:54.876 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:31.167349+0000", 2024-09-17T07:30:54.876 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 23.680296201000001, 2024-09-17T07:30:54.876 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00284856, 2024-09-17T07:30:54.876 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.876 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.876 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.876 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.876 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 116410 2024-09-17T07:30:54.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.877 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167349+0000", 2024-09-17T07:30:54.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.877 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.877 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167349+0000", 2024-09-17T07:30:54.877 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167351+0000", 2024-09-17T07:30:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.359e-06 2024-09-17T07:30:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167354+0000", 2024-09-17T07:30:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5920000000000001e-06 2024-09-17T07:30:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167375+0000", 2024-09-17T07:30:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0488000000000001e-05 2024-09-17T07:30:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167388+0000", 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2618e-05 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167451+0000", 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3700000000000003e-05 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167768+0000", 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000316645 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.879 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:30:54.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.167882+0000", 2024-09-17T07:30:54.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011408000000000001 2024-09-17T07:30:54.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.168600+0000", 2024-09-17T07:30:54.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00071769200000000003 2024-09-17T07:30:54.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.880 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.880 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170093+0000", 2024-09-17T07:30:54.880 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001493767 2024-09-17T07:30:54.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170155+0000", 2024-09-17T07:30:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1779999999999995e-05 2024-09-17T07:30:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:31.170198+0000", 2024-09-17T07:30:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2839000000000001e-05 2024-09-17T07:30:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.881 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117007 3.8 3:11c9baa3:::rbd_object_map.118394b259f9:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-17T07:30:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:36.120261+0000", 2024-09-17T07:30:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 18.727384976, 2024-09-17T07:30:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045697289999999998, 2024-09-17T07:30:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117007 2024-09-17T07:30:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.882 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120261+0000", 2024-09-17T07:30:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120261+0000", 2024-09-17T07:30:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120262+0000", 2024-09-17T07:30:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.77e-06 2024-09-17T07:30:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120270+0000", 2024-09-17T07:30:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0080000000000002e-06 2024-09-17T07:30:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120287+0000", 2024-09-17T07:30:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6847e-05 2024-09-17T07:30:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120297+0000", 2024-09-17T07:30:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8830000000000001e-06 2024-09-17T07:30:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120368+0000", 2024-09-17T07:30:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0689e-05 2024-09-17T07:30:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:30:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120375+0000", 2024-09-17T07:30:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.187e-06 2024-09-17T07:30:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.122574+0000", 2024-09-17T07:30:54.885 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021992980000000001 2024-09-17T07:30:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.122670+0000", 2024-09-17T07:30:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5830999999999998e-05 2024-09-17T07:30:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:30:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.122866+0000", 2024-09-17T07:30:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000195303 2024-09-17T07:30:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124084+0000", 2024-09-17T07:30:54.886 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001218076 2024-09-17T07:30:54.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124153+0000", 2024-09-17T07:30:54.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9839999999999995e-05 2024-09-17T07:30:54.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124784+0000", 2024-09-17T07:30:54.887 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063082299999999997 2024-09-17T07:30:54.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.887 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.887 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.887 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124830+0000", 2024-09-17T07:30:54.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6174000000000002e-05 2024-09-17T07:30:54.888 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.888 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.888 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.888 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.888 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117147 3.7 3:e1d03a89:::rbd_header.11832ac10dcc:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T07:30:54.888 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.085788+0000", 2024-09-17T07:30:54.888 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.761857500000001, 2024-09-17T07:30:54.888 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0032128040000000001, 2024-09-17T07:30:54.888 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.888 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.888 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.888 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.888 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.889 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.889 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117147 2024-09-17T07:30:54.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.889 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085788+0000", 2024-09-17T07:30:54.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.889 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.889 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085788+0000", 2024-09-17T07:30:54.889 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.889 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085790+0000", 2024-09-17T07:30:54.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1179999999999999e-06 2024-09-17T07:30:54.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085798+0000", 2024-09-17T07:30:54.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1300000000000001e-06 2024-09-17T07:30:54.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.890 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.890 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085812+0000", 2024-09-17T07:30:54.890 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4065e-05 2024-09-17T07:30:54.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085824+0000", 2024-09-17T07:30:54.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1307e-05 2024-09-17T07:30:54.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085883+0000", 2024-09-17T07:30:54.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9141999999999997e-05 2024-09-17T07:30:54.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.891 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.891 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085985+0000", 2024-09-17T07:30:54.891 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000101895 2024-09-17T07:30:54.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.891 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:54.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.086372+0000", 2024-09-17T07:30:54.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00038743600000000001 2024-09-17T07:30:54.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.087110+0000", 2024-09-17T07:30:54.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000737887 2024-09-17T07:30:54.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.892 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.892 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.088882+0000", 2024-09-17T07:30:54.892 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017721600000000001 2024-09-17T07:30:54.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.892 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.088947+0000", 2024-09-17T07:30:54.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4719000000000002e-05 2024-09-17T07:30:54.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.893 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.893 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.089001+0000", 2024-09-17T07:30:54.893 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3944999999999999e-05 2024-09-17T07:30:54.893 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.893 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.893 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.893 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.893 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117151 3.7 3:e1d03a89:::rbd_header.11832ac10dcc:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T07:30:54.893 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.097104+0000", 2024-09-17T07:30:54.894 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.75054132, 2024-09-17T07:30:54.894 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062858530000000001, 2024-09-17T07:30:54.894 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.894 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.894 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.894 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.894 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.894 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.894 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117151 2024-09-17T07:30:54.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.894 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.894 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.894 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.894 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097104+0000", 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097104+0000", 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097107+0000", 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7429999999999998e-06 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097121+0000", 2024-09-17T07:30:54.895 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3774000000000001e-05 2024-09-17T07:30:54.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097150+0000", 2024-09-17T07:30:54.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8762e-05 2024-09-17T07:30:54.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.896 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097163+0000", 2024-09-17T07:30:54.896 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3631e-05 2024-09-17T07:30:54.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.896 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.896 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097245+0000", 2024-09-17T07:30:54.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1874999999999998e-05 2024-09-17T07:30:54.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097337+0000", 2024-09-17T07:30:54.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1465000000000006e-05 2024-09-17T07:30:54.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.897 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:54.897 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097553+0000", 2024-09-17T07:30:54.897 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021630999999999999 2024-09-17T07:30:54.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.897 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.098320+0000", 2024-09-17T07:30:54.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076674000000000002 2024-09-17T07:30:54.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.103265+0000", 2024-09-17T07:30:54.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0049452250000000001 2024-09-17T07:30:54.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.898 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.898 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.898 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.103335+0000", 2024-09-17T07:30:54.898 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0463999999999995e-05 2024-09-17T07:30:54.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.899 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.899 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.103390+0000", 2024-09-17T07:30:54.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4864000000000002e-05 2024-09-17T07:30:54.899 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.899 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.899 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.899 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.899 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117237 3.2 3:40c8b221:::rbd_data.11832ac10dcc.0000000000000000:head [delete] snapc 1d=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T07:30:54.899 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.169608+0000", 2024-09-17T07:30:54.899 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.678037323000002, 2024-09-17T07:30:54.899 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003512509, 2024-09-17T07:30:54.899 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.900 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.900 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.900 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.900 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117237 2024-09-17T07:30:54.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.900 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.900 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.900 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.900 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169608+0000", 2024-09-17T07:30:54.900 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169608+0000", 2024-09-17T07:30:54.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169610+0000", 2024-09-17T07:30:54.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8020000000000001e-06 2024-09-17T07:30:54.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.901 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.901 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169618+0000", 2024-09-17T07:30:54.901 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0080000000000002e-06 2024-09-17T07:30:54.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169635+0000", 2024-09-17T07:30:54.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6611000000000001e-05 2024-09-17T07:30:54.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169643+0000", 2024-09-17T07:30:54.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6549999999999993e-06 2024-09-17T07:30:54.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.902 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.902 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.902 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169723+0000", 2024-09-17T07:30:54.902 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0034e-05 2024-09-17T07:30:54.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169860+0000", 2024-09-17T07:30:54.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013645899999999999 2024-09-17T07:30:54.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:30:54.903 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169970+0000", 2024-09-17T07:30:54.903 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000109957 2024-09-17T07:30:54.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.903 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.903 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.171850+0000", 2024-09-17T07:30:54.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001879868 2024-09-17T07:30:54.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172391+0000", 2024-09-17T07:30:54.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054084100000000002 2024-09-17T07:30:54.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.904 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.904 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.904 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172439+0000", 2024-09-17T07:30:54.904 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7916e-05 2024-09-17T07:30:54.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.905 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.905 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173121+0000", 2024-09-17T07:30:54.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00068235799999999999 2024-09-17T07:30:54.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.905 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.905 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.905 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.905 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117245 3.1e 3:7b680344:::rbd_data.11832ac10dcc.0000000000000008:head [delete] snapc 1d=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T07:30:54.905 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.169745+0000", 2024-09-17T07:30:54.905 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.677901043999999, 2024-09-17T07:30:54.905 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002854868, 2024-09-17T07:30:54.906 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.906 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.906 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.906 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.906 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.906 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.906 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117245 2024-09-17T07:30:54.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.906 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.906 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.906 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169745+0000", 2024-09-17T07:30:54.906 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169745+0000", 2024-09-17T07:30:54.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169745+0000", 2024-09-17T07:30:54.907 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7199999999999999e-07 2024-09-17T07:30:54.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.907 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.907 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.907 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169748+0000", 2024-09-17T07:30:54.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5330000000000001e-06 2024-09-17T07:30:54.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.908 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.908 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169757+0000", 2024-09-17T07:30:54.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6260000000000007e-06 2024-09-17T07:30:54.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.908 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.908 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169763+0000", 2024-09-17T07:30:54.908 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0780000000000001e-06 2024-09-17T07:30:54.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.170431+0000", 2024-09-17T07:30:54.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00066795099999999996 2024-09-17T07:30:54.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.170528+0000", 2024-09-17T07:30:54.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6408000000000003e-05 2024-09-17T07:30:54.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.909 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.909 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-17T07:30:54.909 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.170685+0000", 2024-09-17T07:30:54.909 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015698 2024-09-17T07:30:54.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.171771+0000", 2024-09-17T07:30:54.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0010864970000000001 2024-09-17T07:30:54.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172509+0000", 2024-09-17T07:30:54.910 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00073733399999999997 2024-09-17T07:30:54.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.910 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.910 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.910 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172558+0000", 2024-09-17T07:30:54.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9249000000000002e-05 2024-09-17T07:30:54.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.911 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.911 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172600+0000", 2024-09-17T07:30:54.911 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1640000000000001e-05 2024-09-17T07:30:54.911 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.911 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.911 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.911 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:117250 3.5 e128/109 3:a4728dbe:::rbd_data.11832ac10dcc.0000000000000075:head v 128'521, mlcod=126'519)", 2024-09-17T07:30:54.911 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:30:54.911 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 17.674741793999999, 2024-09-17T07:30:54.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0034023949999999999, 2024-09-17T07:30:54.912 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.912 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.912 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.912 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.912 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.912 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:30:54.912 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.912 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.912 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.912 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:30:54.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.913 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:30:54.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5900000000000002e-07 2024-09-17T07:30:54.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.913 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172910+0000", 2024-09-17T07:30:54.913 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2449999999999998e-06 2024-09-17T07:30:54.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.913 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.913 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172919+0000", 2024-09-17T07:30:54.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5389999999999996e-06 2024-09-17T07:30:54.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.914 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173431+0000", 2024-09-17T07:30:54.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00051224699999999996 2024-09-17T07:30:54.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.914 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.914 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173485+0000", 2024-09-17T07:30:54.914 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3936e-05 2024-09-17T07:30:54.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.914 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.915 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.915 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173504+0000", 2024-09-17T07:30:54.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8681e-05 2024-09-17T07:30:54.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.915 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.915 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.176259+0000", 2024-09-17T07:30:54.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027547719999999999 2024-09-17T07:30:54.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.915 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.915 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.915 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.176306+0000", 2024-09-17T07:30:54.915 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7515999999999997e-05 2024-09-17T07:30:54.915 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.916 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.916 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.916 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117888 3.7 3:e7a2ce93:::rbd_data.118344704f8f.000000000000000a:head [delete] snapc 1a=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-17T07:30:54.916 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:44.169045+0000", 2024-09-17T07:30:54.916 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 10.678600238, 2024-09-17T07:30:54.916 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028982180000000001, 2024-09-17T07:30:54.916 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.916 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.916 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.916 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.916 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.917 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.917 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117888 2024-09-17T07:30:54.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.917 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.917 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.917 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:44.169045+0000", 2024-09-17T07:30:54.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.917 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.917 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.917 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:44.169045+0000", 2024-09-17T07:30:54.917 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:44.169047+0000", 2024-09-17T07:30:54.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.243e-06 2024-09-17T07:30:54.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:44.169052+0000", 2024-09-17T07:30:54.918 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4029999999999998e-06 2024-09-17T07:30:54.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.918 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.918 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.918 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:44.169067+0000", 2024-09-17T07:30:54.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4548e-05 2024-09-17T07:30:54.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:44.169074+0000", 2024-09-17T07:30:54.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7910000000000005e-06 2024-09-17T07:30:54.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.919 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.919 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:44.169144+0000", 2024-09-17T07:30:54.919 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9542000000000006e-05 2024-09-17T07:30:54.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.919 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.920 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.920 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:44.169254+0000", 2024-09-17T07:30:54.920 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000109586 2024-09-17T07:30:54.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.920 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:54.920 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:44.169356+0000", 2024-09-17T07:30:54.920 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000102309 2024-09-17T07:30:54.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.920 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.920 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.920 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:44.169841+0000", 2024-09-17T07:30:54.920 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00048551399999999999 2024-09-17T07:30:54.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:44.171844+0000", 2024-09-17T07:30:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020022379999999999 2024-09-17T07:30:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:44.171903+0000", 2024-09-17T07:30:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9110999999999997e-05 2024-09-17T07:30:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.921 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:44.171944+0000", 2024-09-17T07:30:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0933000000000003e-05 2024-09-17T07:30:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(osd.5.0:3808 3.5 e137/109 3:a7d05834:::rbd_data.1183ad87379e.000000000000001d:18 v 137'527, mlcod=129'524)", 2024-09-17T07:30:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:46.173929+0000", 2024-09-17T07:30:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 8.6737163079999995, 2024-09-17T07:30:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027628610000000001, 2024-09-17T07:30:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.922 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:46.173929+0000", 2024-09-17T07:30:54.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:46.173929+0000", 2024-09-17T07:30:54.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.923 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.923 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:46.173930+0000", 2024-09-17T07:30:54.923 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8999999999999996e-07 2024-09-17T07:30:54.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:46.173936+0000", 2024-09-17T07:30:54.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4860000000000002e-06 2024-09-17T07:30:54.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.924 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:46.173947+0000", 2024-09-17T07:30:54.924 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0737000000000001e-05 2024-09-17T07:30:54.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.924 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.924 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:46.173968+0000", 2024-09-17T07:30:54.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0341999999999999e-05 2024-09-17T07:30:54.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:46.174056+0000", 2024-09-17T07:30:54.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8432999999999998e-05 2024-09-17T07:30:54.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.925 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.925 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:46.174074+0000", 2024-09-17T07:30:54.925 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8077e-05 2024-09-17T07:30:54.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.925 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.926 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.926 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:46.176644+0000", 2024-09-17T07:30:54.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025698869999999999 2024-09-17T07:30:54.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.926 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.926 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:46.176692+0000", 2024-09-17T07:30:54.926 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8309000000000001e-05 2024-09-17T07:30:54.926 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.926 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.926 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.926 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.926 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:118889 3.b 3:d08c2e41:::rbd_data.11839a97b753.0000000000000002:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-17T07:30:54.926 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:50.259083+0000", 2024-09-17T07:30:54.927 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.5885628330000001, 2024-09-17T07:30:54.927 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060418879999999996, 2024-09-17T07:30:54.927 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.927 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.927 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.927 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.927 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.927 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 118889 2024-09-17T07:30:54.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.927 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.927 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.927 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.927 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259083+0000", 2024-09-17T07:30:54.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.928 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.928 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259083+0000", 2024-09-17T07:30:54.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.928 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.928 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259085+0000", 2024-09-17T07:30:54.928 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8509999999999999e-06 2024-09-17T07:30:54.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.929 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.929 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259092+0000", 2024-09-17T07:30:54.929 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9500000000000004e-06 2024-09-17T07:30:54.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.929 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.929 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259111+0000", 2024-09-17T07:30:54.929 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9165e-05 2024-09-17T07:30:54.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.929 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.929 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.929 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259122+0000", 2024-09-17T07:30:54.929 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0745e-05 2024-09-17T07:30:54.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259206+0000", 2024-09-17T07:30:54.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4741999999999996e-05 2024-09-17T07:30:54.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259345+0000", 2024-09-17T07:30:54.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000138644 2024-09-17T07:30:54.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.930 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.930 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:54.930 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259470+0000", 2024-09-17T07:30:54.930 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012535899999999999 2024-09-17T07:30:54.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.261565+0000", 2024-09-17T07:30:54.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002094561 2024-09-17T07:30:54.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265016+0000", 2024-09-17T07:30:54.931 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0034515240000000001 2024-09-17T07:30:54.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.931 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.931 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.931 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265080+0000", 2024-09-17T07:30:54.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3927e-05 2024-09-17T07:30:54.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.932 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.932 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265125+0000", 2024-09-17T07:30:54.932 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4419999999999998e-05 2024-09-17T07:30:54.932 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.932 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.932 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.932 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.932 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:118890 3.2 3:4647e1ae:::rbd_data.11839a97b753.0000000000000003:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-17T07:30:54.932 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:50.259150+0000", 2024-09-17T07:30:54.932 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.5884955109999996, 2024-09-17T07:30:54.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003910134, 2024-09-17T07:30:54.933 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.933 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.933 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.933 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.933 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.933 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.933 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 118890 2024-09-17T07:30:54.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.933 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.933 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.933 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259150+0000", 2024-09-17T07:30:54.933 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259150+0000", 2024-09-17T07:30:54.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259151+0000", 2024-09-17T07:30:54.934 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1999999999999999e-07 2024-09-17T07:30:54.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.934 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.934 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.934 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259154+0000", 2024-09-17T07:30:54.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5020000000000001e-06 2024-09-17T07:30:54.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259164+0000", 2024-09-17T07:30:54.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8190000000000003e-06 2024-09-17T07:30:54.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259171+0000", 2024-09-17T07:30:54.935 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9009999999999997e-06 2024-09-17T07:30:54.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.935 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.935 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.935 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.260418+0000", 2024-09-17T07:30:54.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001246891 2024-09-17T07:30:54.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.260587+0000", 2024-09-17T07:30:54.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000168796 2024-09-17T07:30:54.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:30:54.936 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.260773+0000", 2024-09-17T07:30:54.936 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000186269 2024-09-17T07:30:54.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.936 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.936 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262459+0000", 2024-09-17T07:30:54.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016858660000000001 2024-09-17T07:30:54.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262962+0000", 2024-09-17T07:30:54.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00050283499999999998 2024-09-17T07:30:54.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.937 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.937 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.263015+0000", 2024-09-17T07:30:54.937 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3115999999999998e-05 2024-09-17T07:30:54.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.937 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.938 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.938 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.263060+0000", 2024-09-17T07:30:54.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5519000000000002e-05 2024-09-17T07:30:54.938 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.938 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.938 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.938 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.938 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:118891 3.15 3:ae97839c:::rbd_data.11839a97b753.0000000000000004:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-17T07:30:54.938 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:50.259192+0000", 2024-09-17T07:30:54.938 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.5884540410000003, 2024-09-17T07:30:54.938 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031418380000000001, 2024-09-17T07:30:54.938 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.938 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.939 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.939 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.939 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.939 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 118891 2024-09-17T07:30:54.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.939 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259192+0000", 2024-09-17T07:30:54.939 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.939 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.939 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.939 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259192+0000", 2024-09-17T07:30:54.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259192+0000", 2024-09-17T07:30:54.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6199999999999997e-07 2024-09-17T07:30:54.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.940 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259195+0000", 2024-09-17T07:30:54.940 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.18e-06 2024-09-17T07:30:54.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.940 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259206+0000", 2024-09-17T07:30:54.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0291e-05 2024-09-17T07:30:54.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259212+0000", 2024-09-17T07:30:54.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2260000000000002e-06 2024-09-17T07:30:54.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.941 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.941 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259267+0000", 2024-09-17T07:30:54.941 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5217999999999999e-05 2024-09-17T07:30:54.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.941 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259393+0000", 2024-09-17T07:30:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012553299999999999 2024-09-17T07:30:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:30:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259494+0000", 2024-09-17T07:30:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000101643 2024-09-17T07:30:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.261587+0000", 2024-09-17T07:30:54.942 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020926959999999998 2024-09-17T07:30:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262247+0000", 2024-09-17T07:30:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00065972399999999999 2024-09-17T07:30:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262294+0000", 2024-09-17T07:30:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7410000000000002e-05 2024-09-17T07:30:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.943 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262333+0000", 2024-09-17T07:30:54.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9255000000000003e-05 2024-09-17T07:30:54.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.944 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.944 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:118893 3.7 3:e7df9326:::rbd_data.11839a97b753.0000000000000006:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-17T07:30:54.944 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:50.259231+0000", 2024-09-17T07:30:54.944 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.588414244, 2024-09-17T07:30:54.944 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0029118019999999998, 2024-09-17T07:30:54.944 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.944 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.944 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 118893 2024-09-17T07:30:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259231+0000", 2024-09-17T07:30:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259231+0000", 2024-09-17T07:30:54.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259232+0000", 2024-09-17T07:30:54.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0200000000000001e-07 2024-09-17T07:30:54.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259235+0000", 2024-09-17T07:30:54.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.101e-06 2024-09-17T07:30:54.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.946 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.946 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.946 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259243+0000", 2024-09-17T07:30:54.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8509999999999999e-06 2024-09-17T07:30:54.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259250+0000", 2024-09-17T07:30:54.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4309999999999999e-06 2024-09-17T07:30:54.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259313+0000", 2024-09-17T07:30:54.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3764999999999997e-05 2024-09-17T07:30:54.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259445+0000", 2024-09-17T07:30:54.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013151699999999999 2024-09-17T07:30:54.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:54.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259568+0000", 2024-09-17T07:30:54.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000123003 2024-09-17T07:30:54.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.261940+0000", 2024-09-17T07:30:54.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023719399999999999 2024-09-17T07:30:54.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262044+0000", 2024-09-17T07:30:54.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000104181 2024-09-17T07:30:54.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262096+0000", 2024-09-17T07:30:54.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2333999999999997e-05 2024-09-17T07:30:54.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262143+0000", 2024-09-17T07:30:54.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6977000000000001e-05 2024-09-17T07:30:54.950 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.950 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.950 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.950 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:118897 3.b 3:d71b53f6:::rbd_data.11839a97b753.000000000000000a:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-17T07:30:54.950 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:50.262853+0000", 2024-09-17T07:30:54.950 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.5847931710000003, 2024-09-17T07:30:54.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031177959999999999, 2024-09-17T07:30:54.950 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.950 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.950 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.951 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 118897 2024-09-17T07:30:54.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.951 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262853+0000", 2024-09-17T07:30:54.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262853+0000", 2024-09-17T07:30:54.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262854+0000", 2024-09-17T07:30:54.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.804e-06 2024-09-17T07:30:54.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262860+0000", 2024-09-17T07:30:54.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6729999999999997e-06 2024-09-17T07:30:54.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262872+0000", 2024-09-17T07:30:54.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2319e-05 2024-09-17T07:30:54.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262881+0000", 2024-09-17T07:30:54.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4290000000000001e-06 2024-09-17T07:30:54.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262948+0000", 2024-09-17T07:30:54.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7384000000000004e-05 2024-09-17T07:30:54.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.263080+0000", 2024-09-17T07:30:54.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013160400000000001 2024-09-17T07:30:54.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:54.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.263197+0000", 2024-09-17T07:30:54.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011731999999999999 2024-09-17T07:30:54.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.264313+0000", 2024-09-17T07:30:54.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001115731 2024-09-17T07:30:54.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265854+0000", 2024-09-17T07:30:54.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001541132 2024-09-17T07:30:54.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265913+0000", 2024-09-17T07:30:54.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8729e-05 2024-09-17T07:30:54.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265970+0000", 2024-09-17T07:30:54.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7670999999999997e-05 2024-09-17T07:30:54.955 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.955 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.955 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.956 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:118898 3.18 3:1ada4640:::rbd_data.11839a97b753.000000000000000b:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-17T07:30:54.956 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:50.262907+0000", 2024-09-17T07:30:54.956 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 4.5847390130000001, 2024-09-17T07:30:54.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003013637, 2024-09-17T07:30:54.956 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:54.956 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:54.956 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:54.956 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:54.956 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:54.956 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:54.956 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 118898 2024-09-17T07:30:54.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.957 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:54.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:54.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262907+0000", 2024-09-17T07:30:54.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:54.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262907+0000", 2024-09-17T07:30:54.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:54.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:54.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262907+0000", 2024-09-17T07:30:54.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4799999999999998e-07 2024-09-17T07:30:54.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:54.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262911+0000", 2024-09-17T07:30:54.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8789999999999996e-06 2024-09-17T07:30:54.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:54.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262920+0000", 2024-09-17T07:30:54.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2320000000000007e-06 2024-09-17T07:30:54.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:54.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262928+0000", 2024-09-17T07:30:54.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.96e-06 2024-09-17T07:30:54.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:54.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262991+0000", 2024-09-17T07:30:54.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2728e-05 2024-09-17T07:30:54.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:54.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.263117+0000", 2024-09-17T07:30:54.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012585100000000001 2024-09-17T07:30:54.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:54.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.263253+0000", 2024-09-17T07:30:54.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013641400000000001 2024-09-17T07:30:54.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:54.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.264559+0000", 2024-09-17T07:30:54.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013053940000000001 2024-09-17T07:30:54.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:54.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265805+0000", 2024-09-17T07:30:54.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001246149 2024-09-17T07:30:54.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:54.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265864+0000", 2024-09-17T07:30:54.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9341e-05 2024-09-17T07:30:54.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:54.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:54.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:54.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265920+0000", 2024-09-17T07:30:54.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6140999999999997e-05 2024-09-17T07:30:54.961 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.961 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.961 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:54.962 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:54.962 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:54.963 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:54.962+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:54.963 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:55.064 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:55.064+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:55.088 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-17T07:30:55.119 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:55.120 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:55.129 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:55.170 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:55.170+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:55.266 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:55.266+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:55.281 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:55.281 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:55.281 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:55.281 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:55.281 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:55.291 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:55.374 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:55.374+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:55.414 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:30:55.441 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:55.441 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:55.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:55.441 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:55.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.441 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:55.441 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:55.441 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 117.307059776, 2024-09-17T07:30:55.441 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:30:55.442 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.442 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.442 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:55.442 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.442 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.442 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.442 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:30:55.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.442 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:30:55.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:30:55.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.444 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.444 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:30:55.444 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:30:55.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:30:55.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:30:55.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.445 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:30:55.445 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:30:55.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.445 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:30:55.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:30:55.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:30:55.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:30:55.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:30:55.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.446 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.446 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:30:55.446 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:30:55.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:55.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:30:55.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:30:55.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:30:55.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:30:55.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.447 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:55.447 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:30:55.447 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:30:55.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.448 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.448 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:30:55.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:30:55.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.448 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.448 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.448 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:55.448 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.001093072, 2024-09-17T07:30:55.448 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:30:55.448 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.449 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:55.449 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.449 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.449 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.449 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:30:55.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.450 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.450 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.450 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.450 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:55.450 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.451 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:55.451 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.451 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.451 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:30:55.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:30:55.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.452 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.452 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.452 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:30:55.452 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:30:55.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.453 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.453 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:30:55.453 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:30:55.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.454 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:30:55.454 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:30:55.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.454 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.454 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:30:55.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:30:55.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.455 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:55.455 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:30:55.455 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:30:55.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.456 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.456 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:30:55.456 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:30:55.456 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.456 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.457 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.457 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.457 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.457 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:55.457 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.001070957, 2024-09-17T07:30:55.457 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:30:55.457 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.457 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.457 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.458 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.458 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.458 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:30:55.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.458 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.458 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.458 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.458 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:55.458 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:55.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:30:55.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:55.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.459 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.459 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.459 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:30:55.459 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:30:55.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:30:55.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:30:55.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:30:55.460 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:30:55.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.460 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.460 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:30:55.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:30:55.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.461 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.461 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:30:55.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:30:55.461 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.461 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.461 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.461 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.461 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:55.461 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.00104685300001, 2024-09-17T07:30:55.461 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:30:55.462 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.462 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.462 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.462 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.462 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.462 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.462 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:30:55.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.462 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.462 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.462 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:55.462 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.462 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:55.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:30:55.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:55.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.463 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.463 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:30:55.463 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:30:55.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.463 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:30:55.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:30:55.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:30:55.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:30:55.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.464 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.464 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:30:55.464 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:30:55.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.464 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.465 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.465 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:30:55.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:30:55.465 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.465 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.465 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.465 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.465 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:55.465 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.001022764, 2024-09-17T07:30:55.465 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:30:55.465 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.465 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.465 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.466 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.466 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.466 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.466 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:30:55.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.466 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:55.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.466 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.466 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:55.466 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:55.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:30:55.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.467 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.467 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:30:55.467 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:30:55.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.468 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.468 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:30:55.468 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:30:55.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.468 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.468 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:30:55.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:30:55.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:30:55.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:30:55.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.469 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.469 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:30:55.469 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:30:55.470 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.470 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.470 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.470 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.470 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.470 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:55.470 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.000998345, 2024-09-17T07:30:55.470 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:30:55.470 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.470 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.471 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.471 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.471 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.471 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:30:55.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.471 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:55.471 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.471 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.471 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:55.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:55.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:30:55.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.472 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:30:55.472 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:30:55.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.472 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.472 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:30:55.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:30:55.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:30:55.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:55.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.473 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.473 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:30:55.473 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:30:55.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.474 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.474 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:30:55.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:30:55.474 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.474 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.474 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.474 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.474 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.474 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:55.474 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.00097664499999, 2024-09-17T07:30:55.474 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:30:55.474 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.475 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.475 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.475 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.475 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.475 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.475 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:30:55.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.475 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.475 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.475 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:55.475 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:55.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:55.476 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:30:55.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.476 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.476 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.476 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:30:55.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:30:55.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:30:55.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:30:55.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.477 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:30:55.477 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:30:55.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.477 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.477 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:30:55.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:30:55.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.478 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.478 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:30:55.478 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:30:55.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.478 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.478 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.478 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:55.478 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.000931913, 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:55.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:30:55.480 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:30:55.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.480 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.480 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.480 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:30:55.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:30:55.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:30:55.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:30:55.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.481 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.481 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.481 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:30:55.481 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:30:55.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.482 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:55.483 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.000903447, 2024-09-17T07:30:55.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:30:55.483 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.483 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.483 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.483 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.483 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:30:55.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.483 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.483 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.483 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.483 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:55.483 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:55.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:55.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:30:55.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.484 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.484 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.484 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:30:55.484 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:30:55.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:30:55.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:30:55.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:30:55.485 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:30:55.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.485 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.485 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:30:55.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:30:55.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.486 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.486 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:30:55.486 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:30:55.486 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.486 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.486 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.486 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.486 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.486 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:55.486 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.000786195, 2024-09-17T07:30:55.487 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:30:55.487 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.487 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.487 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.487 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.487 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.487 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.487 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:30:55.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.487 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.487 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.488 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.488 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:55.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.488 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.488 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:55.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.488 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.488 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.488 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:30:55.488 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:30:55.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:30:55.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:30:55.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.489 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.489 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:30:55.489 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:30:55.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:30:55.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:30:55.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:30:55.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:30:55.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.490 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.490 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:30:55.490 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:30:55.491 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.491 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.491 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.491 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.491 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:55.491 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.000360093, 2024-09-17T07:30:55.491 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:30:55.491 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.492 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.492 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.492 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.492 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.492 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.492 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:30:55.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.492 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.492 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.492 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:55.492 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:55.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:30:55.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:55.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.493 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.493 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:30:55.493 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:30:55.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.494 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.494 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:30:55.494 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:30:55.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.495 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.496 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.496 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.496 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:55.496 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.00034042199999, 2024-09-17T07:30:55.496 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:30:55.496 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.496 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.496 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.496 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.496 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.496 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.496 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:30:55.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.497 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:55.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:55.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.497 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.497 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:55.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:30:55.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:30:55.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:30:55.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:30:55.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:30:55.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.498 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.498 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.498 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:30:55.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:30:55.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:30:55.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:30:55.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:30:55.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:30:55.499 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.499 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.499 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.00031755800001, 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:55.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:55.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.502 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.502 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:55.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:30:55.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:30:55.504 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:30:55.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.504 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:30:55.504 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:30:55.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.504 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:30:55.504 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:30:55.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.504 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.505 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:30:55.505 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:30:55.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.505 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.505 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:30:55.505 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:30:55.505 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.505 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.505 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.505 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.506 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:55.506 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.000296257, 2024-09-17T07:30:55.506 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:30:55.506 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.506 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.506 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.506 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.506 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.506 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:30:55.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.506 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.506 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.506 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.506 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:55.507 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.507 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.507 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:55.507 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.507 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.507 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:30:55.507 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:30:55.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.508 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.508 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:30:55.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:30:55.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.508 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.508 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:30:55.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:30:55.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.508 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.508 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:30:55.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:30:55.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.509 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.509 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:30:55.509 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:30:55.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.509 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.509 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:30:55.509 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:30:55.509 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.509 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.510 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.510 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.510 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:55.510 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.000248469, 2024-09-17T07:30:55.510 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:30:55.510 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.510 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.510 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.510 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.510 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.510 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.510 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:30:55.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.511 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.511 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.511 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:55.511 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.511 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.511 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:55.511 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.512 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.512 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:30:55.512 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:30:55.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.512 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.512 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:30:55.512 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:30:55.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.512 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.512 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:30:55.513 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:30:55.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.513 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.513 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:30:55.513 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:30:55.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.513 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.513 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:30:55.513 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:30:55.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.513 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.514 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:30:55.514 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:30:55.514 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.514 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.514 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.514 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.514 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:55.514 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.000221247, 2024-09-17T07:30:55.514 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:30:55.514 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.514 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.514 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.516 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.516 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:55.516 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:30:55.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.516 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.516 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:30:55.516 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:30:55.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.516 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.516 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:30:55.516 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:30:55.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.517 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.517 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:30:55.517 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:30:55.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.517 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.517 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:30:55.517 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:30:55.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.517 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.517 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:30:55.517 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.00020086000001, 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.518 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:30:55.519 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:30:55.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.520 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.520 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.520 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:30:55.520 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:30:55.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.520 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.520 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.520 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:30:55.520 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:30:55.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.520 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.520 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.521 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:30:55.521 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:30:55.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.521 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.521 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:30:55.521 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:30:55.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.521 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.521 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:30:55.521 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:30:55.521 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.521 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.522 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.522 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.522 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:55.522 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.00018115500001, 2024-09-17T07:30:55.522 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:30:55.523 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.523 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.523 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.523 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.523 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.523 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.523 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:30:55.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.523 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.523 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.523 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:55.523 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.524 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.524 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:55.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.524 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.524 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:30:55.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:30:55.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.525 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.525 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:30:55.525 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:30:55.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.525 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.525 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:30:55.525 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:30:55.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.525 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.525 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:30:55.525 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:30:55.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.526 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.526 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:30:55.526 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:30:55.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.526 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.526 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:30:55.526 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:30:55.526 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.526 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.526 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.526 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.527 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:55.527 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.00015807699999, 2024-09-17T07:30:55.527 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:30:55.527 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.527 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.527 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.527 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.527 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.527 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.527 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:30:55.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.527 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.527 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.527 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.527 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:55.528 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.528 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.528 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:55.528 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.528 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.528 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:30:55.528 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:30:55.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.528 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:30:55.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:30:55.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.529 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:30:55.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:30:55.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.529 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:30:55.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:30:55.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.530 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.530 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:30:55.530 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:30:55.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.530 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.530 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:30:55.530 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:30:55.530 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.530 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.530 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.530 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:55.530 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:55.531 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 107.00013751500001, 2024-09-17T07:30:55.531 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:30:55.531 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:55.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:55.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:55.531 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:55.531 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:55.531 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:55.531 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:30:55.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.531 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:55.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.531 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:55.531 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:55.532 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.532 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:55.532 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:55.532 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:55.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.532 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:55.532 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:55.532 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:30:55.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.532 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:55.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:30:55.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:30:55.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:55.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:30:55.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:30:55.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:55.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:30:55.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:30:55.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:55.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:30:55.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:30:55.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:55.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:55.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:55.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:30:55.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:30:55.534 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.534 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.534 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.534 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:55.535 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:55.535 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:55.536 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:55.537 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:55.535+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:55.539 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:55.539 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:55.556 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:55.556 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:55.573 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:55.589 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:55.589 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:55.606 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:55.606 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:55.623 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:55.623 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:55.636 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:55.636+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:55.639 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:55.640 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:55.647 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:55.647 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:55.656 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:55.657 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:55.657 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:55.737 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:55.737+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:55.773 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:55.773 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:55.782 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:55.838 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:55.838+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:55.932 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:55.932 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:55.939 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:55.939+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:55.940 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:56.041 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:56.040+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:56.088 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:56.088 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:56.101 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:56.144 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:56.144+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:56.216 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:56.216 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:56.225 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:30:56.242 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:56.241+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:56.343 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:56.343+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:56.372 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:56.373 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:56.381 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:30:56.444 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:56.444+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:56.485 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:56.485 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:56.494 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:30:56.578 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:56.578+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:56.637 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:56.637 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:56.646 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:56.645+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:56.647 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:30:56.746 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:56.746+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:56.789 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:56.789 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:56.798 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:30:56.882 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:56.882+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:56.943 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:56.943 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:56.948 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:56.948+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:56.951 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:30:57.049 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:57.048+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:57.099 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:57.099 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:57.108 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:30:57.108 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T07:30:57.153 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:57.152+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:57.250 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:57.250+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:57.258 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:57.258 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:57.266 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:30:57.351 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:57.351+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:57.409 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:57.409 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:57.418 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:30:57.452 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:57.452+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:57.553 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:57.553+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:57.560 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:57.560 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:57.569 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:30:57.654 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:57.654+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:57.669 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:30:57.712 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:57.712 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:57.720 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:30:57.763 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:57.763+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:57.795 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:30:57.796 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:30:57.812 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:30:57.813 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:30:57.823 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:30:57.823 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:30:57.828 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:30:57.829 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:30:57.832 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:30:57.845 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:30:57.845 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:30:57.856 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:57.856+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:57.862 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:30:57.862 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:30:57.879 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:30:57.879 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:30:57.896 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:30:57.896 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:30:57.913 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:30:57.913 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:30:57.965 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:57.964+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:57.976 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:57.976 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:30:57.976 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:30:57.976 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:30:57.976 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:57.985 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:30:58.058 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:30:58.058+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:30:58.089 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:30:58.089 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:30:58.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:30:58.089 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:30:58.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.089 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:30:58.089 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:58.090 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 151.75273868799999, 2024-09-17T07:30:58.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:30:58.090 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.090 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.090 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:30:58.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:30:58.090 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:30:58.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.090 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:58.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:30:58.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:30:58.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:30:58.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:30:58.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:30:58.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:30:58.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:30:58.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:30:58.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:30:58.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:30:58.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:30:58.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:58.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:30:58.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:30:58.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:30:58.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:30:58.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:30:58.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:30:58.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:58.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:30:58.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:30:58.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:58.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:30:58.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:30:58.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:30:58.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:30:58.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:58.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:30:58.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:30:58.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:30:58.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:30:58.096 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.096 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.096 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.096 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.096 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:58.097 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.16368653799999, 2024-09-17T07:30:58.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:30:58.097 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.097 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.097 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.097 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:30:58.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.098 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:58.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:30:58.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:30:58.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:30:58.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:30:58.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:30:58.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:30:58.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:30:58.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:30:58.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:30:58.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:30:58.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:30:58.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:30:58.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:30:58.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:30:58.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:30:58.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.103 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.104 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.104 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:58.104 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.15840342200001, 2024-09-17T07:30:58.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:30:58.104 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.104 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.104 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.104 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:30:58.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.105 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:58.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:58.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:30:58.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:30:58.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:30:58.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:30:58.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:30:58.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:30:58.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:30:58.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:30:58.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:30:58.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:30:58.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:30:58.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:30:58.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:30:58.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:30:58.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.108 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.108 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.109 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:58.109 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.158372442, 2024-09-17T07:30:58.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:30:58.109 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.109 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.109 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.109 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:30:58.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.109 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:58.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:30:58.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:30:58.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:30:58.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:30:58.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:30:58.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:30:58.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:30:58.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:30:58.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:30:58.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:30:58.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:30:58.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:30:58.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:30:58.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:30:58.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:30:58.114 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.114 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.114 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.114 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.114 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:58.114 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.158342393, 2024-09-17T07:30:58.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:30:58.114 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.114 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.114 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.114 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.115 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:30:58.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.115 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:58.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:30:58.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:30:58.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:30:58.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:30:58.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:30:58.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:30:58.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:30:58.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:30:58.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:30:58.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:30:58.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:30:58.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:30:58.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:30:58.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:30:58.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:30:58.118 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.118 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.118 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.119 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.119 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:58.119 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.158313986, 2024-09-17T07:30:58.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:30:58.119 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.119 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.119 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.119 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.119 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.119 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.119 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:30:58.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.119 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:58.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:58.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:30:58.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:58.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:30:58.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:30:58.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:30:58.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:30:58.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:30:58.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:30:58.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:30:58.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:30:58.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:30:58.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:30:58.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:30:58.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:30:58.123 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.123 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.124 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.124 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:58.124 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.158294165, 2024-09-17T07:30:58.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:30:58.124 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.124 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.124 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.124 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.124 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.124 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.125 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:30:58.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.125 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:58.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:30:58.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:30:58.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:30:58.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:30:58.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:30:58.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:30:58.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:30:58.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:30:58.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:30:58.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:30:58.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:30:58.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:30:58.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:30:58.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:30:58.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:30:58.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.128 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.128 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.128 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:58.128 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.158269913, 2024-09-17T07:30:58.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:30:58.129 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.129 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.129 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.129 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:30:58.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.129 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:58.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:30:58.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:30:58.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:30:58.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:30:58.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:30:58.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:30:58.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:30:58.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:30:58.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:30:58.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:30:58.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:30:58.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:30:58.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:30:58.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:30:58.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:30:58.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.133 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.133 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.133 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:58.133 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.152847325, 2024-09-17T07:30:58.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:30:58.133 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.133 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.133 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.134 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:30:58.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.134 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:58.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:30:58.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:30:58.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:30:58.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:30:58.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:30:58.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:30:58.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:30:58.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:30:58.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:30:58.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:30:58.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:30:58.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:30:58.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:30:58.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:30:58.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:30:58.137 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.137 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.138 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.138 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:58.138 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.15280045, 2024-09-17T07:30:58.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:30:58.138 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.138 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.138 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.139 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:30:58.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.139 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:58.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:58.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:30:58.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:30:58.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:30:58.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:30:58.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:30:58.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:30:58.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:30:58.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:30:58.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:30:58.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:30:58.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:30:58.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:30:58.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:30:58.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:30:58.143 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.143 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.143 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.143 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.143 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:58.143 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.15278574600001, 2024-09-17T07:30:58.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:30:58.143 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.143 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.144 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.144 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:30:58.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.144 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:58.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:58.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:30:58.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:30:58.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:30:58.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:30:58.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:30:58.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:30:58.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:30:58.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:30:58.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:30:58.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:30:58.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:30:58.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:30:58.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:30:58.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:30:58.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.150 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.150 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.150 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:58.150 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.152762651, 2024-09-17T07:30:58.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:30:58.150 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.150 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.150 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.151 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:30:58.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.151 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:58.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:58.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:30:58.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:30:58.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:30:58.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:30:58.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:30:58.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:30:58.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:30:58.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:30:58.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:30:58.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:30:58.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:30:58.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:30:58.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:30:58.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:30:58.154 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.154 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.154 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.155 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.155 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:58.155 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.152731276, 2024-09-17T07:30:58.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:30:58.155 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.155 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.155 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.155 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:30:58.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.156 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:58.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:30:58.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:30:58.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:30:58.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:30:58.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:30:58.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:30:58.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:30:58.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:30:58.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:30:58.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:30:58.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:30:58.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:30:58.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:30:58.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:30:58.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:30:58.159 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.159 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.159 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.159 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.159 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:58.160 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.152716016, 2024-09-17T07:30:58.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:30:58.160 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.160 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.160 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.160 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:30:58.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.160 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:58.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:58.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:30:58.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:30:58.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:30:58.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:30:58.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:30:58.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:30:58.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:30:58.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:30:58.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:30:58.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:30:58.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:30:58.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:30:58.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:30:58.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:30:58.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.164 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.164 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.164 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:58.164 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.15269594599999, 2024-09-17T07:30:58.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:30:58.164 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.165 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.165 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.165 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:30:58.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.165 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:58.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:58.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:30:58.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:30:58.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:30:58.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:30:58.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:30:58.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:30:58.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:30:58.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:30:58.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:30:58.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:30:58.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:30:58.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:30:58.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:30:58.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:30:58.171 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.172 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.172 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.172 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.172 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:58.172 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.152684006, 2024-09-17T07:30:58.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:30:58.172 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.172 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.172 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.172 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.173 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.173 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.173 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:30:58.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.173 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:58.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:58.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:30:58.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:30:58.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:30:58.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:30:58.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:30:58.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:30:58.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:30:58.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:30:58.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:30:58.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:30:58.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:30:58.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:30:58.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:30:58.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:30:58.177 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.177 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.177 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.177 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:30:58.177 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:58.177 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.152660848, 2024-09-17T07:30:58.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:30:58.178 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.178 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.178 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.178 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.178 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:30:58.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.178 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:58.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:30:58.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:30:58.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:30:58.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:30:58.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:30:58.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:30:58.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:30:58.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:30:58.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:30:58.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:30:58.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:30:58.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:30:58.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:30:58.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:30:58.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:30:58.182 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.182 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.182 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.182 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:115644 3.15 3:af37e0f6:::rbd_header.118311e0d6f8:head [watch reconnect cookie 140227662450992 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T07:30:58.183 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:30:58.183 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.009503954000003, 2024-09-17T07:30:58.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.013658722, 2024-09-17T07:30:58.183 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.183 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.183 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:30:58.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.183 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115644 2024-09-17T07:30:58.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.183 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:30:58.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:30:58.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077907+0000", 2024-09-17T07:30:58.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.08e-06 2024-09-17T07:30:58.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077916+0000", 2024-09-17T07:30:58.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-09-17T07:30:58.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077938+0000", 2024-09-17T07:30:58.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1883999999999999e-05 2024-09-17T07:30:58.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077950+0000", 2024-09-17T07:30:58.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2298e-05 2024-09-17T07:30:58.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.078037+0000", 2024-09-17T07:30:58.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6726999999999998e-05 2024-09-17T07:30:58.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-17T07:30:58.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.078046+0000", 2024-09-17T07:30:58.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6010000000000001e-06 2024-09-17T07:30:58.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085705+0000", 2024-09-17T07:30:58.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007659084 2024-09-17T07:30:58.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:58.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085718+0000", 2024-09-17T07:30:58.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2081e-05 2024-09-17T07:30:58.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.089697+0000", 2024-09-17T07:30:58.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.003979373 2024-09-17T07:30:58.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.089875+0000", 2024-09-17T07:30:58.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017760000000000001 2024-09-17T07:30:58.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.091563+0000", 2024-09-17T07:30:58.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001688836 2024-09-17T07:30:58.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.189 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.189 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:115645 3.15 3:af37e0f6:::rbd_header.118311e0d6f8:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T07:30:58.189 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:30:58.190 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.006686434000002, 2024-09-17T07:30:58.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.011649171, 2024-09-17T07:30:58.190 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.190 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.190 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.190 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115645 2024-09-17T07:30:58.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.190 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:30:58.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:30:58.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080724+0000", 2024-09-17T07:30:58.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-17T07:30:58.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080735+0000", 2024-09-17T07:30:58.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1544e-05 2024-09-17T07:30:58.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080751+0000", 2024-09-17T07:30:58.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5244e-05 2024-09-17T07:30:58.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080761+0000", 2024-09-17T07:30:58.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0372e-05 2024-09-17T07:30:58.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080834+0000", 2024-09-17T07:30:58.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3263999999999995e-05 2024-09-17T07:30:58.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:58.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080842+0000", 2024-09-17T07:30:58.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2209999999999997e-06 2024-09-17T07:30:58.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085755+0000", 2024-09-17T07:30:58.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049130729999999996 2024-09-17T07:30:58.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:58.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085766+0000", 2024-09-17T07:30:58.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0288e-05 2024-09-17T07:30:58.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.090746+0000", 2024-09-17T07:30:58.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0049801190000001 2024-09-17T07:30:58.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.090798+0000", 2024-09-17T07:30:58.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2502e-05 2024-09-17T07:30:58.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-09-17T07:30:58.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.091037+0000", 2024-09-17T07:30:58.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0002385 2024-09-17T07:30:58.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092119+0000", 2024-09-17T07:30:58.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010820249999999999 2024-09-17T07:30:58.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:58.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092276+0000", 2024-09-17T07:30:58.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000156737 2024-09-17T07:30:58.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:58.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092330+0000", 2024-09-17T07:30:58.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4666999999999997e-05 2024-09-17T07:30:58.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092371+0000", 2024-09-17T07:30:58.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1043e-05 2024-09-17T07:30:58.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.198 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.198 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:116418 3.13 3:cdf2692d:::rbd_data.1183cb746bb8.0000000000000024:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T07:30:58.198 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:58.198 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.917270037000002, 2024-09-17T07:30:58.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92140365899999999, 2024-09-17T07:30:58.199 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:30:58.199 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:30:58.199 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:30:58.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:30:58.199 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.199 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116418 2024-09-17T07:30:58.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.199 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:30:58.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:30:58.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:58.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:30:58.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:30:58.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:30:58.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:30:58.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170140+0000", 2024-09-17T07:30:58.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.398e-06 2024-09-17T07:30:58.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:30:58.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170144+0000", 2024-09-17T07:30:58.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4399999999999998e-06 2024-09-17T07:30:58.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:30:58.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170157+0000", 2024-09-17T07:30:58.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2133999999999999e-05 2024-09-17T07:30:58.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170164+0000", 2024-09-17T07:30:58.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7200000000000006e-06 2024-09-17T07:30:58.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170227+0000", 2024-09-17T07:30:58.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3035999999999998e-05 2024-09-17T07:30:58.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:30:58.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170238+0000", 2024-09-17T07:30:58.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0397000000000001e-05 2024-09-17T07:30:58.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:30:58.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086367+0000", 2024-09-17T07:30:58.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.916128889 2024-09-17T07:30:58.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:30:58.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086523+0000", 2024-09-17T07:30:58.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015638800000000001 2024-09-17T07:30:58.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:30:58.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086652+0000", 2024-09-17T07:30:58.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000128806 2024-09-17T07:30:58.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:30:58.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091261+0000", 2024-09-17T07:30:58.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046091020000000003 2024-09-17T07:30:58.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091449+0000", 2024-09-17T07:30:58.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000188486 2024-09-17T07:30:58.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:30:58.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091503+0000", 2024-09-17T07:30:58.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3254999999999997e-05 2024-09-17T07:30:58.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:30:58.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:30:58.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:30:58.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091542+0000", 2024-09-17T07:30:58.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9607999999999998e-05 2024-09-17T07:30:58.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.206 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.207 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:30:58.207 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:30:58.207 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:30:58.207 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 5, 6, 7, 3, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-17T07:30:58.207 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T07:30:58.208 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-17T07:30:58.210 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:30:58.210 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:58.206+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:58.309 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:58.307+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:58.363 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:58.363 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:58.363 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:58.363 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:58.372 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:30:58.410 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:58.409+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:58.480 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:58.480 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:58.480 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:58.480 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:58.480 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:58.489 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:30:58.510 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:58.509+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:30:58.598 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:58.598 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.18 109/106 [PushOp(3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e, version: 44'134, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e@44'134, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:18.083461+0000", 2024-09-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 40.513291596000002, 2024-09-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003057976, 2024-09-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.599 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083461+0000", 2024-09-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083461+0000", 2024-09-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083462+0000", 2024-09-17T07:30:58.600 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1500000000000004e-07 2024-09-17T07:30:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083603+0000", 2024-09-17T07:30:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140863 2024-09-17T07:30:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083623+0000", 2024-09-17T07:30:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.003e-05 2024-09-17T07:30:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.601 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.083642+0000", 2024-09-17T07:30:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9734000000000001e-05 2024-09-17T07:30:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.084564+0000", 2024-09-17T07:30:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00092196200000000002 2024-09-17T07:30:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.084580+0000", 2024-09-17T07:30:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5464e-05 2024-09-17T07:30:58.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:18.086519+0000", 2024-09-17T07:30:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001939208 2024-09-17T07:30:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115899 3.b 3:d406e0ff:::rbd_data.118340e4fbf.0000000000000001:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:58.603 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 36.405817923999997, 2024-09-17T07:30:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0040641219999999999, 2024-09-17T07:30:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115899 2024-09-17T07:30:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:30:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190936+0000", 2024-09-17T07:30:58.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.26e-06 2024-09-17T07:30:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190941+0000", 2024-09-17T07:30:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0499999999999999e-06 2024-09-17T07:30:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190958+0000", 2024-09-17T07:30:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6548000000000001e-05 2024-09-17T07:30:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190966+0000", 2024-09-17T07:30:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6799999999999999e-06 2024-09-17T07:30:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191028+0000", 2024-09-17T07:30:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2082999999999995e-05 2024-09-17T07:30:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191161+0000", 2024-09-17T07:30:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000132461 2024-09-17T07:30:58.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191284+0000", 2024-09-17T07:30:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012301200000000001 2024-09-17T07:30:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192371+0000", 2024-09-17T07:30:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00108709 2024-09-17T07:30:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.608 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194898+0000", 2024-09-17T07:30:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025272710000000002 2024-09-17T07:30:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194957+0000", 2024-09-17T07:30:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8792000000000002e-05 2024-09-17T07:30:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194999+0000", 2024-09-17T07:30:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1875000000000001e-05 2024-09-17T07:30:58.609 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115903 3.18 3:1fc9b9ef:::rbd_data.118340e4fbf.0000000000000005:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 36.405019801000002, 2024-09-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00477466, 2024-09-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.610 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115903 2024-09-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:58.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9800000000000004e-07 2024-09-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191736+0000", 2024-09-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6620000000000001e-06 2024-09-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191743+0000", 2024-09-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6939999999999998e-06 2024-09-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191748+0000", 2024-09-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9749999999999998e-06 2024-09-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191823+0000", 2024-09-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4954000000000001e-05 2024-09-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191933+0000", 2024-09-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000110708 2024-09-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192031+0000", 2024-09-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7974999999999996e-05 2024-09-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193329+0000", 2024-09-17T07:30:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012976419999999999 2024-09-17T07:30:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196404+0000", 2024-09-17T07:30:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003075174 2024-09-17T07:30:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196461+0000", 2024-09-17T07:30:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6663999999999998e-05 2024-09-17T07:30:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196507+0000", 2024-09-17T07:30:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6714e-05 2024-09-17T07:30:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115905 3.18 3:19f08345:::rbd_data.118340e4fbf.0000000000000009:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:30:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 36.404988930999998, 2024-09-17T07:30:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0049339900000000001, 2024-09-17T07:30:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115905 2024-09-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:30:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0699999999999998e-07 2024-09-17T07:30:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191766+0000", 2024-09-17T07:30:58.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.928e-06 2024-09-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191773+0000", 2024-09-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2749999999999998e-06 2024-09-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191778+0000", 2024-09-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7849999999999999e-06 2024-09-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192756+0000", 2024-09-17T07:30:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097834399999999992 2024-09-17T07:30:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192868+0000", 2024-09-17T07:30:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111575 2024-09-17T07:30:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:30:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192970+0000", 2024-09-17T07:30:58.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010192300000000001 2024-09-17T07:30:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193865+0000", 2024-09-17T07:30:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089556100000000001 2024-09-17T07:30:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196583+0000", 2024-09-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027174180000000001 2024-09-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196648+0000", 2024-09-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4743e-05 2024-09-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196698+0000", 2024-09-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0031000000000003e-05 2024-09-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.622 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117007 3.8 3:11c9baa3:::rbd_object_map.118394b259f9:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:36.120261+0000", 2024-09-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.476491873000001, 2024-09-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045697289999999998, 2024-09-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.623 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117007 2024-09-17T07:30:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120261+0000", 2024-09-17T07:30:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120261+0000", 2024-09-17T07:30:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120262+0000", 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.77e-06 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120270+0000", 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0080000000000002e-06 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120287+0000", 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6847e-05 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120297+0000", 2024-09-17T07:30:58.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8830000000000001e-06 2024-09-17T07:30:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120368+0000", 2024-09-17T07:30:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0689e-05 2024-09-17T07:30:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:30:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120375+0000", 2024-09-17T07:30:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.187e-06 2024-09-17T07:30:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.122574+0000", 2024-09-17T07:30:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021992980000000001 2024-09-17T07:30:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.122670+0000", 2024-09-17T07:30:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5830999999999998e-05 2024-09-17T07:30:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:30:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.122866+0000", 2024-09-17T07:30:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000195303 2024-09-17T07:30:58.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124084+0000", 2024-09-17T07:30:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001218076 2024-09-17T07:30:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124153+0000", 2024-09-17T07:30:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9839999999999995e-05 2024-09-17T07:30:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124784+0000", 2024-09-17T07:30:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063082299999999997 2024-09-17T07:30:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124830+0000", 2024-09-17T07:30:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6174000000000002e-05 2024-09-17T07:30:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117147 3.7 3:e1d03a89:::rbd_header.11832ac10dcc:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T07:30:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.085788+0000", 2024-09-17T07:30:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 21.510964396999999, 2024-09-17T07:30:58.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0032128040000000001, 2024-09-17T07:30:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117147 2024-09-17T07:30:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085788+0000", 2024-09-17T07:30:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085788+0000", 2024-09-17T07:30:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085790+0000", 2024-09-17T07:30:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1179999999999999e-06 2024-09-17T07:30:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085798+0000", 2024-09-17T07:30:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1300000000000001e-06 2024-09-17T07:30:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085812+0000", 2024-09-17T07:30:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4065e-05 2024-09-17T07:30:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085824+0000", 2024-09-17T07:30:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1307e-05 2024-09-17T07:30:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085883+0000", 2024-09-17T07:30:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9141999999999997e-05 2024-09-17T07:30:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.085985+0000", 2024-09-17T07:30:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000101895 2024-09-17T07:30:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.086372+0000", 2024-09-17T07:30:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00038743600000000001 2024-09-17T07:30:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.087110+0000", 2024-09-17T07:30:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000737887 2024-09-17T07:30:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.088882+0000", 2024-09-17T07:30:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017721600000000001 2024-09-17T07:30:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.088947+0000", 2024-09-17T07:30:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4719000000000002e-05 2024-09-17T07:30:58.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.089001+0000", 2024-09-17T07:30:58.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3944999999999999e-05 2024-09-17T07:30:58.636 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.636 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.636 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.636 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117151 3.7 3:e1d03a89:::rbd_header.11832ac10dcc:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T07:30:58.636 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.097104+0000", 2024-09-17T07:30:58.636 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 21.499648217000001, 2024-09-17T07:30:58.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062858530000000001, 2024-09-17T07:30:58.637 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.637 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.637 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.637 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.637 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.637 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117151 2024-09-17T07:30:58.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.637 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097104+0000", 2024-09-17T07:30:58.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097104+0000", 2024-09-17T07:30:58.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097107+0000", 2024-09-17T07:30:58.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7429999999999998e-06 2024-09-17T07:30:58.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097121+0000", 2024-09-17T07:30:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3774000000000001e-05 2024-09-17T07:30:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097150+0000", 2024-09-17T07:30:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8762e-05 2024-09-17T07:30:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097163+0000", 2024-09-17T07:30:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3631e-05 2024-09-17T07:30:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097245+0000", 2024-09-17T07:30:58.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1874999999999998e-05 2024-09-17T07:30:58.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097337+0000", 2024-09-17T07:30:58.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1465000000000006e-05 2024-09-17T07:30:58.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:58.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097553+0000", 2024-09-17T07:30:58.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021630999999999999 2024-09-17T07:30:58.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.098320+0000", 2024-09-17T07:30:58.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076674000000000002 2024-09-17T07:30:58.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.103265+0000", 2024-09-17T07:30:58.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0049452250000000001 2024-09-17T07:30:58.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.103335+0000", 2024-09-17T07:30:58.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0463999999999995e-05 2024-09-17T07:30:58.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.103390+0000", 2024-09-17T07:30:58.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4864000000000002e-05 2024-09-17T07:30:58.642 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.642 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.642 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.642 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117237 3.2 3:40c8b221:::rbd_data.11832ac10dcc.0000000000000000:head [delete] snapc 1d=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T07:30:58.642 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.169608+0000", 2024-09-17T07:30:58.642 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 21.427144219999999, 2024-09-17T07:30:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003512509, 2024-09-17T07:30:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117237 2024-09-17T07:30:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169608+0000", 2024-09-17T07:30:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169608+0000", 2024-09-17T07:30:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169610+0000", 2024-09-17T07:30:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8020000000000001e-06 2024-09-17T07:30:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.644 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169618+0000", 2024-09-17T07:30:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0080000000000002e-06 2024-09-17T07:30:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169635+0000", 2024-09-17T07:30:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6611000000000001e-05 2024-09-17T07:30:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169643+0000", 2024-09-17T07:30:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6549999999999993e-06 2024-09-17T07:30:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169723+0000", 2024-09-17T07:30:58.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0034e-05 2024-09-17T07:30:58.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169860+0000", 2024-09-17T07:30:58.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013645899999999999 2024-09-17T07:30:58.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:30:58.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169970+0000", 2024-09-17T07:30:58.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000109957 2024-09-17T07:30:58.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.647 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.171850+0000", 2024-09-17T07:30:58.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001879868 2024-09-17T07:30:58.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.647 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172391+0000", 2024-09-17T07:30:58.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054084100000000002 2024-09-17T07:30:58.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.647 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172439+0000", 2024-09-17T07:30:58.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7916e-05 2024-09-17T07:30:58.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.648 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173121+0000", 2024-09-17T07:30:58.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00068235799999999999 2024-09-17T07:30:58.648 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.648 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.648 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.648 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:117250 3.5 e128/109 3:a4728dbe:::rbd_data.11832ac10dcc.0000000000000075:head v 128'521, mlcod=126'519)", 2024-09-17T07:30:58.648 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:30:58.648 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 21.423848691, 2024-09-17T07:30:58.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0034023949999999999, 2024-09-17T07:30:58.648 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.649 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.649 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:30:58.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:30:58.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:30:58.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5900000000000002e-07 2024-09-17T07:30:58.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172910+0000", 2024-09-17T07:30:58.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2449999999999998e-06 2024-09-17T07:30:58.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172919+0000", 2024-09-17T07:30:58.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5389999999999996e-06 2024-09-17T07:30:58.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173431+0000", 2024-09-17T07:30:58.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00051224699999999996 2024-09-17T07:30:58.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173485+0000", 2024-09-17T07:30:58.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3936e-05 2024-09-17T07:30:58.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173504+0000", 2024-09-17T07:30:58.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8681e-05 2024-09-17T07:30:58.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.176259+0000", 2024-09-17T07:30:58.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027547719999999999 2024-09-17T07:30:58.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.176306+0000", 2024-09-17T07:30:58.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7515999999999997e-05 2024-09-17T07:30:58.652 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.652 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.652 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.653 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:118889 3.b 3:d08c2e41:::rbd_data.11839a97b753.0000000000000002:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-17T07:30:58.653 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:50.259083+0000", 2024-09-17T07:30:58.653 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 8.33766973, 2024-09-17T07:30:58.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060418879999999996, 2024-09-17T07:30:58.653 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.653 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.653 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.653 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.653 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.653 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.653 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 118889 2024-09-17T07:30:58.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.653 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259083+0000", 2024-09-17T07:30:58.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259083+0000", 2024-09-17T07:30:58.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259085+0000", 2024-09-17T07:30:58.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8509999999999999e-06 2024-09-17T07:30:58.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259092+0000", 2024-09-17T07:30:58.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9500000000000004e-06 2024-09-17T07:30:58.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259111+0000", 2024-09-17T07:30:58.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9165e-05 2024-09-17T07:30:58.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259122+0000", 2024-09-17T07:30:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0745e-05 2024-09-17T07:30:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259206+0000", 2024-09-17T07:30:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4741999999999996e-05 2024-09-17T07:30:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259345+0000", 2024-09-17T07:30:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000138644 2024-09-17T07:30:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259470+0000", 2024-09-17T07:30:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012535899999999999 2024-09-17T07:30:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.261565+0000", 2024-09-17T07:30:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002094561 2024-09-17T07:30:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265016+0000", 2024-09-17T07:30:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0034515240000000001 2024-09-17T07:30:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265080+0000", 2024-09-17T07:30:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3927e-05 2024-09-17T07:30:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265125+0000", 2024-09-17T07:30:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4419999999999998e-05 2024-09-17T07:30:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.658 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:118890 3.2 3:4647e1ae:::rbd_data.11839a97b753.0000000000000003:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-17T07:30:58.659 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:50.259150+0000", 2024-09-17T07:30:58.659 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 8.3376024080000004, 2024-09-17T07:30:58.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003910134, 2024-09-17T07:30:58.659 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.659 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.659 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.659 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.659 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.659 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 118890 2024-09-17T07:30:58.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.659 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259150+0000", 2024-09-17T07:30:58.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259150+0000", 2024-09-17T07:30:58.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259151+0000", 2024-09-17T07:30:58.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1999999999999999e-07 2024-09-17T07:30:58.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259154+0000", 2024-09-17T07:30:58.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5020000000000001e-06 2024-09-17T07:30:58.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259164+0000", 2024-09-17T07:30:58.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8190000000000003e-06 2024-09-17T07:30:58.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259171+0000", 2024-09-17T07:30:58.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9009999999999997e-06 2024-09-17T07:30:58.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.260418+0000", 2024-09-17T07:30:58.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001246891 2024-09-17T07:30:58.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.260587+0000", 2024-09-17T07:30:58.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000168796 2024-09-17T07:30:58.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:30:58.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.260773+0000", 2024-09-17T07:30:58.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000186269 2024-09-17T07:30:58.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262459+0000", 2024-09-17T07:30:58.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016858660000000001 2024-09-17T07:30:58.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262962+0000", 2024-09-17T07:30:58.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00050283499999999998 2024-09-17T07:30:58.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.263015+0000", 2024-09-17T07:30:58.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3115999999999998e-05 2024-09-17T07:30:58.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.263060+0000", 2024-09-17T07:30:58.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5519000000000002e-05 2024-09-17T07:30:58.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.664 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.665 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:118891 3.15 3:ae97839c:::rbd_data.11839a97b753.0000000000000004:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-17T07:30:58.665 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:50.259192+0000", 2024-09-17T07:30:58.665 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 8.3375609379999993, 2024-09-17T07:30:58.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031418380000000001, 2024-09-17T07:30:58.665 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.665 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.665 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.665 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.665 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.665 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 118891 2024-09-17T07:30:58.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.665 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259192+0000", 2024-09-17T07:30:58.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259192+0000", 2024-09-17T07:30:58.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259192+0000", 2024-09-17T07:30:58.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6199999999999997e-07 2024-09-17T07:30:58.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259195+0000", 2024-09-17T07:30:58.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.18e-06 2024-09-17T07:30:58.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259206+0000", 2024-09-17T07:30:58.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0291e-05 2024-09-17T07:30:58.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259212+0000", 2024-09-17T07:30:58.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2260000000000002e-06 2024-09-17T07:30:58.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259267+0000", 2024-09-17T07:30:58.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5217999999999999e-05 2024-09-17T07:30:58.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259393+0000", 2024-09-17T07:30:58.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012553299999999999 2024-09-17T07:30:58.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:30:58.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259494+0000", 2024-09-17T07:30:58.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000101643 2024-09-17T07:30:58.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.261587+0000", 2024-09-17T07:30:58.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020926959999999998 2024-09-17T07:30:58.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262247+0000", 2024-09-17T07:30:58.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00065972399999999999 2024-09-17T07:30:58.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262294+0000", 2024-09-17T07:30:58.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7410000000000002e-05 2024-09-17T07:30:58.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.670 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.670 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262333+0000", 2024-09-17T07:30:58.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9255000000000003e-05 2024-09-17T07:30:58.670 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.670 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.670 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.670 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:118897 3.b 3:d71b53f6:::rbd_data.11839a97b753.000000000000000a:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-17T07:30:58.670 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:50.262853+0000", 2024-09-17T07:30:58.670 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 8.3339000680000002, 2024-09-17T07:30:58.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031177959999999999, 2024-09-17T07:30:58.670 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.671 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.671 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.671 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.671 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.671 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.671 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 118897 2024-09-17T07:30:58.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.671 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.671 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262853+0000", 2024-09-17T07:30:58.671 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262853+0000", 2024-09-17T07:30:58.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262854+0000", 2024-09-17T07:30:58.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.804e-06 2024-09-17T07:30:58.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262860+0000", 2024-09-17T07:30:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6729999999999997e-06 2024-09-17T07:30:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262872+0000", 2024-09-17T07:30:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2319e-05 2024-09-17T07:30:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262881+0000", 2024-09-17T07:30:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4290000000000001e-06 2024-09-17T07:30:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262948+0000", 2024-09-17T07:30:58.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7384000000000004e-05 2024-09-17T07:30:58.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.263080+0000", 2024-09-17T07:30:58.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013160400000000001 2024-09-17T07:30:58.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:58.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.263197+0000", 2024-09-17T07:30:58.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011731999999999999 2024-09-17T07:30:58.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.675 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.675 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.264313+0000", 2024-09-17T07:30:58.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001115731 2024-09-17T07:30:58.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.675 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.675 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265854+0000", 2024-09-17T07:30:58.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001541132 2024-09-17T07:30:58.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.676 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.676 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265913+0000", 2024-09-17T07:30:58.676 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8729e-05 2024-09-17T07:30:58.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.676 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.676 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265970+0000", 2024-09-17T07:30:58.676 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7670999999999997e-05 2024-09-17T07:30:58.676 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.676 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.676 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.677 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:119884 3.18 3:1a94cd5b:::rbd_header.1183ce4061a0:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T07:30:58.677 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:58.210641+0000", 2024-09-17T07:30:58.677 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.38611126699999998, 2024-09-17T07:30:58.677 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010663832999999999, 2024-09-17T07:30:58.677 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.677 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.677 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.677 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.677 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.677 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 119884 2024-09-17T07:30:58.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.678 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.678 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.678 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.210641+0000", 2024-09-17T07:30:58.678 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.678 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.678 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.210641+0000", 2024-09-17T07:30:58.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.210644+0000", 2024-09-17T07:30:58.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2400000000000002e-06 2024-09-17T07:30:58.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.210650+0000", 2024-09-17T07:30:58.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9299999999999997e-06 2024-09-17T07:30:58.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.210663+0000", 2024-09-17T07:30:58.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2931e-05 2024-09-17T07:30:58.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.210671+0000", 2024-09-17T07:30:58.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7729999999999999e-06 2024-09-17T07:30:58.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.215471+0000", 2024-09-17T07:30:58.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004799462 2024-09-17T07:30:58.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.681 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.681 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.215568+0000", 2024-09-17T07:30:58.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7116999999999999e-05 2024-09-17T07:30:58.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.681 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:30:58.681 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.215951+0000", 2024-09-17T07:30:58.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000382791 2024-09-17T07:30:58.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.217671+0000", 2024-09-17T07:30:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017203990000000001 2024-09-17T07:30:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.221194+0000", 2024-09-17T07:30:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035229990000000002 2024-09-17T07:30:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.221256+0000", 2024-09-17T07:30:58.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2114999999999997e-05 2024-09-17T07:30:58.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.683 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.221305+0000", 2024-09-17T07:30:58.683 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9076000000000002e-05 2024-09-17T07:30:58.683 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.683 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.683 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.683 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop_reply(osd.3.0:271 3.18 e149/133 ondisk, result = 0)", 2024-09-17T07:30:58.683 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:58.213549+0000", 2024-09-17T07:30:58.683 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.38320324700000002, 2024-09-17T07:30:58.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033847119999999998, 2024-09-17T07:30:58.684 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.684 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:58.684 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.684 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.684 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213549+0000", 2024-09-17T07:30:58.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.684 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.684 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213549+0000", 2024-09-17T07:30:58.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.685 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.685 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213550+0000", 2024-09-17T07:30:58.685 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.06e-07 2024-09-17T07:30:58.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.685 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.685 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213554+0000", 2024-09-17T07:30:58.685 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-06 2024-09-17T07:30:58.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.685 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213564+0000", 2024-09-17T07:30:58.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5109999999999992e-06 2024-09-17T07:30:58.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.686 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213573+0000", 2024-09-17T07:30:58.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2790000000000004e-06 2024-09-17T07:30:58.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.686 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.216900+0000", 2024-09-17T07:30:58.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033272559999999998 2024-09-17T07:30:58.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.216918+0000", 2024-09-17T07:30:58.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7479e-05 2024-09-17T07:30:58.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.216934+0000", 2024-09-17T07:30:58.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6070999999999999e-05 2024-09-17T07:30:58.687 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.687 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.687 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.688 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(osd.1.0:2942 3.19 e149/108 3:9c9ac020:::rbd_data.1183ce4061a0.000000000000002a:e v 149'426, mlcod=139'423)", 2024-09-17T07:30:58.688 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:58.213882+0000", 2024-09-17T07:30:58.688 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.38287099299999999, 2024-09-17T07:30:58.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003450023, 2024-09-17T07:30:58.688 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.688 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.688 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.688 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213882+0000", 2024-09-17T07:30:58.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.689 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213882+0000", 2024-09-17T07:30:58.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.689 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213882+0000", 2024-09-17T07:30:58.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:30:58.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.689 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213889+0000", 2024-09-17T07:30:58.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5780000000000004e-06 2024-09-17T07:30:58.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213899+0000", 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.012e-05 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213906+0000", 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.215356+0000", 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014503949999999999 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.691 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.691 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.215378+0000", 2024-09-17T07:30:58.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1770999999999999e-05 2024-09-17T07:30:58.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.691 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.691 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.217286+0000", 2024-09-17T07:30:58.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019081090000000001 2024-09-17T07:30:58.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.691 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.691 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.217332+0000", 2024-09-17T07:30:58.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5343999999999998e-05 2024-09-17T07:30:58.691 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.692 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.692 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.692 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:119974 3.15 3:abe93d35:::rbd_data.1183ce4061a0.0000000000000006:head [delete] snapc e=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T07:30:58.692 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:58.298891+0000", 2024-09-17T07:30:58.692 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.297861239, 2024-09-17T07:30:58.692 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0047785229999999998, 2024-09-17T07:30:58.692 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.692 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.692 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.692 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.692 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.693 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.693 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 119974 2024-09-17T07:30:58.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.693 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.693 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.693 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298891+0000", 2024-09-17T07:30:58.693 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.693 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.693 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298891+0000", 2024-09-17T07:30:58.693 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.694 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.694 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298893+0000", 2024-09-17T07:30:58.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4309999999999999e-06 2024-09-17T07:30:58.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.694 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.694 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298898+0000", 2024-09-17T07:30:58.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6169999999999998e-06 2024-09-17T07:30:58.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.694 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.694 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298913+0000", 2024-09-17T07:30:58.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4803e-05 2024-09-17T07:30:58.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298923+0000", 2024-09-17T07:30:58.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9010000000000006e-06 2024-09-17T07:30:58.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298995+0000", 2024-09-17T07:30:58.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2278000000000002e-05 2024-09-17T07:30:58.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.299125+0000", 2024-09-17T07:30:58.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012987400000000001 2024-09-17T07:30:58.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:30:58.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.299253+0000", 2024-09-17T07:30:58.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000127732 2024-09-17T07:30:58.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.300124+0000", 2024-09-17T07:30:58.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087097800000000001 2024-09-17T07:30:58.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.697 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.697 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.303580+0000", 2024-09-17T07:30:58.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0034557469999999999 2024-09-17T07:30:58.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.697 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.697 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.303630+0000", 2024-09-17T07:30:58.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0300000000000003e-05 2024-09-17T07:30:58.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.697 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.697 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.303670+0000", 2024-09-17T07:30:58.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9861999999999999e-05 2024-09-17T07:30:58.697 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.697 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.698 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.698 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.698 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:119978 3.7 3:e6f882de:::rbd_data.1183ce4061a0.000000000000000b:head [delete] snapc e=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T07:30:58.698 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:58.298977+0000", 2024-09-17T07:30:58.698 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.29777520000000002, 2024-09-17T07:30:58.698 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0046515039999999999, 2024-09-17T07:30:58.698 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.698 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.698 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.698 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.698 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.698 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.698 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 119978 2024-09-17T07:30:58.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.699 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.699 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.699 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298977+0000", 2024-09-17T07:30:58.699 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.699 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.699 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298977+0000", 2024-09-17T07:30:58.699 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.699 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.699 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298978+0000", 2024-09-17T07:30:58.699 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:30:58.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.700 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.700 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.700 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298980+0000", 2024-09-17T07:30:58.700 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2069999999999998e-06 2024-09-17T07:30:58.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.700 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.700 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.700 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298998+0000", 2024-09-17T07:30:58.700 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7489000000000001e-05 2024-09-17T07:30:58.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.700 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.700 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.701 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.299003+0000", 2024-09-17T07:30:58.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8000000000000004e-06 2024-09-17T07:30:58.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.701 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.701 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.299065+0000", 2024-09-17T07:30:58.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1758000000000001e-05 2024-09-17T07:30:58.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.701 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.701 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.299174+0000", 2024-09-17T07:30:58.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000109281 2024-09-17T07:30:58.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:30:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.299265+0000", 2024-09-17T07:30:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1098000000000001e-05 2024-09-17T07:30:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.300109+0000", 2024-09-17T07:30:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00084360399999999997 2024-09-17T07:30:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.303538+0000", 2024-09-17T07:30:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0034284910000000001 2024-09-17T07:30:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.703 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.703 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.303590+0000", 2024-09-17T07:30:58.703 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1984999999999998e-05 2024-09-17T07:30:58.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.703 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.703 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.303629+0000", 2024-09-17T07:30:58.703 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9359e-05 2024-09-17T07:30:58.703 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.703 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.703 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.704 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:119984 3.15 3:a9fb6512:::rbd_data.1183ce4061a0.0000000000000012:head [delete] snapc e=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T07:30:58.704 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:58.301188+0000", 2024-09-17T07:30:58.704 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.29556437499999999, 2024-09-17T07:30:58.704 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00515352, 2024-09-17T07:30:58.704 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.704 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.704 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.704 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.704 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.704 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.704 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 119984 2024-09-17T07:30:58.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.704 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.705 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.705 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301188+0000", 2024-09-17T07:30:58.705 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.705 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.705 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301188+0000", 2024-09-17T07:30:58.705 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.705 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.705 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301190+0000", 2024-09-17T07:30:58.705 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.342e-06 2024-09-17T07:30:58.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.706 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.706 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301195+0000", 2024-09-17T07:30:58.706 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3170000000000001e-06 2024-09-17T07:30:58.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.706 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.706 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301209+0000", 2024-09-17T07:30:58.706 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3716e-05 2024-09-17T07:30:58.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.706 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.707 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301216+0000", 2024-09-17T07:30:58.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9929999999999998e-06 2024-09-17T07:30:58.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.707 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.707 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301293+0000", 2024-09-17T07:30:58.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7775000000000007e-05 2024-09-17T07:30:58.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.707 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.707 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301382+0000", 2024-09-17T07:30:58.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9087999999999998e-05 2024-09-17T07:30:58.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.708 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:30:58.708 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301500+0000", 2024-09-17T07:30:58.708 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000117176 2024-09-17T07:30:58.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.708 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.708 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.708 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.302158+0000", 2024-09-17T07:30:58.708 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00065859200000000005 2024-09-17T07:30:58.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.708 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.708 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.708 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.306286+0000", 2024-09-17T07:30:58.708 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041275069999999999 2024-09-17T07:30:58.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.709 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.709 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.306319+0000", 2024-09-17T07:30:58.709 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3563999999999999e-05 2024-09-17T07:30:58.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.709 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.709 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.306342+0000", 2024-09-17T07:30:58.709 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.245e-05 2024-09-17T07:30:58.709 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.709 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.709 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.710 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:119988 3.2 3:431863b4:::rbd_data.1183ce4061a0.0000000000000016:head [delete] snapc e=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T07:30:58.710 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:58.301677+0000", 2024-09-17T07:30:58.710 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.295075276, 2024-09-17T07:30:58.710 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045348599999999999, 2024-09-17T07:30:58.710 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:58.710 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:58.710 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:30:58.710 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:58.710 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:58.710 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:58.710 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 119988 2024-09-17T07:30:58.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.710 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:58.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.711 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:58.711 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301677+0000", 2024-09-17T07:30:58.711 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.711 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.711 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:58.711 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301677+0000", 2024-09-17T07:30:58.711 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:58.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.711 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.711 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:58.711 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301678+0000", 2024-09-17T07:30:58.711 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1500000000000004e-07 2024-09-17T07:30:58.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.712 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.712 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:58.712 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301683+0000", 2024-09-17T07:30:58.712 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4689999999999996e-06 2024-09-17T07:30:58.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.712 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.712 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:58.712 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301693+0000", 2024-09-17T07:30:58.712 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1989999999999995e-06 2024-09-17T07:30:58.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.712 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.712 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:58.712 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301700+0000", 2024-09-17T07:30:58.713 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9229999999999996e-06 2024-09-17T07:30:58.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.713 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:58.713 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301759+0000", 2024-09-17T07:30:58.713 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9197999999999998e-05 2024-09-17T07:30:58.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.713 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:58.713 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301826+0000", 2024-09-17T07:30:58.713 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7096999999999998e-05 2024-09-17T07:30:58.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.714 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:30:58.714 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301917+0000", 2024-09-17T07:30:58.714 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1668000000000004e-05 2024-09-17T07:30:58.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.714 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:30:58.714 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.302591+0000", 2024-09-17T07:30:58.714 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00067327800000000003 2024-09-17T07:30:58.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.714 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:30:58.714 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.306145+0000", 2024-09-17T07:30:58.714 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003554504 2024-09-17T07:30:58.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.715 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.715 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:30:58.715 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.306181+0000", 2024-09-17T07:30:58.715 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5952999999999999e-05 2024-09-17T07:30:58.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:58.715 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:58.715 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:58.715 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.306212+0000", 2024-09-17T07:30:58.715 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0956e-05 2024-09-17T07:30:58.715 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.715 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.715 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.715 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:58.715 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:58.716 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:58.717 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:30:58.718 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:30:58.715+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:30:58.818 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:30:58.817+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:30:58.825 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:58.825 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:58.826 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:58.826 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:58.835 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:30:58.919 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:58.919+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:58.941 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:58.942 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:58.942 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:58.942 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:58.942 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:58.950 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:30:59.034 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:30:59.034+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:30:59.099 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:59.100 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:59.100 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:59.100 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:59.100 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.100 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.100 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:59.100 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 120.965959967, 2024-09-17T07:30:59.100 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:30:59.100 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.100 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.101 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.101 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.101 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:30:59.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.101 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:59.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:30:59.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:30:59.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:30:59.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:30:59.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:30:59.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.103 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:30:59.103 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:30:59.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.103 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.103 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:30:59.103 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:30:59.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.103 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.103 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:30:59.103 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:30:59.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:30:59.104 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:30:59.104 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:30:59.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.104 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:30:59.104 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:30:59.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.104 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:30:59.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:30:59.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.105 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.105 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.105 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:30:59.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:30:59.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.105 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.105 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.105 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:30:59.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:30:59.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.105 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.106 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.106 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:30:59.106 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:30:59.106 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.106 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.106 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.106 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.106 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.106 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:59.106 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.659993263, 2024-09-17T07:30:59.106 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:30:59.106 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.106 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.107 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.107 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.107 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.107 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.107 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:30:59.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.107 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.107 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.107 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:59.107 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.108 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:30:59.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.108 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:30:59.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:30:59.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.108 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:30:59.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:30:59.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.109 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.109 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:30:59.109 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:30:59.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.109 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.109 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:30:59.109 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:30:59.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.109 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.109 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:30:59.109 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:30:59.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.110 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.110 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.110 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:30:59.110 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:30:59.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.110 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.110 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.110 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:30:59.110 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:30:59.110 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.110 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.110 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.111 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.111 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.111 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:59.111 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.659971148, 2024-09-17T07:30:59.111 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:30:59.111 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.111 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.111 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.111 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.111 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.111 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.111 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:30:59.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.112 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.112 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.112 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:59.112 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.112 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.112 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:30:59.112 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.113 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.113 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:30:59.113 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:59.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.113 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.113 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:30:59.113 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:30:59.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.113 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.113 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:30:59.113 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:30:59.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.114 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.114 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:30:59.114 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:30:59.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.114 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.114 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:30:59.114 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:30:59.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.114 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.115 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:30:59.115 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:30:59.115 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.115 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.115 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.115 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.115 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.115 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:59.115 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.65994704400001, 2024-09-17T07:30:59.115 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:30:59.115 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.115 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.116 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.116 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.116 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.116 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:30:59.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.116 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.116 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.116 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:59.116 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.117 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.117 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:30:59.117 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.117 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.117 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:30:59.117 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:30:59.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.117 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.117 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:30:59.117 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:30:59.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.118 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.118 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:30:59.118 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:30:59.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.118 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.118 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:30:59.118 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:30:59.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.118 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.119 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:30:59.119 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:30:59.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.119 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.119 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:30:59.119 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:30:59.119 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.119 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.119 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.119 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.120 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:59.120 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.659922955, 2024-09-17T07:30:59.120 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:30:59.120 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.120 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.120 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.120 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.120 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.120 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:30:59.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.120 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.120 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.121 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:59.121 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.121 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.121 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:59.121 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.121 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.121 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:30:59.121 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:30:59.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.122 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.122 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:30:59.122 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:30:59.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.122 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.122 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:30:59.122 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:30:59.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.122 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.122 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:30:59.122 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:30:59.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.123 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.123 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:30:59.123 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:30:59.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.123 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.123 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:30:59.123 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:30:59.123 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.123 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.123 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.124 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.124 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:59.124 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.659898536, 2024-09-17T07:30:59.124 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:30:59.124 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.124 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.124 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.124 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.124 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.124 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.124 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:30:59.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.124 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.125 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.125 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:59.125 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.125 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.125 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:59.125 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.125 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.125 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:30:59.125 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:30:59.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.126 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.126 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:30:59.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:30:59.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.126 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.126 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:30:59.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:30:59.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.126 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.126 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:30:59.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.127 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.128 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:59.128 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.659876836, 2024-09-17T07:30:59.128 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:30:59.128 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.128 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.128 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.128 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.128 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.128 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.128 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:30:59.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.128 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.128 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:59.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.129 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:59.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.129 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:30:59.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:30:59.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.130 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.131 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:30:59.131 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:30:59.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.131 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.131 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:30:59.131 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:30:59.131 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.131 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.131 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.131 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.131 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:59.131 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.659832104, 2024-09-17T07:30:59.132 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:30:59.132 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.132 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.132 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.132 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.132 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.132 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.132 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:30:59.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.132 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.132 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.132 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.132 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:59.133 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.133 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.133 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:30:59.133 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.133 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.133 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:30:59.133 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:30:59.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.133 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.133 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:30:59.134 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:30:59.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.134 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.134 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:30:59.134 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:30:59.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.134 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.134 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:30:59.134 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:30:59.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.134 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.135 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:30:59.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:30:59.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.135 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.135 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.135 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:30:59.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:30:59.135 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.135 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.135 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.135 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.135 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.135 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:59.136 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.659803638, 2024-09-17T07:30:59.136 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:30:59.136 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.136 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.136 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.136 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.136 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.136 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.136 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:30:59.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.136 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.136 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.136 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.136 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:59.137 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.137 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:59.137 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.137 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:30:59.137 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:30:59.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.138 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:30:59.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:30:59.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.138 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.138 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:30:59.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:30:59.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.139 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.139 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:30:59.139 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:30:59.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.139 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.139 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:30:59.139 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:30:59.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.139 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.139 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:30:59.139 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:30:59.140 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.140 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.140 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.140 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.140 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:59.140 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.659686386, 2024-09-17T07:30:59.140 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:30:59.140 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.140 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.140 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.140 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.141 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.141 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.141 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:30:59.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.141 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.141 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.141 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:59.141 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.141 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.141 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:30:59.141 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.142 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.142 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.142 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:30:59.142 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:30:59.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.142 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.142 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.142 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:30:59.142 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:30:59.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.142 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.142 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.143 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:30:59.143 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:30:59.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.143 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.143 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:30:59.143 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:30:59.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.143 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.143 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:30:59.143 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:30:59.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.144 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.144 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.144 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:30:59.144 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:30:59.144 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.144 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.144 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.144 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.144 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.144 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:59.144 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.659260284, 2024-09-17T07:30:59.144 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:30:59.144 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.145 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.145 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.145 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.145 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.145 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.145 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:30:59.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.145 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.145 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.145 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.145 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:59.145 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:30:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:30:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:30:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:30:59.146 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:30:59.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.147 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.147 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.147 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:30:59.147 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:30:59.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.147 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.147 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.147 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:30:59.147 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:30:59.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.147 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.147 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.147 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:30:59.148 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:30:59.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.148 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.148 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.148 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:30:59.148 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:30:59.148 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.148 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.148 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.148 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.148 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.148 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:59.148 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.65924061299999, 2024-09-17T07:30:59.149 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:30:59.149 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.149 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.149 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.149 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.149 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.149 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.149 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:30:59.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.149 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.149 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.149 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:59.150 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.150 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.150 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:59.150 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.150 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.150 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:30:59.150 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:30:59.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.151 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.151 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:30:59.151 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:30:59.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.151 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.151 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:30:59.151 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:30:59.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.151 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.151 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:30:59.151 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:30:59.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.152 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.152 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:30:59.152 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:30:59.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.152 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.152 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:30:59.152 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:30:59.152 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.152 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.152 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.152 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.153 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:59.153 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.65921774900001, 2024-09-17T07:30:59.153 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:30:59.153 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.153 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.153 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.153 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.153 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.153 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.153 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:30:59.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.153 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.153 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.153 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:59.154 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.154 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.154 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:59.154 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.154 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.154 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:30:59.154 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:30:59.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.154 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.155 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:30:59.155 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:30:59.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.155 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.155 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:30:59.155 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:30:59.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.155 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.155 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:30:59.155 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:30:59.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.156 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.156 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:30:59.156 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:30:59.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.156 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.156 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:30:59.156 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:30:59.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.156 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.156 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.156 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.157 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:59.157 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.659196448, 2024-09-17T07:30:59.157 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:30:59.157 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.157 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.157 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.157 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.157 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.157 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:30:59.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.157 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.158 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.158 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:59.158 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.158 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.158 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:30:59.158 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.158 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.158 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:30:59.158 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:30:59.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.160 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:30:59.160 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:30:59.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.160 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:30:59.160 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:30:59.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.160 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.161 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.161 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:59.161 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.65914866, 2024-09-17T07:30:59.161 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:30:59.161 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.161 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.161 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.161 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.161 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.161 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:30:59.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.162 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.162 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.162 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:59.162 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.162 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.162 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:30:59.162 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.163 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.163 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:30:59.163 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:30:59.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.163 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.163 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:30:59.163 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:30:59.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.163 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.163 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:30:59.163 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:30:59.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:30:59.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:30:59.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:30:59.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:30:59.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:30:59.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:30:59.165 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.165 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.165 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.165 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.165 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:59.165 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.659121438, 2024-09-17T07:30:59.165 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:30:59.165 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.165 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.165 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.166 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.166 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.166 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:30:59.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.166 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.166 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.166 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:59.166 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.166 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.166 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:59.166 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.167 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.167 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:30:59.167 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:30:59.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.167 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.167 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:30:59.167 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:30:59.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.167 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.167 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:30:59.167 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:30:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:30:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:30:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:30:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:30:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.168 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.65910105099999, 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.169 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.170 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:30:59.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.170 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.170 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.170 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:59.170 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.170 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.170 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:30:59.170 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.170 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.171 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:30:59.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:30:59.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.171 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.171 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:30:59.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:30:59.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.171 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.171 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:30:59.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:30:59.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.172 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:30:59.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:30:59.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.172 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:30:59.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:30:59.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.172 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:30:59.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:30:59.172 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.173 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.173 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.173 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.173 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:59.173 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.65908134599999, 2024-09-17T07:30:59.173 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:30:59.173 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.173 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.173 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.173 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.173 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.174 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.175 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:30:59.175 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:30:59.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.175 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.175 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:30:59.175 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:30:59.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.175 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.175 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:30:59.175 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:30:59.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.175 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.176 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:30:59.176 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:30:59.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.176 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.176 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:30:59.176 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:30:59.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.176 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.176 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:30:59.176 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:30:59.177 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.177 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.177 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.177 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.177 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:59.177 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.659058268, 2024-09-17T07:30:59.177 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:30:59.177 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.177 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.177 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.177 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.178 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.178 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.178 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:30:59.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.178 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.178 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.178 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:59.178 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.178 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.178 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:30:59.178 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.179 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.179 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:30:59.179 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:30:59.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.179 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.179 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:30:59.179 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:30:59.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.179 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.179 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:30:59.179 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:30:59.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.180 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.180 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:30:59.180 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:30:59.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.180 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.180 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:30:59.180 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:30:59.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.181 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.181 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:30:59.181 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:30:59.181 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.181 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.181 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.182 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:30:59.182 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:59.182 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 110.65903770600001, 2024-09-17T07:30:59.182 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:30:59.182 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.182 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.182 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:30:59.182 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.182 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.182 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.182 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:30:59.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.183 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.183 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:59.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.183 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:59.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.183 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:30:59.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:30:59.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.184 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.184 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:30:59.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:30:59.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.184 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.184 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:30:59.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:30:59.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.184 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.184 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:30:59.185 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:30:59.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.185 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.185 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:30:59.185 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:30:59.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.185 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.185 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:30:59.185 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:30:59.185 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.185 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.185 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.186 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.186 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.186 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:59.187 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:30:59.188 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:30:59.186+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:30:59.211 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T07:30:59.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T07:30:59.316 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:59.316+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:59.331 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:59.332 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:59.332 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:59.332 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:59.341 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:30:59.389 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:59.387+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:59.489 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:59.490 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:59.490 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:30:59.490 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:30:59.490 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:59.490 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:59.489+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:59.499 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:30:59.591 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:59.590+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:59.648 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:59.648 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:30:59.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:30:59.648 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:30:59.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.649 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.649 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:59.649 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.51372751700001, 2024-09-17T07:30:59.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:30:59.649 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.649 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.649 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.649 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.649 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.649 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.649 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:30:59.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.650 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:59.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:30:59.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:30:59.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:30:59.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:30:59.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:30:59.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:30:59.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:30:59.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:30:59.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:30:59.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:30:59.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:30:59.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:30:59.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:30:59.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:30:59.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:30:59.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:30:59.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:30:59.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:30:59.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:30:59.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:30:59.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:30:59.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:30:59.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:30:59.654 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.654 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.655 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.655 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.655 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:59.655 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.51369062000001, 2024-09-17T07:30:59.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:30:59.655 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.655 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.655 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.655 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.655 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.655 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.655 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:30:59.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.656 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:59.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:30:59.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:30:59.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:30:59.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:30:59.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:30:59.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:30:59.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:30:59.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:30:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:30:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:30:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:30:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:30:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:30:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:30:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:30:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:30:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:30:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:30:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:30:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:30:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:30:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:30:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.660 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.661 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.661 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.661 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:59.661 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.51366465300001, 2024-09-17T07:30:59.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:30:59.661 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.661 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.661 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.661 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.661 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.662 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:30:59.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.662 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:59.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:30:59.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:30:59.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:30:59.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:30:59.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:30:59.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:30:59.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:30:59.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:30:59.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:30:59.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:30:59.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:30:59.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:30:59.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:30:59.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:30:59.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:30:59.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:30:59.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:30:59.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:30:59.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:30:59.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:30:59.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:30:59.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:30:59.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:30:59.667 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.667 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.667 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.667 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.667 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:59.667 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.51364005799999, 2024-09-17T07:30:59.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:30:59.667 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.667 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.667 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.668 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.668 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.668 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:30:59.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.668 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:59.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:59.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:30:59.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:30:59.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:30:59.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:30:59.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.670 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:30:59.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:30:59.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.670 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.670 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:30:59.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:30:59.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.670 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:30:59.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:30:59.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:30:59.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:30:59.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.673 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:30:59.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:30:59.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.673 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:30:59.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:30:59.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:30:59.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:30:59.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:30:59.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:30:59.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:30:59.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:30:59.675 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.675 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.675 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.675 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.675 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:59.675 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.513616571, 2024-09-17T07:30:59.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:30:59.675 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.676 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.676 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.676 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.676 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.676 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:30:59.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.676 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:59.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:30:59.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:30:59.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:30:59.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.681 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:30:59.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:30:59.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.681 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.681 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:30:59.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:30:59.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.681 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.681 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:30:59.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:30:59.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:30:59.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:30:59.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:59.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:30:59.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:30:59.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:30:59.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.683 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:30:59.683 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:30:59.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.683 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:30:59.683 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:30:59.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.683 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:30:59.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:30:59.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.684 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.684 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:30:59.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:30:59.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.684 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.684 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.684 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.684 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:59.685 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.51360283299999, 2024-09-17T07:30:59.685 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:30:59.685 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.685 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.685 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.685 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.685 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.685 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.685 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:30:59.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.685 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.685 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:59.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.686 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:59.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.686 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:30:59.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:30:59.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:30:59.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:30:59.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:30:59.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:30:59.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:30:59.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:30:59.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:59.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:30:59.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:59.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:30:59.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:30:59.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:30:59.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.689 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:30:59.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:30:59.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.689 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:30:59.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:30:59.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.690 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.690 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:30:59.690 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:30:59.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.690 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.690 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:30:59.690 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:30:59.690 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.690 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.690 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.690 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.691 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:59.691 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.51357264400001, 2024-09-17T07:30:59.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:30:59.691 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.691 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.691 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.691 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.691 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.691 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.691 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:30:59.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.691 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.691 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.692 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:59.692 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.692 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.692 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:30:59.692 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.692 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.692 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:30:59.692 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:30:59.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:30:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:30:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:30:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:30:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:30:59.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:30:59.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.694 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.694 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:30:59.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:30:59.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.694 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.694 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:30:59.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:30:59.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.694 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:30:59.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:30:59.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:30:59.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:59.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:30:59.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:30:59.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:30:59.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:30:59.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:30:59.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:30:59.696 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.696 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.696 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.697 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.697 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:59.697 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.513536139, 2024-09-17T07:30:59.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:30:59.697 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.697 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.697 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.697 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.697 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.697 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.697 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:30:59.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.697 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.698 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.698 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:59.698 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.698 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.698 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.698 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:59.698 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.698 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.698 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.698 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:30:59.698 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:30:59.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.699 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.699 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:30:59.699 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:30:59.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.699 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.699 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:30:59.699 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:30:59.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.699 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.699 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.699 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:30:59.700 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:30:59.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.700 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.700 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.700 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:30:59.700 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:30:59.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.700 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.700 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.700 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:59.700 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:30:59.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.701 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.701 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:30:59.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:30:59.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.701 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.701 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:30:59.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:30:59.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.701 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.701 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.701 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:30:59.701 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:30:59.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.702 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.702 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:30:59.702 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:30:59.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.702 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.702 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.702 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:30:59.702 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:30:59.702 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.702 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.702 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.703 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.703 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:59.703 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.51350838, 2024-09-17T07:30:59.703 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:30:59.703 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.703 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.703 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.703 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.703 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.703 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.703 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:30:59.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.704 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.704 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.704 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.704 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:59.704 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.704 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.704 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.704 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:30:59.704 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.704 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.705 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.705 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:30:59.705 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:30:59.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.705 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.705 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:30:59.705 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:30:59.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.705 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.705 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.705 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:30:59.705 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:30:59.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.706 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.706 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:30:59.706 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:30:59.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.706 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.706 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:59.706 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:30:59.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.706 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.706 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.706 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:30:59.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:30:59.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.707 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.707 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:30:59.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:30:59.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.707 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.707 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:30:59.707 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:30:59.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.707 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.707 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.708 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:30:59.708 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:30:59.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.708 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.708 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.708 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:30:59.708 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:30:59.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.708 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.708 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.708 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:30:59.708 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:30:59.708 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.708 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.709 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.709 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.709 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:59.709 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.51348622, 2024-09-17T07:30:59.709 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:30:59.709 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.709 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.709 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.709 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.709 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.709 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.709 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:30:59.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.710 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.710 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.710 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:59.710 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.710 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.710 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:59.710 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.710 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.710 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.711 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.711 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:30:59.711 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:30:59.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.711 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.711 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.711 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:30:59.711 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:30:59.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.711 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.711 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.711 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:30:59.711 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:30:59.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.712 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.712 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.712 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:30:59.712 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:30:59.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.712 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.712 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.712 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:30:59.712 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:30:59.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.712 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.712 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.712 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:30:59.713 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:30:59.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.713 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.713 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:30:59.713 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:30:59.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.713 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.713 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:30:59.713 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:30:59.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.713 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.713 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.714 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:30:59.714 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:30:59.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.714 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.714 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:30:59.714 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:30:59.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.714 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.714 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.714 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:30:59.714 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:30:59.714 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.714 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.715 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.715 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.715 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.715 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:59.715 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.51347305900001, 2024-09-17T07:30:59.715 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:30:59.715 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.715 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.715 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.715 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.715 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.715 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.716 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:30:59.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.716 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.716 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.716 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.716 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:59.716 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.716 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.716 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.716 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:59.716 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.716 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.717 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.717 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:30:59.717 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:30:59.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.717 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.717 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.717 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:30:59.717 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:30:59.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.717 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.717 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.717 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:30:59.717 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:30:59.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.718 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.718 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.718 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:30:59.718 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:30:59.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.718 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.718 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.718 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:30:59.718 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:30:59.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.718 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.718 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.719 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:59.719 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:30:59.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.719 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.719 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.719 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:30:59.719 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:30:59.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.719 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.719 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.719 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:30:59.719 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:30:59.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.719 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.720 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.720 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:30:59.720 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:30:59.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.720 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.720 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.720 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:30:59.720 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:30:59.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.720 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.720 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.720 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:30:59.720 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:30:59.721 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.721 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.721 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.721 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.721 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.721 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:59.721 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.513454471, 2024-09-17T07:30:59.721 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:30:59.721 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.721 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.721 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.721 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.721 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.722 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.722 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:30:59.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.722 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.722 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.722 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.722 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:59.722 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.722 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.722 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.722 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:30:59.722 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.723 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.723 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:30:59.723 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:30:59.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.723 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.723 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:30:59.723 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:30:59.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.723 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.723 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.724 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:30:59.724 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:30:59.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.724 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.724 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:30:59.724 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:30:59.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.724 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.724 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:59.724 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:30:59.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.724 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.725 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.725 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:59.725 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:30:59.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.725 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.725 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.725 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:30:59.725 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:30:59.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.725 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.725 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.725 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:30:59.725 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:30:59.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.726 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.726 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:30:59.726 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:30:59.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.726 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.726 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:30:59.726 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:30:59.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.726 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.726 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.727 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:30:59.727 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:30:59.727 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.727 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.727 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.727 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.727 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.727 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:59.727 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.51342166000001, 2024-09-17T07:30:59.727 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:30:59.727 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.727 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.727 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.728 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.728 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.728 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.728 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:30:59.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.728 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.728 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.728 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.728 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:59.728 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.728 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.728 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:30:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:30:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:30:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:30:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.729 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:30:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:30:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:30:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:30:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:30:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:30:59.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.731 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:59.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:59.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.731 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:30:59.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:30:59.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.731 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.731 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.731 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:30:59.731 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:30:59.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.732 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:30:59.732 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:30:59.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.732 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:30:59.732 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:30:59.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.732 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.732 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.733 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:30:59.733 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:30:59.733 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.733 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.733 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.733 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.733 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.733 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:59.733 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.513402603, 2024-09-17T07:30:59.733 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:30:59.733 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.734 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.734 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.734 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.734 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.734 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:30:59.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.734 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.734 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.734 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.734 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:59.734 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.734 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.734 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:30:59.735 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:30:59.735 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:30:59.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.735 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.735 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:30:59.735 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:30:59.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.735 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.736 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.736 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:30:59.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:30:59.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.736 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.736 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:30:59.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:30:59.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.736 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.736 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.736 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:30:59.736 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:30:59.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.737 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.737 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:30:59.737 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:30:59.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.737 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.737 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:30:59.737 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:30:59.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.737 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.737 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.737 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:30:59.738 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:30:59.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.738 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.738 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:30:59.738 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:30:59.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.738 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.738 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:30:59.738 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:30:59.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.738 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.739 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.739 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:30:59.739 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:30:59.739 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.739 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.739 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.739 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.739 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.739 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:59.739 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.513380494, 2024-09-17T07:30:59.739 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:30:59.739 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.739 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.740 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.740 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.740 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.740 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.740 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:30:59.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.740 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.740 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.740 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:59.740 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.740 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.741 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.741 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:30:59.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.741 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.741 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:30:59.741 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:30:59.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.741 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.742 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:30:59.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:30:59.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.742 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:30:59.742 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:30:59.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.742 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.742 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.742 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:30:59.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:30:59.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:30:59.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:30:59.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.743 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.743 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:30:59.743 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:30:59.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.743 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:30:59.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:30:59.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:30:59.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:30:59.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.744 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.744 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.744 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:30:59.744 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:30:59.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.745 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.745 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:30:59.745 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:30:59.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.745 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.745 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.745 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:30:59.745 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:30:59.745 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.745 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.745 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.746 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.746 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.746 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:59.746 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.51335438700001, 2024-09-17T07:30:59.746 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:30:59.746 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.746 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.746 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.746 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.746 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.746 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.746 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:30:59.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.747 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.747 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.747 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:59.747 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.747 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.747 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:30:59.747 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.747 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.748 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.748 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:30:59.748 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:30:59.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.748 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.748 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:30:59.748 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:30:59.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.748 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.748 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.748 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:30:59.749 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:30:59.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.749 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.749 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:30:59.749 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:30:59.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.749 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.749 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:30:59.749 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:30:59.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.749 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.749 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.750 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:30:59.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:30:59.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.750 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.750 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:30:59.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:30:59.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.750 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.750 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.750 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:30:59.750 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:30:59.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:30:59.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:30:59.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:30:59.751 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:30:59.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.751 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.751 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.751 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:30:59.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:30:59.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.752 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.752 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.752 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.752 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.752 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:59.752 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.51332524199999, 2024-09-17T07:30:59.752 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:30:59.752 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.752 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.752 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.752 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.753 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.753 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.753 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:30:59.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.753 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.753 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.753 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:59.753 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.753 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.753 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.753 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:59.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:30:59.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:30:59.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:30:59.754 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:30:59.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.754 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.754 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:30:59.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:30:59.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.755 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.755 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:30:59.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:30:59.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.755 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.755 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:30:59.755 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:30:59.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.755 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.755 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.755 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:30:59.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:30:59.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:30:59.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:30:59.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.756 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:30:59.756 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:30:59.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.756 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.756 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.757 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:30:59.757 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:30:59.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.757 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.757 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:30:59.757 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:30:59.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.757 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.757 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.757 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:30:59.757 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:30:59.757 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.757 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.758 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.758 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.758 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.758 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:59.758 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.51329343899999, 2024-09-17T07:30:59.758 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:30:59.758 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.758 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.758 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.758 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.758 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.758 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.759 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:30:59.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.759 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:59.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.759 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.759 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:30:59.759 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:30:59.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:30:59.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:30:59.760 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:30:59.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.760 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.760 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.760 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:30:59.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:30:59.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.761 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:30:59.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:30:59.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.761 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:30:59.761 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:30:59.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.761 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.761 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.762 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:30:59.762 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:30:59.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.762 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.762 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:30:59.762 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:30:59.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.762 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.762 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.762 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:30:59.762 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:30:59.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.763 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:30:59.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:30:59.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.763 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:30:59.763 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:30:59.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.763 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.763 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.763 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:30:59.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:30:59.764 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.764 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.764 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.764 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.764 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:59.764 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.513266492, 2024-09-17T07:30:59.764 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:30:59.764 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.764 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.764 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.765 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.765 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.765 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.765 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:30:59.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.765 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:59.765 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.765 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.765 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.765 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:30:59.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:30:59.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:30:59.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.766 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:30:59.766 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:30:59.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.766 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.766 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.767 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:30:59.767 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:30:59.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.767 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.767 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:30:59.767 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:30:59.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.767 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.767 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.767 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:30:59.767 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:30:59.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.768 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.768 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:30:59.768 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:30:59.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.768 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.768 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:30:59.768 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:30:59.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.768 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.768 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:30:59.768 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:30:59.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:30:59.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:30:59.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:30:59.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:30:59.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.769 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.769 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.769 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:30:59.769 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:30:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:30:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 121.513225543, 2024-09-17T07:30:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:30:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:30:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:30:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:30:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:30:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:30:59.771 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:30:59.771 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:30:59.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.771 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:30:59.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.771 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:30:59.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:59.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.771 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:30:59.771 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:30:59.771 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:30:59.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.771 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.772 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:30:59.772 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:30:59.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:30:59.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.772 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:30:59.772 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:30:59.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:30:59.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.772 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.772 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:30:59.772 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:30:59.772 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:30:59.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:30:59.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:30:59.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:30:59.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:30:59.773 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:30:59.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.773 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.773 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:30:59.773 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:30:59.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:30:59.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.774 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:30:59.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:30:59.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:30:59.774 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:30:59.774 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:30:59.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.774 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.774 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:30:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:30:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:30:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:30:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:30:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:30:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:30:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:30:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:30:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:30:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:30:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.775 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.776 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:30:59.776 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:30:59.776 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:59.777 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:30:59.778 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:30:59.777+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:30:59.878 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:30:59.877+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:30:59.925 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:30:59.926 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:30:59.926 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:30:59.926 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:30:59.926 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:30:59.936 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:30:59.978 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:30:59.977+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:00.066 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:31:00.113 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:31:00.114 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:31:00.114 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:31:00.115 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:00.115 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:31:00.115 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:31:00.115 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:31:00.115 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:00.116 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:31:00.116 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:31:00.118 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:31:00.119 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:31:00.119 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:31:00.122 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:00.121+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:00.135 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:31:00.136 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:31:00.153 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:31:00.154 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:31:00.170 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:31:00.171 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:31:00.188 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:31:00.188 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:31:00.210 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T07:31:00.221 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:00.221+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:00.229 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T07:31:00.233 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T07:31:00.234 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:00.234 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:00.244 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:31:00.322 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:31:00.321+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:31:00.363 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:00.363 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:00.372 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:31:00.423 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:00.422+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:00.475 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:00.475 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:00.484 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:31:00.527 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:31:00.526+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:31:00.586 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:00.587 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:00.595 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:31:00.624 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:00.623+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:00.724 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:31:00.723+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:31:00.739 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:00.740 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:00.748 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:31:00.825 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:00.825+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:00.892 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:00.892 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:00.902 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:31:00.926 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:00.926+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:01.005 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:01.005 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:01.015 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:31:01.027 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:01.027+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:01.118 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:01.118 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:01.127 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:31:01.128 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:01.127+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:01.222 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-17T07:31:01.222 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T07:31:01.229 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:01.228+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:01.234 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:01.235 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:01.245 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:31:01.249 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T07:31:01.333 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:31:01.333+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:31:01.406 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:01.406 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:01.416 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:31:01.459 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:01.458+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:01.520 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:01.520 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:01.528 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:31:01.532 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:01.531+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:01.633 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:01.633+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:01.672 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:01.672 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:01.680 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:31:01.734 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:01.732+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:01.783 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:01.784 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:01.792 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:31:01.836 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:01.836+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:01.901 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:01.901 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:01.910 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:31:01.952 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:01.951+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:02.035 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:02.035+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:02.058 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:02.058 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:02.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:31:02.087 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T07:31:02.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-17T07:31:02.149 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:02.149+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:02.189 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:02.189 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:02.198 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:31:02.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:31:02.237 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:02.235+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:02.338 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:31:02.337+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:31:02.353 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:02.354 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:02.366 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:31:02.374 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:31:02.374 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:31:02.393 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:31:02.393 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:31:02.410 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:31:02.410 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:31:02.426 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:31:02.427 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:31:02.439 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:02.438+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:02.444 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:31:02.444 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:31:02.462 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:31:02.462 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:31:02.475 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:02.475 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:31:02.475 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:31:02.476 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:31:02.476 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:02.479 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:31:02.479 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:31:02.484 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:31:02.496 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:31:02.496 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:31:02.540 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:02.539+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:02.605 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:02.605 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:31:02.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:31:02.605 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:31:02.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.605 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:02.606 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:31:02.606 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.47132363599999, 2024-09-17T07:31:02.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:31:02.606 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.606 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.606 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:02.606 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.606 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.606 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.606 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:31:02.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.606 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:31:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:31:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:31:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:31:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:31:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:31:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:31:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:31:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.608 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:31:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:31:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:31:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:31:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:31:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:31:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:31:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:31:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:31:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:31:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:31:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:31:02.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:31:02.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:02.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:31:02.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:31:02.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:31:02.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:31:02.611 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.611 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.611 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.612 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.612 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:31:02.612 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.16535693199999, 2024-09-17T07:31:02.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:31:02.612 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.612 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.612 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:02.612 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.612 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.612 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.612 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:31:02.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.613 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:31:02.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:31:02.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:31:02.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:31:02.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:31:02.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:31:02.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.614 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.614 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:31:02.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:31:02.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:31:02.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:31:02.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:31:02.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:31:02.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:02.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:31:02.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:31:02.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:31:02.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:31:02.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.616 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.616 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.616 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:31:02.616 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.165334817, 2024-09-17T07:31:02.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:31:02.617 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.617 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.617 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.617 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.617 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.617 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.617 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:31:02.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.617 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:31:02.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:31:02.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:31:02.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:31:02.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:31:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:31:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:31:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:31:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:31:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:31:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:31:02.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:31:02.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:31:02.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:31:02.620 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.620 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.620 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.621 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.621 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:31:02.621 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.165310713, 2024-09-17T07:31:02.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:31:02.621 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.621 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.621 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.621 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.621 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:31:02.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.621 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:31:02.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:31:02.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:31:02.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:31:02.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.623 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:31:02.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:31:02.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:31:02.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:31:02.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:31:02.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:31:02.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:31:02.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:31:02.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:31:02.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:31:02.625 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.625 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.625 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.625 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.626 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:31:02.626 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.165286624, 2024-09-17T07:31:02.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:31:02.626 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.626 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.626 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.626 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.626 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.626 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.626 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:31:02.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.626 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:31:02.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:31:02.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:31:02.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:31:02.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:31:02.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:31:02.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:31:02.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:31:02.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:31:02.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:31:02.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:31:02.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:31:02.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:31:02.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:31:02.629 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.629 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.630 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.630 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.630 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:31:02.630 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.165262205, 2024-09-17T07:31:02.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:31:02.630 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.630 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.630 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.630 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.630 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:31:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:31:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:31:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:31:02.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:31:02.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:31:02.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:31:02.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:31:02.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:31:02.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:31:02.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:31:02.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:31:02.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:31:02.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:31:02.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:31:02.634 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.634 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.634 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.635 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.635 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:31:02.635 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.165240505, 2024-09-17T07:31:02.635 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:31:02.635 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.635 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.635 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.635 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.635 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.635 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:31:02.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.636 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:31:02.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:31:02.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:31:02.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:31:02.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:31:02.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:31:02.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:31:02.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:31:02.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:31:02.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:31:02.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:31:02.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:31:02.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:31:02.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:31:02.639 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.639 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.639 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.639 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.639 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:31:02.639 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.16519577299999, 2024-09-17T07:31:02.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:31:02.639 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.639 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.639 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.639 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.639 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.640 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.640 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:31:02.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.640 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:31:02.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:31:02.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:31:02.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:31:02.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:31:02.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:31:02.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:31:02.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:31:02.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:31:02.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:31:02.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:31:02.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:31:02.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:31:02.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:31:02.643 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.643 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.643 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.643 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.643 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:31:02.643 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.165167307, 2024-09-17T07:31:02.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:31:02.643 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.643 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.644 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.644 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.644 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.644 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.644 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:31:02.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.644 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.644 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:31:02.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:31:02.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:31:02.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:31:02.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:31:02.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:31:02.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:31:02.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:31:02.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:31:02.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:31:02.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:31:02.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:31:02.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.647 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:31:02.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:31:02.647 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.647 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.647 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.647 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.647 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:31:02.647 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.16505005499999, 2024-09-17T07:31:02.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:31:02.648 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.648 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.648 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.648 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.648 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.648 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:31:02.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.648 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.648 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:31:02.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:31:02.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:31:02.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:31:02.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:31:02.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:31:02.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:31:02.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:31:02.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:31:02.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:31:02.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:31:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:31:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:31:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:31:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.651 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:31:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.164623953, 2024-09-17T07:31:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:31:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.652 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:31:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:31:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:31:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:31:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:31:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:31:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:31:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:31:02.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:31:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:31:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:31:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:31:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:31:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:31:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:31:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:31:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.164604282, 2024-09-17T07:31:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:31:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:31:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:31:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:31:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:31:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:31:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:31:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:31:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:31:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:31:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:31:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:31:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:31:02.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:31:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:31:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:31:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:31:02.660 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.164581418, 2024-09-17T07:31:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:31:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:31:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:31:02.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:31:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:31:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:31:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:31:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:31:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:31:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:31:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:31:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:31:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:31:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:31:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:31:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:31:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.664 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:31:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.16456011699999, 2024-09-17T07:31:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:31:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:31:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:31:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:31:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:31:02.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:31:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:31:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:31:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:31:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:31:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:31:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:31:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:31:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:31:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:31:02.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:31:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:31:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.164512329, 2024-09-17T07:31:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:31:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.669 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:31:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:31:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:31:02.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:31:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:31:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:31:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:31:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:31:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:31:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:31:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:31:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:31:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:31:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.672 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:31:02.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:31:02.673 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.673 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.673 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.673 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.676 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:31:02.676 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.164485107, 2024-09-17T07:31:02.676 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:31:02.677 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.677 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.677 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.677 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.677 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.677 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:31:02.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.677 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.677 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.677 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:31:02.677 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.678 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.678 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:31:02.678 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.678 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.678 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:31:02.678 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:31:02.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.678 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.678 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:31:02.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:31:02.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:31:02.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:31:02.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:31:02.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:31:02.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:31:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:31:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:31:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:31:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.681 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.681 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:31:02.681 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.16446472, 2024-09-17T07:31:02.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:31:02.681 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.681 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.681 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.681 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.681 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.681 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:31:02.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.681 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:31:02.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:31:02.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.683 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:31:02.683 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:31:02.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.683 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:31:02.683 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:31:02.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.684 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.684 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:31:02.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:31:02.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.684 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.684 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:31:02.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:31:02.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.684 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.685 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:31:02.685 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:31:02.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.685 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.685 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:31:02.685 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:31:02.685 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.685 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.685 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.685 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.686 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:31:02.686 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.164445015, 2024-09-17T07:31:02.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:31:02.686 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.686 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.686 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.686 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.686 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.686 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:31:02.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.686 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.686 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:31:02.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:31:02.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:31:02.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:31:02.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:31:02.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:31:02.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:31:02.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:31:02.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:31:02.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:31:02.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.689 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:31:02.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:31:02.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.689 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:31:02.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:31:02.689 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.689 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.689 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.690 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.690 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:31:02.690 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.164421937, 2024-09-17T07:31:02.690 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:31:02.690 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.690 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.690 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.690 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.690 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.690 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.690 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:31:02.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.690 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.691 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.691 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:31:02.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.691 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.691 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:31:02.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.691 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.691 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:31:02.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:31:02.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.692 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.692 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:31:02.692 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:31:02.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.692 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.692 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:31:02.692 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:31:02.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.692 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.692 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:31:02.692 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:31:02.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.693 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.693 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:31:02.693 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:31:02.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.693 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.693 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:31:02.693 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:31:02.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.693 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.694 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:02.694 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:31:02.694 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 114.164401375, 2024-09-17T07:31:02.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:31:02.694 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:02.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:02.694 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:02.694 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:02.694 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:02.694 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:02.694 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:31:02.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.694 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:31:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:31:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:31:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:31:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:31:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:31:02.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.697 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:02.697 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:31:02.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:31:02.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:02.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:02.697 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:02.697 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:31:02.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:31:02.697 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.697 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.697 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.697 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:02.698 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:02.698 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:02.699 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:31:02.700 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:02.698+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:02.800 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:02.799+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:02.807 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:31:02.807 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:31:02.807 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:02.817 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:31:02.901 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:02.901+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:02.923 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:02.923 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:31:02.923 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:31:02.924 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:31:02.924 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:02.932 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:31:03.002 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:31:03.001+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:31:03.082 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:03.082 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:31:03.082 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:31:03.082 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:31:03.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.082 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14441 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.082 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:31:03.082 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.94702487799999, 2024-09-17T07:31:03.082 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0067779960000001, 2024-09-17T07:31:03.082 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.083 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.083 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.083 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.083 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.083 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14441 2024-09-17T07:31:03.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.083 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.083 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.083 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.083 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:31:03.083 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.084 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.084 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132890+0000", 2024-09-17T07:31:03.084 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.084 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.084 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.084 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132891+0000", 2024-09-17T07:31:03.084 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9599999999999999e-07 2024-09-17T07:31:03.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.085 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.085 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132894+0000", 2024-09-17T07:31:03.085 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7350000000000001e-06 2024-09-17T07:31:03.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.085 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.085 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.085 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132900+0000", 2024-09-17T07:31:03.086 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8970000000000002e-06 2024-09-17T07:31:03.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.086 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.086 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.086 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132906+0000", 2024-09-17T07:31:03.086 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7180000000000001e-06 2024-09-17T07:31:03.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.086 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.086 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.086 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133018+0000", 2024-09-17T07:31:03.087 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111517 2024-09-17T07:31:03.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.087 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.087 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133020+0000", 2024-09-17T07:31:03.087 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5509999999999998e-06 2024-09-17T07:31:03.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.087 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.087 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135324+0000", 2024-09-17T07:31:03.087 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023037769999999999 2024-09-17T07:31:03.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.087 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.087 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.087 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135332+0000", 2024-09-17T07:31:03.088 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1189999999999997e-06 2024-09-17T07:31:03.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.088 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.088 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139479+0000", 2024-09-17T07:31:03.088 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0041471360000001 2024-09-17T07:31:03.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.088 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.088 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139555+0000", 2024-09-17T07:31:03.088 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6073000000000002e-05 2024-09-17T07:31:03.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.088 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.088 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.089 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139668+0000", 2024-09-17T07:31:03.089 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000112977 2024-09-17T07:31:03.089 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.089 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.089 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.089 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14442 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6029312~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.089 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:31:03.089 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.94698798100001, 2024-09-17T07:31:03.089 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006956682, 2024-09-17T07:31:03.089 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.090 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.090 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.090 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.090 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.090 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14442 2024-09-17T07:31:03.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.090 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.091 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.091 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:31:03.091 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.091 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.091 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132927+0000", 2024-09-17T07:31:03.091 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.091 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.091 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.091 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132928+0000", 2024-09-17T07:31:03.091 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3700000000000001e-07 2024-09-17T07:31:03.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.092 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.092 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132932+0000", 2024-09-17T07:31:03.092 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7419999999999997e-06 2024-09-17T07:31:03.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.092 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.092 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132940+0000", 2024-09-17T07:31:03.092 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:31:03.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.092 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.092 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.092 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132945+0000", 2024-09-17T07:31:03.093 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6210000000000001e-06 2024-09-17T07:31:03.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.093 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.093 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133050+0000", 2024-09-17T07:31:03.093 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105023 2024-09-17T07:31:03.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.093 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.094 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.094 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:31:03.094 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8210000000000001e-06 2024-09-17T07:31:03.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.094 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.094 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135382+0000", 2024-09-17T07:31:03.094 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023263170000000001 2024-09-17T07:31:03.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.094 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.094 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.094 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135391+0000", 2024-09-17T07:31:03.095 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4539999999999996e-06 2024-09-17T07:31:03.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.095 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.095 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139714+0000", 2024-09-17T07:31:03.095 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0043228019999999 2024-09-17T07:31:03.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.095 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.095 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139763+0000", 2024-09-17T07:31:03.095 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.952e-05 2024-09-17T07:31:03.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.095 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.095 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.096 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139884+0000", 2024-09-17T07:31:03.096 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012058200000000001 2024-09-17T07:31:03.096 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.096 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.096 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.096 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14443 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6815744~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.096 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:31:03.097 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.94696201399999, 2024-09-17T07:31:03.097 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0071900469999999, 2024-09-17T07:31:03.097 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.097 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.097 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.097 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.097 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.097 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.097 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14443 2024-09-17T07:31:03.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.097 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.097 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.097 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.097 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:31:03.098 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.098 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132953+0000", 2024-09-17T07:31:03.098 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.098 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132954+0000", 2024-09-17T07:31:03.098 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:31:03.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.098 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.098 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.099 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132955+0000", 2024-09-17T07:31:03.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8279999999999999e-06 2024-09-17T07:31:03.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.099 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.099 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132961+0000", 2024-09-17T07:31:03.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8109999999999999e-06 2024-09-17T07:31:03.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.099 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.099 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132966+0000", 2024-09-17T07:31:03.099 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2059999999999998e-06 2024-09-17T07:31:03.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133080+0000", 2024-09-17T07:31:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011411799999999999 2024-09-17T07:31:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133083+0000", 2024-09-17T07:31:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3099999999999999e-06 2024-09-17T07:31:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135425+0000", 2024-09-17T07:31:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002342692 2024-09-17T07:31:03.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135434+0000", 2024-09-17T07:31:03.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.799e-06 2024-09-17T07:31:03.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139934+0000", 2024-09-17T07:31:03.101 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004499955 2024-09-17T07:31:03.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.101 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.101 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140032+0000", 2024-09-17T07:31:03.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7498999999999996e-05 2024-09-17T07:31:03.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.102 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.102 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140143+0000", 2024-09-17T07:31:03.102 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00011147299999999999 2024-09-17T07:31:03.102 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.102 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.102 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.102 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14444 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7602176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.102 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:31:03.102 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.94693741899999, 2024-09-17T07:31:03.103 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.007388854, 2024-09-17T07:31:03.103 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.103 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.103 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.103 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.103 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.103 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.103 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14444 2024-09-17T07:31:03.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.103 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.103 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.103 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.103 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:31:03.104 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.104 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:31:03.104 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.104 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.104 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132978+0000", 2024-09-17T07:31:03.104 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4900000000000001e-07 2024-09-17T07:31:03.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.104 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.105 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.105 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132981+0000", 2024-09-17T07:31:03.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3420000000000002e-06 2024-09-17T07:31:03.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.105 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.105 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.105 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132987+0000", 2024-09-17T07:31:03.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5470000000000002e-06 2024-09-17T07:31:03.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.105 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.105 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.105 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132992+0000", 2024-09-17T07:31:03.105 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.899e-06 2024-09-17T07:31:03.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.106 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.106 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.106 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133111+0000", 2024-09-17T07:31:03.106 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000119029 2024-09-17T07:31:03.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.106 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.106 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.107 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133116+0000", 2024-09-17T07:31:03.107 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9860000000000002e-06 2024-09-17T07:31:03.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.107 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.107 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.107 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135463+0000", 2024-09-17T07:31:03.107 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023474799999999999 2024-09-17T07:31:03.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.108 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135470+0000", 2024-09-17T07:31:03.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0190000000000003e-06 2024-09-17T07:31:03.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.108 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.108 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140172+0000", 2024-09-17T07:31:03.108 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004702239 2024-09-17T07:31:03.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.108 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140261+0000", 2024-09-17T07:31:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9544999999999997e-05 2024-09-17T07:31:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140367+0000", 2024-09-17T07:31:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000105319 2024-09-17T07:31:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.109 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14461 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2359296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.110 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:31:03.110 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.946913932, 2024-09-17T07:31:03.110 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0075574679999999, 2024-09-17T07:31:03.110 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.110 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.110 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.110 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.110 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.110 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.110 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14461 2024-09-17T07:31:03.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.110 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.110 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.110 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.111 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:31:03.111 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.111 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.111 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.111 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133001+0000", 2024-09-17T07:31:03.111 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.111 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.111 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.111 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133002+0000", 2024-09-17T07:31:03.111 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:31:03.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.112 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.112 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133004+0000", 2024-09-17T07:31:03.112 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.181e-06 2024-09-17T07:31:03.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.112 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.112 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.112 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133008+0000", 2024-09-17T07:31:03.112 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2289999999999997e-06 2024-09-17T07:31:03.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.113 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.113 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133011+0000", 2024-09-17T07:31:03.113 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8729999999999999e-06 2024-09-17T07:31:03.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.113 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.113 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133143+0000", 2024-09-17T07:31:03.113 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000131636 2024-09-17T07:31:03.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.113 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.113 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.113 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:31:03.114 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2809999999999998e-06 2024-09-17T07:31:03.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.114 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.114 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135514+0000", 2024-09-17T07:31:03.114 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023695919999999998 2024-09-17T07:31:03.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.114 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.114 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135522+0000", 2024-09-17T07:31:03.114 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7139999999999993e-06 2024-09-17T07:31:03.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.114 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.114 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.115 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140406+0000", 2024-09-17T07:31:03.115 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.004883733 2024-09-17T07:31:03.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.115 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.115 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.115 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140471+0000", 2024-09-17T07:31:03.115 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5582999999999999e-05 2024-09-17T07:31:03.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.115 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.115 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.115 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140559+0000", 2024-09-17T07:31:03.115 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.7267000000000003e-05 2024-09-17T07:31:03.116 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.116 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.116 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.116 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.116 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14462 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3145728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.116 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:31:03.116 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.94690019399999, 2024-09-17T07:31:03.116 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0077213490000001, 2024-09-17T07:31:03.116 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.116 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.116 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.116 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.116 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.117 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.117 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14462 2024-09-17T07:31:03.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.117 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.117 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.117 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:31:03.117 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.118 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.118 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:31:03.118 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.118 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.118 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133015+0000", 2024-09-17T07:31:03.118 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2399999999999999e-07 2024-09-17T07:31:03.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.118 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.118 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133019+0000", 2024-09-17T07:31:03.118 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4359999999999998e-06 2024-09-17T07:31:03.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.119 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.119 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133022+0000", 2024-09-17T07:31:03.119 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1999999999999999e-06 2024-09-17T07:31:03.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.119 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.120 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133026+0000", 2024-09-17T07:31:03.120 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3030000000000001e-06 2024-09-17T07:31:03.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.120 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.120 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:31:03.120 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00014558400000000001 2024-09-17T07:31:03.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.120 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.120 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:31:03.120 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3729999999999998e-06 2024-09-17T07:31:03.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.121 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.121 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135553+0000", 2024-09-17T07:31:03.121 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023742799999999999 2024-09-17T07:31:03.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.121 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.121 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135559+0000", 2024-09-17T07:31:03.121 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3459999999999999e-06 2024-09-17T07:31:03.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.121 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.122 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140592+0000", 2024-09-17T07:31:03.122 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0050333520000001 2024-09-17T07:31:03.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.122 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.122 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140645+0000", 2024-09-17T07:31:03.122 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2420999999999998e-05 2024-09-17T07:31:03.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.122 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.122 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140736+0000", 2024-09-17T07:31:03.122 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1630000000000002e-05 2024-09-17T07:31:03.122 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.123 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.123 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.123 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14463 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3932160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.123 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:31:03.123 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.94687000499999, 2024-09-17T07:31:03.123 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0078705299999999, 2024-09-17T07:31:03.123 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.123 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.123 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.123 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.123 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.124 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.124 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14463 2024-09-17T07:31:03.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.124 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.124 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.124 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:31:03.124 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.124 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.124 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133045+0000", 2024-09-17T07:31:03.124 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.125 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.125 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133046+0000", 2024-09-17T07:31:03.125 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.34e-07 2024-09-17T07:31:03.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.125 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.125 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133048+0000", 2024-09-17T07:31:03.125 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2570000000000002e-06 2024-09-17T07:31:03.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.126 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.126 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133055+0000", 2024-09-17T07:31:03.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8090000000000004e-06 2024-09-17T07:31:03.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.126 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.126 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133064+0000", 2024-09-17T07:31:03.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6889999999999999e-06 2024-09-17T07:31:03.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.126 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.126 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133207+0000", 2024-09-17T07:31:03.126 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000142459 2024-09-17T07:31:03.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.127 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.127 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133209+0000", 2024-09-17T07:31:03.127 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2639999999999998e-06 2024-09-17T07:31:03.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.127 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.127 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135589+0000", 2024-09-17T07:31:03.127 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0023801209999999998 2024-09-17T07:31:03.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.128 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.128 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135593+0000", 2024-09-17T07:31:03.128 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:31:03.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.128 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.128 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140775+0000", 2024-09-17T07:31:03.128 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005181493 2024-09-17T07:31:03.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.128 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.128 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140831+0000", 2024-09-17T07:31:03.128 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6480000000000001e-05 2024-09-17T07:31:03.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.129 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.129 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.129 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140916+0000", 2024-09-17T07:31:03.129 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4351999999999995e-05 2024-09-17T07:31:03.129 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.129 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.129 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.130 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.130 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14464 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4718592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.130 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:31:03.130 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.9468335, 2024-09-17T07:31:03.130 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0079763639999999, 2024-09-17T07:31:03.130 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.130 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.130 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.130 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.130 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14464 2024-09-17T07:31:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:31:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:31:03.131 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133082+0000", 2024-09-17T07:31:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1899999999999998e-07 2024-09-17T07:31:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133088+0000", 2024-09-17T07:31:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0220000000000001e-06 2024-09-17T07:31:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.132 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133095+0000", 2024-09-17T07:31:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4540000000000001e-06 2024-09-17T07:31:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133101+0000", 2024-09-17T07:31:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.772e-06 2024-09-17T07:31:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133234+0000", 2024-09-17T07:31:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000133208 2024-09-17T07:31:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.133 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:31:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2830000000000001e-06 2024-09-17T07:31:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135642+0000", 2024-09-17T07:31:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024026220000000001 2024-09-17T07:31:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135650+0000", 2024-09-17T07:31:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5820000000000004e-06 2024-09-17T07:31:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.134 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140932+0000", 2024-09-17T07:31:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0052822400000001 2024-09-17T07:31:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.140980+0000", 2024-09-17T07:31:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7497999999999998e-05 2024-09-17T07:31:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141058+0000", 2024-09-17T07:31:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8363999999999997e-05 2024-09-17T07:31:03.135 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14465 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5505024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:31:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.94680574100001, 2024-09-17T07:31:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0081327069999999, 2024-09-17T07:31:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.136 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14465 2024-09-17T07:31:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:31:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133109+0000", 2024-09-17T07:31:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133110+0000", 2024-09-17T07:31:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.96e-07 2024-09-17T07:31:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133112+0000", 2024-09-17T07:31:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.165e-06 2024-09-17T07:31:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133119+0000", 2024-09-17T07:31:03.138 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1080000000000004e-06 2024-09-17T07:31:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133124+0000", 2024-09-17T07:31:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2299999999999999e-06 2024-09-17T07:31:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:31:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000140245 2024-09-17T07:31:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.139 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133267+0000", 2024-09-17T07:31:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2929999999999999e-06 2024-09-17T07:31:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135693+0000", 2024-09-17T07:31:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024258420000000001 2024-09-17T07:31:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135700+0000", 2024-09-17T07:31:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:31:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.140 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141097+0000", 2024-09-17T07:31:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0053971850000001 2024-09-17T07:31:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141149+0000", 2024-09-17T07:31:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2543000000000002e-05 2024-09-17T07:31:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141242+0000", 2024-09-17T07:31:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2849999999999994e-05 2024-09-17T07:31:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.141 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14466 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6291456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:31:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.94678358100001, 2024-09-17T07:31:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008250855, 2024-09-17T07:31:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.142 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14466 2024-09-17T07:31:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:31:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:31:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.143 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133132+0000", 2024-09-17T07:31:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.7500000000000001e-07 2024-09-17T07:31:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133134+0000", 2024-09-17T07:31:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0310000000000001e-06 2024-09-17T07:31:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133138+0000", 2024-09-17T07:31:03.144 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.172e-06 2024-09-17T07:31:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133141+0000", 2024-09-17T07:31:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7010000000000001e-06 2024-09-17T07:31:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133298+0000", 2024-09-17T07:31:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015699600000000001 2024-09-17T07:31:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.145 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133303+0000", 2024-09-17T07:31:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6979999999999997e-06 2024-09-17T07:31:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135740+0000", 2024-09-17T07:31:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00243736 2024-09-17T07:31:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135744+0000", 2024-09-17T07:31:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5210000000000004e-06 2024-09-17T07:31:03.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141257+0000", 2024-09-17T07:31:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0055126780000001 2024-09-17T07:31:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141304+0000", 2024-09-17T07:31:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480999999999998e-05 2024-09-17T07:31:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.147 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141382+0000", 2024-09-17T07:31:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8842000000000002e-05 2024-09-17T07:31:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14467 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7077888~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:31:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.94677042000001, 2024-09-17T07:31:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0084104789999999, 2024-09-17T07:31:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.148 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14467 2024-09-17T07:31:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:31:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.149 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:31:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133145+0000", 2024-09-17T07:31:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7399999999999999e-07 2024-09-17T07:31:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133146+0000", 2024-09-17T07:31:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.184e-06 2024-09-17T07:31:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.150 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133152+0000", 2024-09-17T07:31:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7010000000000001e-06 2024-09-17T07:31:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133157+0000", 2024-09-17T07:31:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0590000000000002e-06 2024-09-17T07:31:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133328+0000", 2024-09-17T07:31:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017080200000000001 2024-09-17T07:31:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.151 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.152 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.152 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:31:03.152 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2050000000000001e-06 2024-09-17T07:31:03.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.152 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.152 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135785+0000", 2024-09-17T07:31:03.152 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024552160000000001 2024-09-17T07:31:03.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.152 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.152 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.152 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135793+0000", 2024-09-17T07:31:03.152 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0190000000000006e-06 2024-09-17T07:31:03.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.153 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.153 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.153 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141420+0000", 2024-09-17T07:31:03.153 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.005626548 2024-09-17T07:31:03.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.154 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.154 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141471+0000", 2024-09-17T07:31:03.154 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1431000000000003e-05 2024-09-17T07:31:03.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.154 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.154 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141555+0000", 2024-09-17T07:31:03.154 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4040000000000002e-05 2024-09-17T07:31:03.154 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.154 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.154 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.154 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.155 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14468 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7864320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.155 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:31:03.155 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.946751832, 2024-09-17T07:31:03.155 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.00853225, 2024-09-17T07:31:03.155 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.155 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.155 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.155 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.155 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.155 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.155 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14468 2024-09-17T07:31:03.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.155 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.155 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.156 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.156 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:31:03.156 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.156 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.156 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133163+0000", 2024-09-17T07:31:03.156 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.156 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.156 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.156 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133164+0000", 2024-09-17T07:31:03.156 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0600000000000001e-07 2024-09-17T07:31:03.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.157 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.157 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133166+0000", 2024-09-17T07:31:03.157 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.933e-06 2024-09-17T07:31:03.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.157 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.157 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133172+0000", 2024-09-17T07:31:03.157 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8900000000000004e-06 2024-09-17T07:31:03.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.157 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.157 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.157 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133178+0000", 2024-09-17T07:31:03.158 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0680000000000002e-06 2024-09-17T07:31:03.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.158 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.158 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:31:03.158 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000183015 2024-09-17T07:31:03.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.158 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.158 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:31:03.158 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0669999999999999e-06 2024-09-17T07:31:03.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.158 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135844+0000", 2024-09-17T07:31:03.159 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002477964 2024-09-17T07:31:03.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135848+0000", 2024-09-17T07:31:03.159 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7180000000000004e-06 2024-09-17T07:31:03.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.159 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.159 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141573+0000", 2024-09-17T07:31:03.159 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0057242710000001 2024-09-17T07:31:03.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.159 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.160 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141621+0000", 2024-09-17T07:31:03.160 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8529999999999998e-05 2024-09-17T07:31:03.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.160 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.160 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141696+0000", 2024-09-17T07:31:03.160 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4388000000000007e-05 2024-09-17T07:31:03.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.160 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.160 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.160 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.161 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14485 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 2621440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.161 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:31:03.161 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.94671902100001, 2024-09-17T07:31:03.161 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008693418, 2024-09-17T07:31:03.161 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.161 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.161 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.161 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.161 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.161 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14485 2024-09-17T07:31:03.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.161 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.162 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.162 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:31:03.162 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.162 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.162 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:31:03.162 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.162 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.162 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.162 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133196+0000", 2024-09-17T07:31:03.162 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1e-07 2024-09-17T07:31:03.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.163 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.163 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133198+0000", 2024-09-17T07:31:03.163 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6249999999999999e-06 2024-09-17T07:31:03.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.163 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.163 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133203+0000", 2024-09-17T07:31:03.163 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6909999999999996e-06 2024-09-17T07:31:03.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.163 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133206+0000", 2024-09-17T07:31:03.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2030000000000001e-06 2024-09-17T07:31:03.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133390+0000", 2024-09-17T07:31:03.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000184107 2024-09-17T07:31:03.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.164 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.164 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:31:03.164 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:31:03.164 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.164 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.165 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.165 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135878+0000", 2024-09-17T07:31:03.165 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024861000000000002 2024-09-17T07:31:03.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.165 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.165 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135882+0000", 2024-09-17T07:31:03.165 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:31:03.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.165 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.165 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.165 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141760+0000", 2024-09-17T07:31:03.165 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0058781880000001 2024-09-17T07:31:03.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.166 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.166 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141807+0000", 2024-09-17T07:31:03.166 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6610000000000003e-05 2024-09-17T07:31:03.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.166 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.166 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.166 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141890+0000", 2024-09-17T07:31:03.166 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2522999999999996e-05 2024-09-17T07:31:03.166 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.166 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.166 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.167 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.167 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14486 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 3407872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.167 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:31:03.167 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.946699964, 2024-09-17T07:31:03.167 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0088370900000001, 2024-09-17T07:31:03.167 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.167 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.167 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.167 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.167 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.167 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.167 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14486 2024-09-17T07:31:03.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.167 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.168 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.168 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:31:03.168 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.168 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.168 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133215+0000", 2024-09-17T07:31:03.168 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.168 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.168 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133216+0000", 2024-09-17T07:31:03.168 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6800000000000001e-07 2024-09-17T07:31:03.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.169 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.169 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133218+0000", 2024-09-17T07:31:03.169 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.973e-06 2024-09-17T07:31:03.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.169 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.169 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133224+0000", 2024-09-17T07:31:03.169 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2779999999999999e-06 2024-09-17T07:31:03.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.169 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.169 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133229+0000", 2024-09-17T07:31:03.170 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.938e-06 2024-09-17T07:31:03.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.170 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.170 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133422+0000", 2024-09-17T07:31:03.170 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000192976 2024-09-17T07:31:03.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.170 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.170 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133427+0000", 2024-09-17T07:31:03.170 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3249999999999998e-06 2024-09-17T07:31:03.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.170 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.170 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.171 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135907+0000", 2024-09-17T07:31:03.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002479934 2024-09-17T07:31:03.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.171 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.171 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135910+0000", 2024-09-17T07:31:03.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3519999999999998e-06 2024-09-17T07:31:03.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.171 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.171 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141922+0000", 2024-09-17T07:31:03.171 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0060115679999999 2024-09-17T07:31:03.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.171 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.141975+0000", 2024-09-17T07:31:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2735999999999998e-05 2024-09-17T07:31:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142052+0000", 2024-09-17T07:31:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7642e-05 2024-09-17T07:31:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.172 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14487 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4194304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.173 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:31:03.173 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.946677855, 2024-09-17T07:31:03.173 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.008988719, 2024-09-17T07:31:03.173 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.173 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.173 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.173 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.173 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.173 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.173 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14487 2024-09-17T07:31:03.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.173 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.173 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.174 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.174 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:31:03.174 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.174 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.174 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133237+0000", 2024-09-17T07:31:03.174 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.174 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.174 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133238+0000", 2024-09-17T07:31:03.174 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:31:03.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.175 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.175 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133240+0000", 2024-09-17T07:31:03.175 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8819999999999999e-06 2024-09-17T07:31:03.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.175 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.175 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133246+0000", 2024-09-17T07:31:03.175 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9229999999999999e-06 2024-09-17T07:31:03.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.175 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.175 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.175 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133251+0000", 2024-09-17T07:31:03.175 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:31:03.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.176 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.176 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133454+0000", 2024-09-17T07:31:03.176 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203241 2024-09-17T07:31:03.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.176 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.176 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133457+0000", 2024-09-17T07:31:03.176 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6049999999999998e-06 2024-09-17T07:31:03.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.176 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.176 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.177 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135933+0000", 2024-09-17T07:31:03.177 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024758900000000001 2024-09-17T07:31:03.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.177 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.177 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135937+0000", 2024-09-17T07:31:03.177 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.225e-06 2024-09-17T07:31:03.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.177 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.177 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142089+0000", 2024-09-17T07:31:03.177 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006152545 2024-09-17T07:31:03.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.178 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.178 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142141+0000", 2024-09-17T07:31:03.178 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2197000000000002e-05 2024-09-17T07:31:03.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.178 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.178 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142226+0000", 2024-09-17T07:31:03.178 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4683000000000002e-05 2024-09-17T07:31:03.178 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.178 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.178 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.178 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.178 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.179 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14488 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 4980736~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.179 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:31:03.179 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.94665174799999, 2024-09-17T07:31:03.179 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0091012479999999, 2024-09-17T07:31:03.179 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.179 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.179 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.179 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.179 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.179 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.179 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14488 2024-09-17T07:31:03.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.179 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.179 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.179 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.180 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:31:03.180 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.180 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.180 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133263+0000", 2024-09-17T07:31:03.180 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.180 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.180 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.180 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133264+0000", 2024-09-17T07:31:03.180 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.9299999999999999e-07 2024-09-17T07:31:03.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.181 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.181 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133265+0000", 2024-09-17T07:31:03.181 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5910000000000001e-06 2024-09-17T07:31:03.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.181 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.181 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133270+0000", 2024-09-17T07:31:03.181 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:31:03.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.181 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.181 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.181 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133275+0000", 2024-09-17T07:31:03.181 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2750000000000003e-06 2024-09-17T07:31:03.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.182 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.182 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133498+0000", 2024-09-17T07:31:03.182 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022335500000000001 2024-09-17T07:31:03.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.182 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.182 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133504+0000", 2024-09-17T07:31:03.182 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1830000000000002e-06 2024-09-17T07:31:03.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.182 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.183 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135958+0000", 2024-09-17T07:31:03.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024542740000000002 2024-09-17T07:31:03.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.183 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135961+0000", 2024-09-17T07:31:03.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.168e-06 2024-09-17T07:31:03.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.183 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.183 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142242+0000", 2024-09-17T07:31:03.183 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006281186 2024-09-17T07:31:03.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.183 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.184 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.184 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142289+0000", 2024-09-17T07:31:03.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6576999999999998e-05 2024-09-17T07:31:03.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.184 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.184 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142365+0000", 2024-09-17T07:31:03.184 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5951000000000005e-05 2024-09-17T07:31:03.184 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.184 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.184 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.184 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.185 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14489 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 5767168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.185 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:31:03.185 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.94662260299999, 2024-09-17T07:31:03.185 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009235777, 2024-09-17T07:31:03.185 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.185 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.185 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.185 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.185 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.185 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.185 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14489 2024-09-17T07:31:03.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.185 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.185 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.186 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.186 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:31:03.186 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.186 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.186 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:31:03.186 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.186 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.186 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.186 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133293+0000", 2024-09-17T07:31:03.186 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4099999999999999e-07 2024-09-17T07:31:03.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.187 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.187 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133295+0000", 2024-09-17T07:31:03.187 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2450000000000001e-06 2024-09-17T07:31:03.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.187 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.187 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133302+0000", 2024-09-17T07:31:03.187 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7649999999999996e-06 2024-09-17T07:31:03.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.187 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.187 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.187 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133316+0000", 2024-09-17T07:31:03.187 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3679000000000001e-05 2024-09-17T07:31:03.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133533+0000", 2024-09-17T07:31:03.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021743000000000001 2024-09-17T07:31:03.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133535+0000", 2024-09-17T07:31:03.188 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3659999999999999e-06 2024-09-17T07:31:03.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.188 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.188 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.188 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135982+0000", 2024-09-17T07:31:03.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024469639999999998 2024-09-17T07:31:03.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.189 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.189 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.135986+0000", 2024-09-17T07:31:03.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.128e-06 2024-09-17T07:31:03.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.189 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.189 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142400+0000", 2024-09-17T07:31:03.189 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006414135 2024-09-17T07:31:03.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.189 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.189 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.190 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142453+0000", 2024-09-17T07:31:03.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3606000000000001e-05 2024-09-17T07:31:03.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.190 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.190 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142528+0000", 2024-09-17T07:31:03.190 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5018000000000006e-05 2024-09-17T07:31:03.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.190 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.190 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.190 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14490 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 6553600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.191 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:31:03.191 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.9465908, 2024-09-17T07:31:03.191 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.009346802, 2024-09-17T07:31:03.191 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.191 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.191 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.191 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.191 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.191 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.191 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14490 2024-09-17T07:31:03.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.191 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:31:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133324+0000", 2024-09-17T07:31:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133325+0000", 2024-09-17T07:31:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0699999999999998e-07 2024-09-17T07:31:03.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133330+0000", 2024-09-17T07:31:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2839999999999997e-06 2024-09-17T07:31:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133337+0000", 2024-09-17T07:31:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0539999999999997e-06 2024-09-17T07:31:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133342+0000", 2024-09-17T07:31:03.193 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3680000000000001e-06 2024-09-17T07:31:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133562+0000", 2024-09-17T07:31:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021953000000000001 2024-09-17T07:31:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133565+0000", 2024-09-17T07:31:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.627e-06 2024-09-17T07:31:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.194 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136006+0000", 2024-09-17T07:31:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024417660000000002 2024-09-17T07:31:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136009+0000", 2024-09-17T07:31:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0180000000000002e-06 2024-09-17T07:31:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142554+0000", 2024-09-17T07:31:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0065450140000001 2024-09-17T07:31:03.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142600+0000", 2024-09-17T07:31:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5683999999999998e-05 2024-09-17T07:31:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142671+0000", 2024-09-17T07:31:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.1149999999999995e-05 2024-09-17T07:31:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14491 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 7340032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:31:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.946563853, 2024-09-17T07:31:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0095098680000001, 2024-09-17T07:31:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14491 2024-09-17T07:31:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.197 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:31:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133351+0000", 2024-09-17T07:31:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.198 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133352+0000", 2024-09-17T07:31:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9299999999999999e-07 2024-09-17T07:31:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133354+0000", 2024-09-17T07:31:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.007e-06 2024-09-17T07:31:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133361+0000", 2024-09-17T07:31:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5830000000000001e-06 2024-09-17T07:31:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.199 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133366+0000", 2024-09-17T07:31:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8910000000000003e-06 2024-09-17T07:31:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133596+0000", 2024-09-17T07:31:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022975200000000001 2024-09-17T07:31:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133598+0000", 2024-09-17T07:31:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4540000000000001e-06 2024-09-17T07:31:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.200 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136030+0000", 2024-09-17T07:31:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024317969999999999 2024-09-17T07:31:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136033+0000", 2024-09-17T07:31:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0230000000000001e-06 2024-09-17T07:31:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142713+0000", 2024-09-17T07:31:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006680056 2024-09-17T07:31:03.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142772+0000", 2024-09-17T07:31:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9176999999999999e-05 2024-09-17T07:31:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142861+0000", 2024-09-17T07:31:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8734999999999997e-05 2024-09-17T07:31:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14492 3.8 3:107e0f74:::rbd_data.1183b59abaf1.0000000000000004:9 [sparse-read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:31:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 124.94652290400001, 2024-09-17T07:31:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0096399810000001, 2024-09-17T07:31:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14492 2024-09-17T07:31:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.203 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:31:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133392+0000", 2024-09-17T07:31:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133393+0000", 2024-09-17T07:31:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:31:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.204 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133398+0000", 2024-09-17T07:31:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6459999999999998e-06 2024-09-17T07:31:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133406+0000", 2024-09-17T07:31:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7680000000000008e-06 2024-09-17T07:31:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133412+0000", 2024-09-17T07:31:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5060000000000003e-06 2024-09-17T07:31:03.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133615+0000", 2024-09-17T07:31:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000203127 2024-09-17T07:31:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.133617+0000", 2024-09-17T07:31:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4049999999999998e-06 2024-09-17T07:31:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.206 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136049+0000", 2024-09-17T07:31:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0024320119999999999 2024-09-17T07:31:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.136052+0000", 2024-09-17T07:31:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1159999999999999e-06 2024-09-17T07:31:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142898+0000", 2024-09-17T07:31:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006845505 2024-09-17T07:31:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.207 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.142946+0000", 2024-09-17T07:31:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8063999999999999e-05 2024-09-17T07:31:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.143032+0000", 2024-09-17T07:31:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6489000000000005e-05 2024-09-17T07:31:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.208 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.208 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:03.209 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T07:31:03.210 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:31:03.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-17T07:31:03.211 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:03.209+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:03.251 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T07:31:03.310 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:03.310+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:03.365 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:03.366 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:31:03.366 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:31:03.366 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:03.376 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:31:03.411 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:03.410+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:03.485 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:31:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:31:03.486 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:31:03.486 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:03.494 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:31:03.512 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:03.511+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:03.603 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:31:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:31:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:31:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17069 3.0 e41/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 41'110, mlcod=40'108)", 2024-09-17T07:31:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:31:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 120.86268638, 2024-09-17T07:31:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.011408908000000001, 2024-09-17T07:31:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.604 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:31:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739752+0000", 2024-09-17T07:31:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.739753+0000", 2024-09-17T07:31:03.605 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:31:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743532+0000", 2024-09-17T07:31:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037794510000000001 2024-09-17T07:31:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743557+0000", 2024-09-17T07:31:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5253e-05 2024-09-17T07:31:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743569+0000", 2024-09-17T07:31:03.606 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1161999999999999e-05 2024-09-17T07:31:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743650+0000", 2024-09-17T07:31:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1087999999999993e-05 2024-09-17T07:31:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.743674+0000", 2024-09-17T07:31:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4562e-05 2024-09-17T07:31:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751095+0000", 2024-09-17T07:31:03.607 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0074211140000000004 2024-09-17T07:31:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:02.751161+0000", 2024-09-17T07:31:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5707e-05 2024-09-17T07:31:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:17240 3.0 e42/23 3:008e58c8:::rbd_data.11838dd88fc3.0000000000000000:head v 42'113, mlcod=41'110)", 2024-09-17T07:31:03.608 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:31:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 120.377027896, 2024-09-17T07:31:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0064392440000000002, 2024-09-17T07:31:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.609 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:31:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:31:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225411+0000", 2024-09-17T07:31:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1600000000000002e-07 2024-09-17T07:31:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225418+0000", 2024-09-17T07:31:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8360000000000003e-06 2024-09-17T07:31:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.611 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225426+0000", 2024-09-17T07:31:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2670000000000006e-06 2024-09-17T07:31:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225434+0000", 2024-09-17T07:31:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0560000000000005e-06 2024-09-17T07:31:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225514+0000", 2024-09-17T07:31:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9979000000000001e-05 2024-09-17T07:31:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.225535+0000", 2024-09-17T07:31:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0965e-05 2024-09-17T07:31:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231797+0000", 2024-09-17T07:31:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062621129999999997 2024-09-17T07:31:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.613 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:03.231850+0000", 2024-09-17T07:31:03.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2611999999999997e-05 2024-09-17T07:31:03.614 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.614 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.614 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.614 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.614 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33735 3.1f e55/52 3:fea54097:::rbd_data.11839a97b753.0000000000000000:head v 55'260, mlcod=55'258)", 2024-09-17T07:31:03.614 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:31:03.614 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 105.77786901499999, 2024-09-17T07:31:03.614 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035215200000000002, 2024-09-17T07:31:03.614 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.614 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.614 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.614 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:31:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824570+0000", 2024-09-17T07:31:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.824571+0000", 2024-09-17T07:31:03.615 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.164e-06 2024-09-17T07:31:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825118+0000", 2024-09-17T07:31:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054696400000000004 2024-09-17T07:31:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825134+0000", 2024-09-17T07:31:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6342999999999999e-05 2024-09-17T07:31:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825142+0000", 2024-09-17T07:31:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5270000000000001e-06 2024-09-17T07:31:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825200+0000", 2024-09-17T07:31:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.893e-05 2024-09-17T07:31:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.825218+0000", 2024-09-17T07:31:03.617 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7935999999999999e-05 2024-09-17T07:31:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828037+0000", 2024-09-17T07:31:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002818961 2024-09-17T07:31:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.828091+0000", 2024-09-17T07:31:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3695e-05 2024-09-17T07:31:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:33869 3.9 e55/23 3:907b81b5:::rbd_data.11839a97b753.0000000000000001:head v 55'171, mlcod=55'169)", 2024-09-17T07:31:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:31:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 105.760813192, 2024-09-17T07:31:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033990859999999999, 2024-09-17T07:31:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:31:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841625+0000", 2024-09-17T07:31:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.841626+0000", 2024-09-17T07:31:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.023e-06 2024-09-17T07:31:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842067+0000", 2024-09-17T07:31:03.620 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000440612 2024-09-17T07:31:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842080+0000", 2024-09-17T07:31:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.29e-05 2024-09-17T07:31:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842086+0000", 2024-09-17T07:31:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2090000000000002e-06 2024-09-17T07:31:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.621 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842144+0000", 2024-09-17T07:31:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7627999999999998e-05 2024-09-17T07:31:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.842171+0000", 2024-09-17T07:31:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7214000000000001e-05 2024-09-17T07:31:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.844973+0000", 2024-09-17T07:31:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002802232 2024-09-17T07:31:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.622 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:17.845024+0000", 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1267999999999998e-05 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51432 3.0 e59/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 59'177, mlcod=58'174)", 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 100.781216175, 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010790086000000001, 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:31:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821222+0000", 2024-09-17T07:31:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.821224+0000", 2024-09-17T07:31:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.511e-06 2024-09-17T07:31:03.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825174+0000", 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0039497949999999999 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825196+0000", 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2643999999999999e-05 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825206+0000", 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4320000000000005e-06 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.625 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825272+0000", 2024-09-17T07:31:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5829999999999998e-05 2024-09-17T07:31:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.825299+0000", 2024-09-17T07:31:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7107e-05 2024-09-17T07:31:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.831939+0000", 2024-09-17T07:31:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0066398510000000004 2024-09-17T07:31:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.626 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.832012+0000", 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3916000000000002e-05 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:51969 3.0 e59/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 59'179, mlcod=59'177)", 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 100.672347797, 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060297249999999997, 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.627 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:31:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930091+0000", 2024-09-17T07:31:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.930093+0000", 2024-09-17T07:31:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.764e-06 2024-09-17T07:31:03.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931779+0000", 2024-09-17T07:31:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001686379 2024-09-17T07:31:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931802+0000", 2024-09-17T07:31:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2682000000000001e-05 2024-09-17T07:31:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931811+0000", 2024-09-17T07:31:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5640000000000002e-06 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931877+0000", 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6155999999999995e-05 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.931896+0000", 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9111e-05 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936063+0000", 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004166711 2024-09-17T07:31:03.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:22.936121+0000", 2024-09-17T07:31:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7358000000000002e-05 2024-09-17T07:31:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54417 3.0 e60/23 3:00bcad8a:::rbd_data.1183a58293a3.0000000000000000:head v 60'182, mlcod=59'179)", 2024-09-17T07:31:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:31:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 100.164830806, 2024-09-17T07:31:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0059885219999999996, 2024-09-17T07:31:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:31:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437608+0000", 2024-09-17T07:31:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437609+0000", 2024-09-17T07:31:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5829999999999999e-06 2024-09-17T07:31:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437623+0000", 2024-09-17T07:31:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3348e-05 2024-09-17T07:31:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437643+0000", 2024-09-17T07:31:03.633 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0463999999999999e-05 2024-09-17T07:31:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437653+0000", 2024-09-17T07:31:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.6290000000000005e-06 2024-09-17T07:31:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437719+0000", 2024-09-17T07:31:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6428999999999997e-05 2024-09-17T07:31:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.437745+0000", 2024-09-17T07:31:03.634 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5690999999999998e-05 2024-09-17T07:31:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443549+0000", 2024-09-17T07:31:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0058038259999999998 2024-09-17T07:31:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443596+0000", 2024-09-17T07:31:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7552000000000001e-05 2024-09-17T07:31:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.635 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:54419 3.0 e60/23 3:04fe00b9:::rbd_data.1183a58293a3.0000000000000002:head v 60'184, mlcod=59'179)", 2024-09-17T07:31:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:31:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 100.158478449, 2024-09-17T07:31:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004706844, 2024-09-17T07:31:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:31:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.636 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443960+0000", 2024-09-17T07:31:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443961+0000", 2024-09-17T07:31:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:31:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443967+0000", 2024-09-17T07:31:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.268e-06 2024-09-17T07:31:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.637 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443983+0000", 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5248999999999999e-05 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.443990+0000", 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5120000000000002e-06 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444056+0000", 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6197000000000003e-05 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.638 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.444081+0000", 2024-09-17T07:31:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4791e-05 2024-09-17T07:31:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448616+0000", 2024-09-17T07:31:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045346129999999998 2024-09-17T07:31:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:23.448667+0000", 2024-09-17T07:31:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1304000000000002e-05 2024-09-17T07:31:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.639 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:80794 3.9 e72/23 3:90311a61:::rbd_data.1183cb746bb8.0000000000000007:head v 72'222, mlcod=71'220)", 2024-09-17T07:31:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:31:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 86.170489564999997, 2024-09-17T07:31:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004371733, 2024-09-17T07:31:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:31:03.640 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:31:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.431949+0000", 2024-09-17T07:31:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0800000000000001e-07 2024-09-17T07:31:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.641 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434108+0000", 2024-09-17T07:31:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021587569999999999 2024-09-17T07:31:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434120+0000", 2024-09-17T07:31:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1821000000000001e-05 2024-09-17T07:31:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434128+0000", 2024-09-17T07:31:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7579999999999992e-06 2024-09-17T07:31:03.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434191+0000", 2024-09-17T07:31:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3202999999999998e-05 2024-09-17T07:31:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.434218+0000", 2024-09-17T07:31:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6771000000000002e-05 2024-09-17T07:31:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436289+0000", 2024-09-17T07:31:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0020717700000000001 2024-09-17T07:31:03.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:37.436321+0000", 2024-09-17T07:31:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1445000000000001e-05 2024-09-17T07:31:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "MOSDPGPush(3.5 75/74 [PushOp(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head, version: 36'126, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:head@36'126, size: 9431460, copy_subset: [], clone_subset: {3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9: [(0, 9431460)]}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-17T07:31:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:31:03.644 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 81.861581971999996, 2024-09-17T07:31:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041288289999999997, 2024-09-17T07:31:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:31:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740857+0000", 2024-09-17T07:31:03.645 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740858+0000", 2024-09-17T07:31:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1599999999999999e-06 2024-09-17T07:31:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740885+0000", 2024-09-17T07:31:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.707e-05 2024-09-17T07:31:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.646 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740925+0000", 2024-09-17T07:31:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0649000000000003e-05 2024-09-17T07:31:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.740957+0000", 2024-09-17T07:31:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1585999999999999e-05 2024-09-17T07:31:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741185+0000", 2024-09-17T07:31:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00022759700000000001 2024-09-17T07:31:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.647 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.741228+0000", 2024-09-17T07:31:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3705999999999998e-05 2024-09-17T07:31:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:41.744985+0000", 2024-09-17T07:31:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037570609999999999 2024-09-17T07:31:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85061 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'257, mlcod=79'255)", 2024-09-17T07:31:03.648 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:31:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 78.453751015999998, 2024-09-17T07:31:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033850550000000001, 2024-09-17T07:31:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:31:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.649 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:31:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148688+0000", 2024-09-17T07:31:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.9699999999999995e-07 2024-09-17T07:31:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148835+0000", 2024-09-17T07:31:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000146852 2024-09-17T07:31:03.650 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148845+0000", 2024-09-17T07:31:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0232e-05 2024-09-17T07:31:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148852+0000", 2024-09-17T07:31:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6490000000000002e-06 2024-09-17T07:31:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.651 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148913+0000", 2024-09-17T07:31:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1099999999999994e-05 2024-09-17T07:31:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.148931+0000", 2024-09-17T07:31:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7938e-05 2024-09-17T07:31:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152025+0000", 2024-09-17T07:31:03.652 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0030942729999999998 2024-09-17T07:31:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.152073+0000", 2024-09-17T07:31:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7213999999999999e-05 2024-09-17T07:31:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:85063 3.9 e79/23 3:938654d9:::rbd_data.1183c869c667.000000000000002f:head v 79'259, mlcod=79'255)", 2024-09-17T07:31:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:31:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 78.452604062000006, 2024-09-17T07:31:03.653 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037722659999999998, 2024-09-17T07:31:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:31:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:31:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149835+0000", 2024-09-17T07:31:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5799999999999999e-07 2024-09-17T07:31:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149841+0000", 2024-09-17T07:31:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4840000000000003e-06 2024-09-17T07:31:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149849+0000", 2024-09-17T07:31:03.655 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4670000000000004e-06 2024-09-17T07:31:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149854+0000", 2024-09-17T07:31:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4779999999999999e-06 2024-09-17T07:31:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149920+0000", 2024-09-17T07:31:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5760999999999996e-05 2024-09-17T07:31:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.656 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.149949+0000", 2024-09-17T07:31:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8838000000000001e-05 2024-09-17T07:31:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153558+0000", 2024-09-17T07:31:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0036092939999999999 2024-09-17T07:31:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:45.153607+0000", 2024-09-17T07:31:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8386000000000001e-05 2024-09-17T07:31:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:95007 3.5 e82/76 3:a6489fa0:::rbd_data.118311e0d6f8.0000000000000006:head v 82'422, mlcod=82'420)", 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 73.978775768000006, 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003939901, 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.658 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623663+0000", 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.623664+0000", 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3799999999999999e-06 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626104+0000", 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002440149 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.659 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626120+0000", 2024-09-17T07:31:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.5962000000000001e-05 2024-09-17T07:31:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626127+0000", 2024-09-17T07:31:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.8340000000000001e-06 2024-09-17T07:31:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626186+0000", 2024-09-17T07:31:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9234999999999998e-05 2024-09-17T07:31:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.660 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.626208+0000", 2024-09-17T07:31:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1719e-05 2024-09-17T07:31:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627567+0000", 2024-09-17T07:31:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0013587919999999999 2024-09-17T07:31:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:49.627603+0000", 2024-09-17T07:31:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5830000000000001e-05 2024-09-17T07:31:03.661 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:104928 3.18 e87/76 3:1ef990b1:::rbd_data.1183dd51323.0000000000000006:head v 87'562, mlcod=86'559)", 2024-09-17T07:31:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:31:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 68.060020057000003, 2024-09-17T07:31:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0061095869999999997, 2024-09-17T07:31:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.662 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542419+0000", 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.542421+0000", 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.069e-06 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544190+0000", 2024-09-17T07:31:03.663 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017693539999999999 2024-09-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544215+0000", 2024-09-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5398999999999999e-05 2024-09-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544229+0000", 2024-09-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3865e-05 2024-09-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544322+0000", 2024-09-17T07:31:03.664 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2540000000000005e-05 2024-09-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.544348+0000", 2024-09-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6197999999999999e-05 2024-09-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548475+0000", 2024-09-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041266000000000002 2024-09-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:55.548528+0000", 2024-09-17T07:31:03.665 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3562e-05 2024-09-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115898 3.9 3:910101b0:::rbd_data.118340e4fbf.0000000000000000:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.411505951999999, 2024-09-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0042466279999999997, 2024-09-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.666 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115898 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190933+0000", 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.667 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:31:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0030000000000001e-06 2024-09-17T07:31:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190940+0000", 2024-09-17T07:31:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2349999999999999e-06 2024-09-17T07:31:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190953+0000", 2024-09-17T07:31:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3304e-05 2024-09-17T07:31:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.668 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190995+0000", 2024-09-17T07:31:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2317000000000002e-05 2024-09-17T07:31:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191592+0000", 2024-09-17T07:31:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00059607799999999999 2024-09-17T07:31:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191888+0000", 2024-09-17T07:31:03.669 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00029619999999999999 2024-09-17T07:31:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:31:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192069+0000", 2024-09-17T07:31:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00018125899999999999 2024-09-17T07:31:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193692+0000", 2024-09-17T07:31:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001623364 2024-09-17T07:31:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:03.670 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194968+0000", 2024-09-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012755189999999999 2024-09-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.195037+0000", 2024-09-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9450999999999996e-05 2024-09-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.195179+0000", 2024-09-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000141898 2024-09-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.671 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115936 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [watch watch cookie 140227662410368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.378587000000003, 2024-09-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043930569999999997, 2024-09-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.672 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115936 2024-09-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.673 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223852+0000", 2024-09-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7300000000000005e-07 2024-09-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223856+0000", 2024-09-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9169999999999999e-06 2024-09-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223870+0000", 2024-09-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3926e-05 2024-09-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.674 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223878+0000", 2024-09-17T07:31:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3699999999999997e-06 2024-09-17T07:31:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.223934+0000", 2024-09-17T07:31:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6929000000000003e-05 2024-09-17T07:31:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224053+0000", 2024-09-17T07:31:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000118952 2024-09-17T07:31:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.675 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:31:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224271+0000", 2024-09-17T07:31:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021763300000000001 2024-09-17T07:31:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.224992+0000", 2024-09-17T07:31:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072142099999999998 2024-09-17T07:31:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228099+0000", 2024-09-17T07:31:03.676 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0031060850000000002 2024-09-17T07:31:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228143+0000", 2024-09-17T07:31:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4614999999999999e-05 2024-09-17T07:31:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.228245+0000", 2024-09-17T07:31:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000101436 2024-09-17T07:31:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.677 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115955 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [call rbd.snapshot_trash_add in=21b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:31:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:31:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.359600247000003, 2024-09-17T07:31:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004259054, 2024-09-17T07:31:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115955 2024-09-17T07:31:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.678 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:31:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242838+0000", 2024-09-17T07:31:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242839+0000", 2024-09-17T07:31:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6899999999999998e-07 2024-09-17T07:31:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.679 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242842+0000", 2024-09-17T07:31:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.1350000000000001e-06 2024-09-17T07:31:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242849+0000", 2024-09-17T07:31:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0400000000000004e-06 2024-09-17T07:31:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.242854+0000", 2024-09-17T07:31:03.680 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4940000000000003e-06 2024-09-17T07:31:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243137+0000", 2024-09-17T07:31:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00028217100000000001 2024-09-17T07:31:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243196+0000", 2024-09-17T07:31:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9608999999999998e-05 2024-09-17T07:31:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.681 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:31:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243448+0000", 2024-09-17T07:31:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00025185000000000002 2024-09-17T07:31:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243990+0000", 2024-09-17T07:31:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054153199999999995 2024-09-17T07:31:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.246867+0000", 2024-09-17T07:31:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0028771880000000001 2024-09-17T07:31:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.682 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.246943+0000", 2024-09-17T07:31:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5689e-05 2024-09-17T07:31:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247097+0000", 2024-09-17T07:31:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00015487699999999999 2024-09-17T07:31:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.683 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115956 3.9 3:951b62c5:::rbd_header.11838d8fb7ca:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:31:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:31:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 41.359303662999999, 2024-09-17T07:31:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043741479999999996, 2024-09-17T07:31:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115956 2024-09-17T07:31:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.684 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243135+0000", 2024-09-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243136+0000", 2024-09-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7100000000000001e-07 2024-09-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.685 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243141+0000", 2024-09-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.716e-06 2024-09-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243153+0000", 2024-09-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1198e-05 2024-09-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243160+0000", 2024-09-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6229999999999997e-06 2024-09-17T07:31:03.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243797+0000", 2024-09-17T07:31:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063716399999999996 2024-09-17T07:31:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for rw locks", 2024-09-17T07:31:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.243852+0000", 2024-09-17T07:31:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4147999999999997e-05 2024-09-17T07:31:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247147+0000", 2024-09-17T07:31:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0032955940000000002 2024-09-17T07:31:03.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247271+0000", 2024-09-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012397800000000001 2024-09-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.247509+0000", 2024-09-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000237956 2024-09-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.688 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(osd.1.0:2917 3.0 e140/23 3:01b261f8:::rbd_data.11835e2dca9d.0000000000000000:15 v 140'786, mlcod=138'782)", 2024-09-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:49.187821+0000", 2024-09-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.414617963, 2024-09-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037853959999999999, 2024-09-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:49.187821+0000", 2024-09-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.689 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:49.187821+0000", 2024-09-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:49.187821+0000", 2024-09-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7099999999999998e-07 2024-09-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:49.187826+0000", 2024-09-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8040000000000001e-06 2024-09-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.690 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:49.187840+0000", 2024-09-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4446e-05 2024-09-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:49.187849+0000", 2024-09-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2259999999999994e-06 2024-09-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:49.187909+0000", 2024-09-17T07:31:03.691 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9407000000000002e-05 2024-09-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:49.187935+0000", 2024-09-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.5777000000000001e-05 2024-09-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:49.191557+0000", 2024-09-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003622335 2024-09-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.692 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:49.191606+0000", 2024-09-17T07:31:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9129999999999999e-05 2024-09-17T07:31:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(osd.4.0:2652 3.9 e149/120 3:94beb6c8:::rbd_data.1183ce4061a0.0000000000000021:e v 149'506, mlcod=147'503)", 2024-09-17T07:31:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:58.207408+0000", 2024-09-17T07:31:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 5.3950308509999996, 2024-09-17T07:31:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041455720000000001, 2024-09-17T07:31:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:03.693 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.207408+0000", 2024-09-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.207408+0000", 2024-09-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.207408+0000", 2024-09-17T07:31:03.694 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3000000000000002e-07 2024-09-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.207413+0000", 2024-09-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9169999999999997e-06 2024-09-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.207432+0000", 2024-09-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8944999999999999e-05 2024-09-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.207441+0000", 2024-09-17T07:31:03.695 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1679999999999993e-06 2024-09-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.207511+0000", 2024-09-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0232000000000001e-05 2024-09-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.207536+0000", 2024-09-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.4579999999999998e-05 2024-09-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.211510+0000", 2024-09-17T07:31:03.696 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0039738799999999999 2024-09-17T07:31:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.211553+0000", 2024-09-17T07:31:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3519999999999997e-05 2024-09-17T07:31:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:03.697 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:03.697 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:03.699 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:31:03.699 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:03.698+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:03.799 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:03.798+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:03.811 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:31:03.811 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:31:03.812 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:03.820 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:31:03.900 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:31:03.900+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:31:03.965 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:03.965 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:31:03.965 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:31:03.965 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:31:03.965 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:03.973 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:31:04.017 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:04.016+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:04.078 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:04.078 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:31:04.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:31:04.078 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:31:04.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.078 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:30525 3.1b 3:def7c8f0:::rbd_data.118381ffbbab.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T07:31:04.079 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:31:04.079 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 110.554540058, 2024-09-17T07:31:04.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55515850300000003, 2024-09-17T07:31:04.079 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.079 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.079 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:04.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.079 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30525 2024-09-17T07:31:04.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.079 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:31:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:31:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522300+0000", 2024-09-17T07:31:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:31:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522305+0000", 2024-09-17T07:31:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-17T07:31:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522318+0000", 2024-09-17T07:31:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3030999999999999e-05 2024-09-17T07:31:04.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522326+0000", 2024-09-17T07:31:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4200000000000001e-06 2024-09-17T07:31:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522385+0000", 2024-09-17T07:31:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9005000000000002e-05 2024-09-17T07:31:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:31:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522393+0000", 2024-09-17T07:31:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-06 2024-09-17T07:31:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075347+0000", 2024-09-17T07:31:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55295417800000002 2024-09-17T07:31:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075502+0000", 2024-09-17T07:31:04.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154272 2024-09-17T07:31:04.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:31:04.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075630+0000", 2024-09-17T07:31:04.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012816300000000001 2024-09-17T07:31:04.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:31:04.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077221+0000", 2024-09-17T07:31:04.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015912929999999999 2024-09-17T07:31:04.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:04.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077353+0000", 2024-09-17T07:31:04.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013215699999999999 2024-09-17T07:31:04.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:31:04.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077413+0000", 2024-09-17T07:31:04.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0034999999999997e-05 2024-09-17T07:31:04.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077457+0000", 2024-09-17T07:31:04.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4017999999999998e-05 2024-09-17T07:31:04.086 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.086 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.086 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.086 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44181 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13922304~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.086 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:31:04.086 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.43936141, 2024-09-17T07:31:04.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046010502000000002, 2024-09-17T07:31:04.086 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.086 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.086 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.086 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.087 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.087 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.087 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44181 2024-09-17T07:31:04.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.087 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:31:04.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:31:04.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637478+0000", 2024-09-17T07:31:04.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:31:04.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637480+0000", 2024-09-17T07:31:04.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.843e-06 2024-09-17T07:31:04.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637487+0000", 2024-09-17T07:31:04.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6159999999999996e-06 2024-09-17T07:31:04.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637492+0000", 2024-09-17T07:31:04.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5360000000000003e-06 2024-09-17T07:31:04.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683351+0000", 2024-09-17T07:31:04.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045858758999999999 2024-09-17T07:31:04.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683422+0000", 2024-09-17T07:31:04.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1705999999999994e-05 2024-09-17T07:31:04.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683488+0000", 2024-09-17T07:31:04.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5612999999999997e-05 2024-09-17T07:31:04.090 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.090 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.090 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.090 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44182 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13938688~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.090 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:31:04.091 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.439340739, 2024-09-17T07:31:04.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046153118999999999, 2024-09-17T07:31:04.091 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.091 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.091 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.091 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.091 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.091 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.091 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44182 2024-09-17T07:31:04.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.091 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:31:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:31:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:31:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:31:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637506+0000", 2024-09-17T07:31:04.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1520000000000003e-06 2024-09-17T07:31:04.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637512+0000", 2024-09-17T07:31:04.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-09-17T07:31:04.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637517+0000", 2024-09-17T07:31:04.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-09-17T07:31:04.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683514+0000", 2024-09-17T07:31:04.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045997616999999998 2024-09-17T07:31:04.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683584+0000", 2024-09-17T07:31:04.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9942000000000002e-05 2024-09-17T07:31:04.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683651+0000", 2024-09-17T07:31:04.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6951000000000003e-05 2024-09-17T07:31:04.095 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.095 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.095 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.096 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44183 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13955072~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.096 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:31:04.096 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.439315507, 2024-09-17T07:31:04.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046339793999999997, 2024-09-17T07:31:04.096 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.096 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.097 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.097 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44183 2024-09-17T07:31:04.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.097 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:31:04.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:31:04.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639583+0000", 2024-09-17T07:31:04.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002059412 2024-09-17T07:31:04.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639597+0000", 2024-09-17T07:31:04.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3832000000000001e-05 2024-09-17T07:31:04.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639607+0000", 2024-09-17T07:31:04.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485e-05 2024-09-17T07:31:04.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639614+0000", 2024-09-17T07:31:04.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0849999999999999e-06 2024-09-17T07:31:04.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683705+0000", 2024-09-17T07:31:04.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044091261 2024-09-17T07:31:04.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683792+0000", 2024-09-17T07:31:04.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6197000000000002e-05 2024-09-17T07:31:04.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683863+0000", 2024-09-17T07:31:04.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1521999999999997e-05 2024-09-17T07:31:04.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.103 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.103 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.103 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44199 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14217216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.104 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:31:04.104 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.435797955, 2024-09-17T07:31:04.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045995623999999999, 2024-09-17T07:31:04.104 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.104 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.104 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.104 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.104 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44199 2024-09-17T07:31:04.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.104 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:31:04.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:31:04.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:31:04.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:31:04.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641043+0000", 2024-09-17T07:31:04.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.624e-06 2024-09-17T07:31:04.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641048+0000", 2024-09-17T07:31:04.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4029999999999998e-06 2024-09-17T07:31:04.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641053+0000", 2024-09-17T07:31:04.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6670000000000004e-06 2024-09-17T07:31:04.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686918+0000", 2024-09-17T07:31:04.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045865131000000003 2024-09-17T07:31:04.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686986+0000", 2024-09-17T07:31:04.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8299000000000005e-05 2024-09-17T07:31:04.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687036+0000", 2024-09-17T07:31:04.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0164999999999998e-05 2024-09-17T07:31:04.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.108 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.108 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44200 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14233600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.108 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:31:04.109 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.435780126, 2024-09-17T07:31:04.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046183984999999997, 2024-09-17T07:31:04.109 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.109 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.109 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.109 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.109 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44200 2024-09-17T07:31:04.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.109 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:31:04.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:31:04.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:31:04.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:31:04.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641061+0000", 2024-09-17T07:31:04.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.767e-06 2024-09-17T07:31:04.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641067+0000", 2024-09-17T07:31:04.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7379999999999999e-06 2024-09-17T07:31:04.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641071+0000", 2024-09-17T07:31:04.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-09-17T07:31:04.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687078+0000", 2024-09-17T07:31:04.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046007095999999997 2024-09-17T07:31:04.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687158+0000", 2024-09-17T07:31:04.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0374e-05 2024-09-17T07:31:04.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687243+0000", 2024-09-17T07:31:04.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4221999999999994e-05 2024-09-17T07:31:04.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.115 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.116 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44201 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14249984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.116 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:31:04.116 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.43576247999999, 2024-09-17T07:31:04.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046303182999999998, 2024-09-17T07:31:04.116 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.116 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.116 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.117 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.117 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44201 2024-09-17T07:31:04.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.117 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:31:04.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:31:04.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641120+0000", 2024-09-17T07:31:04.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3933000000000001e-05 2024-09-17T07:31:04.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641123+0000", 2024-09-17T07:31:04.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.191e-06 2024-09-17T07:31:04.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641133+0000", 2024-09-17T07:31:04.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4020000000000008e-06 2024-09-17T07:31:04.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641138+0000", 2024-09-17T07:31:04.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6559999999999997e-06 2024-09-17T07:31:04.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687279+0000", 2024-09-17T07:31:04.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046140278999999999 2024-09-17T07:31:04.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687338+0000", 2024-09-17T07:31:04.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9268999999999998e-05 2024-09-17T07:31:04.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687379+0000", 2024-09-17T07:31:04.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1452999999999998e-05 2024-09-17T07:31:04.121 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.121 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.121 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.121 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44202 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14266368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.121 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:31:04.121 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.435691714, 2024-09-17T07:31:04.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046484299, 2024-09-17T07:31:04.121 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.122 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.122 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.122 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.122 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44202 2024-09-17T07:31:04.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.122 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:31:04.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:31:04.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642810+0000", 2024-09-17T07:31:04.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016627980000000001 2024-09-17T07:31:04.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642815+0000", 2024-09-17T07:31:04.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.091e-06 2024-09-17T07:31:04.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642829+0000", 2024-09-17T07:31:04.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4047e-05 2024-09-17T07:31:04.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642844+0000", 2024-09-17T07:31:04.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4515000000000001e-05 2024-09-17T07:31:04.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687426+0000", 2024-09-17T07:31:04.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044582201000000002 2024-09-17T07:31:04.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687529+0000", 2024-09-17T07:31:04.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103078 2024-09-17T07:31:04.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687631+0000", 2024-09-17T07:31:04.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102569 2024-09-17T07:31:04.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.126 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.127 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44209 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14381056~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.127 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:31:04.127 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.43379356200001, 2024-09-17T07:31:04.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045944444000000001, 2024-09-17T07:31:04.127 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.127 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.127 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.128 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44209 2024-09-17T07:31:04.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.128 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:31:04.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:31:04.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643046+0000", 2024-09-17T07:31:04.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:31:04.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643048+0000", 2024-09-17T07:31:04.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:31:04.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643054+0000", 2024-09-17T07:31:04.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.032e-06 2024-09-17T07:31:04.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643065+0000", 2024-09-17T07:31:04.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1861999999999999e-05 2024-09-17T07:31:04.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688806+0000", 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045740617999999997 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688927+0000", 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000121327 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688990+0000", 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2293999999999997e-05 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.132 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44210 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14397440~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.132 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:31:04.132 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.433766636, 2024-09-17T07:31:04.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046104635999999997, 2024-09-17T07:31:04.132 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.132 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.132 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.132 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.132 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44210 2024-09-17T07:31:04.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.133 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:31:04.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:31:04.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643073+0000", 2024-09-17T07:31:04.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.77e-07 2024-09-17T07:31:04.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643075+0000", 2024-09-17T07:31:04.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-09-17T07:31:04.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643082+0000", 2024-09-17T07:31:04.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7789999999999996e-06 2024-09-17T07:31:04.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643087+0000", 2024-09-17T07:31:04.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.967e-06 2024-09-17T07:31:04.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689022+0000", 2024-09-17T07:31:04.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045934349999999999 2024-09-17T07:31:04.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689112+0000", 2024-09-17T07:31:04.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0039999999999999e-05 2024-09-17T07:31:04.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689177+0000", 2024-09-17T07:31:04.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5041000000000003e-05 2024-09-17T07:31:04.136 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.136 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.136 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.136 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44211 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14413824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.137 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:31:04.137 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.433745503, 2024-09-17T07:31:04.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046309003000000001, 2024-09-17T07:31:04.137 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.137 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.137 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.137 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.137 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.137 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.137 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44211 2024-09-17T07:31:04.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.138 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:31:04.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:31:04.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643094+0000", 2024-09-17T07:31:04.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:31:04.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643095+0000", 2024-09-17T07:31:04.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-17T07:31:04.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643101+0000", 2024-09-17T07:31:04.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8730000000000004e-06 2024-09-17T07:31:04.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643106+0000", 2024-09-17T07:31:04.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6859999999999997e-06 2024-09-17T07:31:04.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689220+0000", 2024-09-17T07:31:04.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046113945000000003 2024-09-17T07:31:04.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689309+0000", 2024-09-17T07:31:04.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9560000000000003e-05 2024-09-17T07:31:04.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689402+0000", 2024-09-17T07:31:04.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2786000000000002e-05 2024-09-17T07:31:04.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.145 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.146 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44212 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14430208~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.146 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:31:04.146 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.433700941, 2024-09-17T07:31:04.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046448912000000002, 2024-09-17T07:31:04.146 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.146 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.147 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.147 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44212 2024-09-17T07:31:04.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.148 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:31:04.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:31:04.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:31:04.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2200000000000004e-07 2024-09-17T07:31:04.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643141+0000", 2024-09-17T07:31:04.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5210000000000001e-06 2024-09-17T07:31:04.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643149+0000", 2024-09-17T07:31:04.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6490000000000008e-06 2024-09-17T07:31:04.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643161+0000", 2024-09-17T07:31:04.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2241e-05 2024-09-17T07:31:04.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689423+0000", 2024-09-17T07:31:04.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046262323000000001 2024-09-17T07:31:04.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689505+0000", 2024-09-17T07:31:04.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2103999999999999e-05 2024-09-17T07:31:04.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689587+0000", 2024-09-17T07:31:04.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1452000000000006e-05 2024-09-17T07:31:04.154 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.154 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.154 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.155 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44213 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14446592~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.155 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:31:04.155 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.433669442, 2024-09-17T07:31:04.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046601780000000002, 2024-09-17T07:31:04.156 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.156 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.156 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.157 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.157 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44213 2024-09-17T07:31:04.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.158 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:31:04.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:31:04.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643170+0000", 2024-09-17T07:31:04.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-17T07:31:04.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643172+0000", 2024-09-17T07:31:04.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1660000000000001e-06 2024-09-17T07:31:04.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643180+0000", 2024-09-17T07:31:04.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-09-17T07:31:04.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643185+0000", 2024-09-17T07:31:04.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-09-17T07:31:04.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689622+0000", 2024-09-17T07:31:04.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046436742000000003 2024-09-17T07:31:04.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689697+0000", 2024-09-17T07:31:04.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5179999999999995e-05 2024-09-17T07:31:04.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689771+0000", 2024-09-17T07:31:04.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3928000000000001e-05 2024-09-17T07:31:04.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.164 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.164 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44214 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14462976~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.164 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:31:04.164 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.43364638200001, 2024-09-17T07:31:04.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046736038000000001, 2024-09-17T07:31:04.164 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.164 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.165 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.165 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44214 2024-09-17T07:31:04.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.165 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:31:04.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:31:04.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643193+0000", 2024-09-17T07:31:04.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:31:04.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643195+0000", 2024-09-17T07:31:04.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0640000000000001e-06 2024-09-17T07:31:04.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643201+0000", 2024-09-17T07:31:04.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2859999999999996e-06 2024-09-17T07:31:04.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643206+0000", 2024-09-17T07:31:04.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-17T07:31:04.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689814+0000", 2024-09-17T07:31:04.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046607742000000001 2024-09-17T07:31:04.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689886+0000", 2024-09-17T07:31:04.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2020999999999994e-05 2024-09-17T07:31:04.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689928+0000", 2024-09-17T07:31:04.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2771000000000001e-05 2024-09-17T07:31:04.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.169 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.169 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44215 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14479360~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.169 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:31:04.169 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.43360248499999, 2024-09-17T07:31:04.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046868543999999998, 2024-09-17T07:31:04.169 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.169 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.169 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.170 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44215 2024-09-17T07:31:04.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.170 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:31:04.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:31:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643237+0000", 2024-09-17T07:31:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:31:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643245+0000", 2024-09-17T07:31:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3270000000000001e-06 2024-09-17T07:31:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643254+0000", 2024-09-17T07:31:04.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6049999999999998e-06 2024-09-17T07:31:04.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643260+0000", 2024-09-17T07:31:04.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7849999999999997e-06 2024-09-17T07:31:04.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689961+0000", 2024-09-17T07:31:04.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046701196 2024-09-17T07:31:04.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690036+0000", 2024-09-17T07:31:04.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5153000000000004e-05 2024-09-17T07:31:04.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690105+0000", 2024-09-17T07:31:04.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8874000000000005e-05 2024-09-17T07:31:04.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.173 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.173 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44216 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14495744~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.174 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:31:04.174 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.433570772, 2024-09-17T07:31:04.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047003047999999999, 2024-09-17T07:31:04.174 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.174 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.174 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.175 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44216 2024-09-17T07:31:04.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.175 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:31:04.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:31:04.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643269+0000", 2024-09-17T07:31:04.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:31:04.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643271+0000", 2024-09-17T07:31:04.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-09-17T07:31:04.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643278+0000", 2024-09-17T07:31:04.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8850000000000001e-06 2024-09-17T07:31:04.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643282+0000", 2024-09-17T07:31:04.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5979999999999999e-06 2024-09-17T07:31:04.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690133+0000", 2024-09-17T07:31:04.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850470999999998 2024-09-17T07:31:04.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690205+0000", 2024-09-17T07:31:04.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2442999999999997e-05 2024-09-17T07:31:04.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690271+0000", 2024-09-17T07:31:04.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5874e-05 2024-09-17T07:31:04.179 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.179 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.179 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.179 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44217 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14512128~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.179 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:31:04.179 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.43354976000001, 2024-09-17T07:31:04.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047166485000000001, 2024-09-17T07:31:04.179 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.179 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.179 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.179 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.180 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.180 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.180 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44217 2024-09-17T07:31:04.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.180 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:31:04.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:31:04.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:31:04.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-17T07:31:04.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643291+0000", 2024-09-17T07:31:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.962e-06 2024-09-17T07:31:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643297+0000", 2024-09-17T07:31:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0700000000000003e-06 2024-09-17T07:31:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643302+0000", 2024-09-17T07:31:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:31:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690308+0000", 2024-09-17T07:31:04.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047005354999999999 2024-09-17T07:31:04.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690385+0000", 2024-09-17T07:31:04.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7541999999999997e-05 2024-09-17T07:31:04.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690455+0000", 2024-09-17T07:31:04.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0328999999999997e-05 2024-09-17T07:31:04.183 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.183 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.184 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.184 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44218 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14528512~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.184 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:31:04.184 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.43353008299999, 2024-09-17T07:31:04.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047321015000000001, 2024-09-17T07:31:04.184 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.184 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.184 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.184 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.184 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.184 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.185 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44218 2024-09-17T07:31:04.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.185 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:31:04.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:31:04.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:31:04.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-17T07:31:04.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643311+0000", 2024-09-17T07:31:04.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.933e-06 2024-09-17T07:31:04.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643318+0000", 2024-09-17T07:31:04.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3980000000000004e-06 2024-09-17T07:31:04.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643322+0000", 2024-09-17T07:31:04.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7609999999999998e-06 2024-09-17T07:31:04.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690484+0000", 2024-09-17T07:31:04.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047161428999999998 2024-09-17T07:31:04.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690564+0000", 2024-09-17T07:31:04.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9845000000000005e-05 2024-09-17T07:31:04.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690630+0000", 2024-09-17T07:31:04.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6021999999999999e-05 2024-09-17T07:31:04.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.189 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.189 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44219 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14544896~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.189 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:31:04.189 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.433509716, 2024-09-17T07:31:04.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048128708999999999, 2024-09-17T07:31:04.190 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.190 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.190 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.190 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44219 2024-09-17T07:31:04.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.190 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:31:04.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:31:04.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:31:04.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:31:04.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643337+0000", 2024-09-17T07:31:04.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.109e-06 2024-09-17T07:31:04.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643350+0000", 2024-09-17T07:31:04.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3046e-05 2024-09-17T07:31:04.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643356+0000", 2024-09-17T07:31:04.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8930000000000002e-06 2024-09-17T07:31:04.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690660+0000", 2024-09-17T07:31:04.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047304061000000001 2024-09-17T07:31:04.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691372+0000", 2024-09-17T07:31:04.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00071281299999999997 2024-09-17T07:31:04.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691458+0000", 2024-09-17T07:31:04.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5352000000000005e-05 2024-09-17T07:31:04.194 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.194 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.194 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.194 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44220 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14561280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:04.194 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:31:04.194 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.433475668, 2024-09-17T07:31:04.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048309849000000002, 2024-09-17T07:31:04.194 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.194 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.194 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.195 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.195 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.195 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44220 2024-09-17T07:31:04.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.195 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:31:04.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:31:04.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646869+0000", 2024-09-17T07:31:04.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003506267 2024-09-17T07:31:04.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646874+0000", 2024-09-17T07:31:04.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7500000000000003e-06 2024-09-17T07:31:04.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646887+0000", 2024-09-17T07:31:04.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3115999999999999e-05 2024-09-17T07:31:04.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646894+0000", 2024-09-17T07:31:04.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5289999999999997e-06 2024-09-17T07:31:04.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691547+0000", 2024-09-17T07:31:04.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044652743000000002 2024-09-17T07:31:04.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691622+0000", 2024-09-17T07:31:04.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5037999999999995e-05 2024-09-17T07:31:04.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691673+0000", 2024-09-17T07:31:04.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-17T07:31:04.200 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.200 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.200 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.200 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.201 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.201 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:04.202 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-17T07:31:04.203 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T07:31:04.204 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:31:04.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T07:31:04.207 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:04.202+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:04.303 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:04.302+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:04.349 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:04.349 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:31:04.349 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:31:04.350 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:04.359 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:31:04.404 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:04.403+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:04.503 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:04.503 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:31:04.503 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:31:04.503 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:31:04.503 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:04.505 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:31:04.504+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:31:04.508 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:31:04.511 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:31:04.606 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:04.605+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:04.632 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:31:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:31:04.650 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:31:04.650 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:31:04.656 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:04.657 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:31:04.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:31:04.657 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:31:04.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.657 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19985 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5275648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.657 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:31:04.657 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.40311107700001, 2024-09-17T07:31:04.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071819998999999995, 2024-09-17T07:31:04.657 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.657 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.657 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.657 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.658 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.658 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19985 2024-09-17T07:31:04.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.658 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:31:04.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:31:04.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:31:04.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:31:04.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252493+0000", 2024-09-17T07:31:04.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6300000000000004e-07 2024-09-17T07:31:04.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252496+0000", 2024-09-17T07:31:04.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-17T07:31:04.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252499+0000", 2024-09-17T07:31:04.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:31:04.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324036+0000", 2024-09-17T07:31:04.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071536587999999998 2024-09-17T07:31:04.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324118+0000", 2024-09-17T07:31:04.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2336999999999995e-05 2024-09-17T07:31:04.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324312+0000", 2024-09-17T07:31:04.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000193709 2024-09-17T07:31:04.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.661 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.662 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.662 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19986 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5505024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.662 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:31:04.662 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.40309992500001, 2024-09-17T07:31:04.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072083761999999996, 2024-09-17T07:31:04.662 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.662 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.662 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.663 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19986 2024-09-17T07:31:04.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.663 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:31:04.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:31:04.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:31:04.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:31:04.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252508+0000", 2024-09-17T07:31:04.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:31:04.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252511+0000", 2024-09-17T07:31:04.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.145e-06 2024-09-17T07:31:04.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252514+0000", 2024-09-17T07:31:04.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5569999999999998e-06 2024-09-17T07:31:04.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324326+0000", 2024-09-17T07:31:04.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071812681000000003 2024-09-17T07:31:04.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324372+0000", 2024-09-17T07:31:04.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5893000000000003e-05 2024-09-17T07:31:04.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324587+0000", 2024-09-17T07:31:04.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214608 2024-09-17T07:31:04.666 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.666 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.666 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.666 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19987 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5734400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.666 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:31:04.666 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.403086307, 2024-09-17T07:31:04.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072276676999999998, 2024-09-17T07:31:04.667 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.667 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.667 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.667 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19987 2024-09-17T07:31:04.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.667 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:31:04.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:31:04.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:31:04.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:31:04.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252518+0000", 2024-09-17T07:31:04.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-17T07:31:04.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252522+0000", 2024-09-17T07:31:04.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2440000000000004e-06 2024-09-17T07:31:04.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252525+0000", 2024-09-17T07:31:04.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5919999999999999e-06 2024-09-17T07:31:04.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324621+0000", 2024-09-17T07:31:04.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072096442999999996 2024-09-17T07:31:04.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324699+0000", 2024-09-17T07:31:04.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8061999999999999e-05 2024-09-17T07:31:04.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324793+0000", 2024-09-17T07:31:04.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4207000000000001e-05 2024-09-17T07:31:04.671 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.671 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.671 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.671 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19988 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.671 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:31:04.671 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.403075336, 2024-09-17T07:31:04.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072585798000000007, 2024-09-17T07:31:04.671 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.671 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.671 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.672 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.672 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.672 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.672 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19988 2024-09-17T07:31:04.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.672 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:31:04.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:31:04.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:31:04.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:31:04.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252529+0000", 2024-09-17T07:31:04.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:31:04.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252532+0000", 2024-09-17T07:31:04.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9579999999999999e-06 2024-09-17T07:31:04.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252534+0000", 2024-09-17T07:31:04.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5730000000000002e-06 2024-09-17T07:31:04.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324825+0000", 2024-09-17T07:31:04.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072290757999999997 2024-09-17T07:31:04.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324874+0000", 2024-09-17T07:31:04.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9360000000000002e-05 2024-09-17T07:31:04.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325113+0000", 2024-09-17T07:31:04.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000239025 2024-09-17T07:31:04.675 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.675 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.675 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.675 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19989 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.676 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:31:04.676 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.403065715, 2024-09-17T07:31:04.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072963494000000004, 2024-09-17T07:31:04.676 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.676 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.676 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.676 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19989 2024-09-17T07:31:04.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.676 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:31:04.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:31:04.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:31:04.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:31:04.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:31:04.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2900000000000002e-07 2024-09-17T07:31:04.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252542+0000", 2024-09-17T07:31:04.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8600000000000003e-06 2024-09-17T07:31:04.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252545+0000", 2024-09-17T07:31:04.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.605e-06 2024-09-17T07:31:04.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325144+0000", 2024-09-17T07:31:04.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072599460000000005 2024-09-17T07:31:04.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325223+0000", 2024-09-17T07:31:04.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8473999999999994e-05 2024-09-17T07:31:04.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325501+0000", 2024-09-17T07:31:04.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000278063 2024-09-17T07:31:04.680 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.680 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.680 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.680 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19990 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6422528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.680 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:31:04.680 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.403055244, 2024-09-17T07:31:04.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073241127000000003, 2024-09-17T07:31:04.680 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.680 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.681 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.681 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.681 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.681 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.681 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19990 2024-09-17T07:31:04.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.681 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:31:04.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:31:04.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:31:04.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:31:04.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252552+0000", 2024-09-17T07:31:04.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4719999999999997e-06 2024-09-17T07:31:04.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252557+0000", 2024-09-17T07:31:04.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3039999999999998e-06 2024-09-17T07:31:04.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252560+0000", 2024-09-17T07:31:04.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8830000000000002e-06 2024-09-17T07:31:04.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325529+0000", 2024-09-17T07:31:04.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072968880999999999 2024-09-17T07:31:04.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325574+0000", 2024-09-17T07:31:04.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466000000000001e-05 2024-09-17T07:31:04.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325789+0000", 2024-09-17T07:31:04.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021490999999999999 2024-09-17T07:31:04.684 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.684 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.684 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.685 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19991 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6651904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.685 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:31:04.685 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.403035942, 2024-09-17T07:31:04.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073720552999999994, 2024-09-17T07:31:04.685 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.685 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.685 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.685 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.685 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.685 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.685 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19991 2024-09-17T07:31:04.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.686 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:31:04.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:31:04.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:31:04.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4699999999999998e-07 2024-09-17T07:31:04.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252568+0000", 2024-09-17T07:31:04.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5300000000000002e-07 2024-09-17T07:31:04.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252572+0000", 2024-09-17T07:31:04.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9160000000000003e-06 2024-09-17T07:31:04.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252575+0000", 2024-09-17T07:31:04.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:31:04.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325820+0000", 2024-09-17T07:31:04.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073245597999999995 2024-09-17T07:31:04.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325901+0000", 2024-09-17T07:31:04.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0913000000000003e-05 2024-09-17T07:31:04.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326288+0000", 2024-09-17T07:31:04.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038646900000000002 2024-09-17T07:31:04.689 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.689 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.689 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.689 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19992 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6881280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.689 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:31:04.689 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.40302558899999, 2024-09-17T07:31:04.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074054683999999996, 2024-09-17T07:31:04.689 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.690 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.690 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.690 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.690 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.690 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.690 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19992 2024-09-17T07:31:04.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.690 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:31:04.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:31:04.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252578+0000", 2024-09-17T07:31:04.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:31:04.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252579+0000", 2024-09-17T07:31:04.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2500000000000004e-07 2024-09-17T07:31:04.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252582+0000", 2024-09-17T07:31:04.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-17T07:31:04.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252585+0000", 2024-09-17T07:31:04.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.542e-06 2024-09-17T07:31:04.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326315+0000", 2024-09-17T07:31:04.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073729921000000004 2024-09-17T07:31:04.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326359+0000", 2024-09-17T07:31:04.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4095999999999999e-05 2024-09-17T07:31:04.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326632+0000", 2024-09-17T07:31:04.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027314499999999997 2024-09-17T07:31:04.694 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.694 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.694 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.694 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19993 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7110656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.694 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:31:04.694 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.403015285, 2024-09-17T07:31:04.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074425142999999999, 2024-09-17T07:31:04.694 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.694 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.694 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.694 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.694 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.694 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.695 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19993 2024-09-17T07:31:04.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.695 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:31:04.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:31:04.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:31:04.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:31:04.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252589+0000", 2024-09-17T07:31:04.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.33e-07 2024-09-17T07:31:04.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252593+0000", 2024-09-17T07:31:04.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:31:04.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252595+0000", 2024-09-17T07:31:04.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.621e-06 2024-09-17T07:31:04.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326666+0000", 2024-09-17T07:31:04.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074070787999999999 2024-09-17T07:31:04.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326752+0000", 2024-09-17T07:31:04.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5748000000000006e-05 2024-09-17T07:31:04.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327013+0000", 2024-09-17T07:31:04.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026098400000000002 2024-09-17T07:31:04.698 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.698 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.698 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.698 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19994 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7340032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.698 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:31:04.699 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.403004826, 2024-09-17T07:31:04.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074765920999999999, 2024-09-17T07:31:04.699 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.699 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.699 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.699 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19994 2024-09-17T07:31:04.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.699 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:31:04.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:31:04.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:31:04.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:31:04.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252599+0000", 2024-09-17T07:31:04.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2099999999999995e-07 2024-09-17T07:31:04.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252603+0000", 2024-09-17T07:31:04.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:31:04.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252606+0000", 2024-09-17T07:31:04.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-09-17T07:31:04.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327047+0000", 2024-09-17T07:31:04.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074441050999999994 2024-09-17T07:31:04.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327121+0000", 2024-09-17T07:31:04.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4283999999999996e-05 2024-09-17T07:31:04.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327364+0000", 2024-09-17T07:31:04.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000243155 2024-09-17T07:31:04.703 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.703 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.703 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.703 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19995 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7569408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.703 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:31:04.703 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.40299412900001, 2024-09-17T07:31:04.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075172694999999998, 2024-09-17T07:31:04.704 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.704 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.704 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.704 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19995 2024-09-17T07:31:04.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.704 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:31:04.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:31:04.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:31:04.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:31:04.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252610+0000", 2024-09-17T07:31:04.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6799999999999999e-07 2024-09-17T07:31:04.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252614+0000", 2024-09-17T07:31:04.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000002e-06 2024-09-17T07:31:04.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252616+0000", 2024-09-17T07:31:04.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.419e-06 2024-09-17T07:31:04.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327407+0000", 2024-09-17T07:31:04.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074790617000000004 2024-09-17T07:31:04.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327498+0000", 2024-09-17T07:31:04.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0758000000000001e-05 2024-09-17T07:31:04.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327782+0000", 2024-09-17T07:31:04.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028395599999999998 2024-09-17T07:31:04.707 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.707 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.707 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.708 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7798784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.708 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:31:04.708 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.40298381300001, 2024-09-17T07:31:04.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075605832999999997, 2024-09-17T07:31:04.708 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.708 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.708 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.708 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:31:04.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.709 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:31:04.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:31:04.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:31:04.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:31:04.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:31:04.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:31:04.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:31:04.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:31:04.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:31:04.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:31:04.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327844+0000", 2024-09-17T07:31:04.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075217709999999993 2024-09-17T07:31:04.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327927+0000", 2024-09-17T07:31:04.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2628999999999998e-05 2024-09-17T07:31:04.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328225+0000", 2024-09-17T07:31:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029800600000000002 2024-09-17T07:31:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8028160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:31:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.40297346, 2024-09-17T07:31:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076061237000000004, 2024-09-17T07:31:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.712 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.713 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.713 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.713 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.713 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:31:04.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.713 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:31:04.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:31:04.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:31:04.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:31:04.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:31:04.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:31:04.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:31:04.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:31:04.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:31:04.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:31:04.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328268+0000", 2024-09-17T07:31:04.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075626323999999995 2024-09-17T07:31:04.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328352+0000", 2024-09-17T07:31:04.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3578999999999994e-05 2024-09-17T07:31:04.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328691+0000", 2024-09-17T07:31:04.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033915099999999999 2024-09-17T07:31:04.716 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.716 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.716 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.716 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8257536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.717 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:31:04.717 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.402958009, 2024-09-17T07:31:04.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076441903000000005, 2024-09-17T07:31:04.717 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.717 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.717 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.717 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:31:04.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.717 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:31:04.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:31:04.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:31:04.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:31:04.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:31:04.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:31:04.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:31:04.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:31:04.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:31:04.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:31:04.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328747+0000", 2024-09-17T07:31:04.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076094192000000005 2024-09-17T07:31:04.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328824+0000", 2024-09-17T07:31:04.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7096999999999997e-05 2024-09-17T07:31:04.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.329087+0000", 2024-09-17T07:31:04.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026249999999999998 2024-09-17T07:31:04.720 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.720 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.721 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.721 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7356416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.721 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:31:04.721 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.39284185, 2024-09-17T07:31:04.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072172616999999994, 2024-09-17T07:31:04.721 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.721 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.721 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.721 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:31:04.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.722 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:31:04.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:31:04.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:31:04.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:31:04.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:31:04.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:31:04.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:31:04.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:31:04.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:31:04.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:31:04.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334558+0000", 2024-09-17T07:31:04.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071780790999999997 2024-09-17T07:31:04.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334647+0000", 2024-09-17T07:31:04.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9448000000000001e-05 2024-09-17T07:31:04.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334934+0000", 2024-09-17T07:31:04.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028654599999999999 2024-09-17T07:31:04.725 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.725 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.725 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.725 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7585792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.725 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:31:04.725 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.392820138, 2024-09-17T07:31:04.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072606135000000002, 2024-09-17T07:31:04.725 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.725 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:31:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:31:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:31:04.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:31:04.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:31:04.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:31:04.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:31:04.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:31:04.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:31:04.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:31:04.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:31:04.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335116+0000", 2024-09-17T07:31:04.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072323897999999998 2024-09-17T07:31:04.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335166+0000", 2024-09-17T07:31:04.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9882000000000001e-05 2024-09-17T07:31:04.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335389+0000", 2024-09-17T07:31:04.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022334300000000001 2024-09-17T07:31:04.729 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.729 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.729 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.730 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7815168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.730 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:31:04.733 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.39280793899999, 2024-09-17T07:31:04.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073051230999999994, 2024-09-17T07:31:04.733 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.733 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.733 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.733 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.734 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.734 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:31:04.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.734 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:31:04.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:31:04.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:31:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:31:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:31:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:31:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:31:04.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:31:04.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:31:04.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:31:04.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335431+0000", 2024-09-17T07:31:04.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072627852000000007 2024-09-17T07:31:04.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335525+0000", 2024-09-17T07:31:04.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3708000000000005e-05 2024-09-17T07:31:04.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335846+0000", 2024-09-17T07:31:04.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032182000000000001 2024-09-17T07:31:04.737 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.737 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.737 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.737 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.737 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:31:04.737 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.392797081, 2024-09-17T07:31:04.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073284199999999994, 2024-09-17T07:31:04.738 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.738 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.738 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.738 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:31:04.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.738 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:31:04.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:31:04.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:31:04.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:31:04.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:31:04.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:31:04.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:31:04.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:31:04.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:31:04.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:31:04.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335863+0000", 2024-09-17T07:31:04.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073048921000000003 2024-09-17T07:31:04.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335904+0000", 2024-09-17T07:31:04.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1545000000000003e-05 2024-09-17T07:31:04.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336090+0000", 2024-09-17T07:31:04.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185787 2024-09-17T07:31:04.741 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.741 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.741 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.742 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8273920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:04.742 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:31:04.742 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.392782821, 2024-09-17T07:31:04.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073670478999999997, 2024-09-17T07:31:04.742 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.742 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.742 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.742 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.742 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.742 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.742 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:31:04.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.743 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:31:04.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:31:04.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:31:04.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:31:04.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:31:04.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:31:04.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:31:04.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:31:04.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:31:04.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:31:04.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336126+0000", 2024-09-17T07:31:04.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073297254000000006 2024-09-17T07:31:04.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336211+0000", 2024-09-17T07:31:04.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4728000000000006e-05 2024-09-17T07:31:04.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336491+0000", 2024-09-17T07:31:04.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027979399999999998 2024-09-17T07:31:04.746 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.746 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.746 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.746 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:116163 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-17T07:31:04.746 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:31:04.746 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.390447952999999, 2024-09-17T07:31:04.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8549867870000001, 2024-09-17T07:31:04.746 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:04.746 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:04.747 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:04.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:04.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:04.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:04.747 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116163 2024-09-17T07:31:04.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.747 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:04.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:04.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:31:04.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:04.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:31:04.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:04.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:04.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265157+0000", 2024-09-17T07:31:04.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-06 2024-09-17T07:31:04.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:04.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265168+0000", 2024-09-17T07:31:04.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0827e-05 2024-09-17T07:31:04.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:04.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265198+0000", 2024-09-17T07:31:04.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0383e-05 2024-09-17T07:31:04.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:04.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265212+0000", 2024-09-17T07:31:04.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3799e-05 2024-09-17T07:31:04.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265338+0000", 2024-09-17T07:31:04.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012576800000000001 2024-09-17T07:31:04.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:31:04.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265357+0000", 2024-09-17T07:31:04.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9417000000000001e-05 2024-09-17T07:31:04.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:27.083102+0000", 2024-09-17T07:31:04.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.81774456299999998 2024-09-17T07:31:04.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:31:04.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:27.083113+0000", 2024-09-17T07:31:04.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1154e-05 2024-09-17T07:31:04.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:04.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:28.119793+0000", 2024-09-17T07:31:04.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.036680365 2024-09-17T07:31:04.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:04.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:28.119943+0000", 2024-09-17T07:31:04.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014940999999999999 2024-09-17T07:31:04.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:04.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:04.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:04.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:28.120142+0000", 2024-09-17T07:31:04.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00019897099999999999 2024-09-17T07:31:04.751 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.752 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.752 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.752 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:04.752 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:04.752 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:04.753 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:31:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:31:04.754 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:31:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:31:04.755 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:31:04.755 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:04.752+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:04.755 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:31:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:31:04.756 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:31:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:31:04.757 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:31:04.757 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:31:04.757 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:31:04.757 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:31:04.854 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:04.854+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:04.897 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:04.897 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:04.905 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:31:04.956 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:04.955+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:05.008 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:05.009 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:05.017 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:31:05.057 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:05.056+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:05.094 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T07:31:05.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T07:31:05.123 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:05.124 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:05.132 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:31:05.193 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:05.193+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:05.236 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:05.236 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:05.245 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:31:05.258 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:05.257+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:05.348 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:05.348 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:05.356 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:31:05.359 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:31:05.357+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:31:05.459 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:05.459 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:05.460 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:05.459+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:05.467 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:31:05.560 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:05.560+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:05.575 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:05.576 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:05.584 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:31:05.668 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:05.668+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:05.732 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:05.732 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:05.740 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:31:05.782 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:05.781+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:05.862 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:05.862+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:05.888 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:05.888 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:05.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:31:05.965 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:05.965+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:06.004 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:06.004 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:06.012 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:31:06.067 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:06.066+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:06.095 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T07:31:06.113 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T07:31:06.113 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:31:06.124 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:06.124 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:06.134 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:31:06.170 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:06.169+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:06.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T07:31:06.272 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:06.269+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:06.291 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:06.291 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:06.302 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:31:06.373 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:06.372+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:06.424 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:06.424 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:06.433 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:31:06.443 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:06.444 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:31:06.460 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-17T07:31:06.093111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":178891682,"num_objects":988,"num_object_clones":159,"num_object_copies":1976,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":988,"num_whiteouts":725,"num_read":154563,"num_read_kb":1151403,"num_write":21674,"num_write_kb":752996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5551,"num_bytes_recovered":574675212,"num_keys_recovered":811,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18733,"ondisk_log_size":18733,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":450256,"kb_used_data":35400,"kb_used_omap":27,"kb_used_meta":414628,"kb_avail":754524464,"statfs":{"total":773094113280,"available":772633051136,"internally_reserved":0,"allocated":36249600,"data_stored":336124342,"data_compressed":18623920,"data_compressed_allocated":17887232,"data_compressed_original":326042278,"omap_allocated":28217,"internal_metadata":424579527},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-386422233,"num_objects":-188,"num_object_clones":-51,"num_object_copies":-376,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":-1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-188,"num_whiteouts":-78,"num_read":-8747,"num_read_kb":-54457,"num_write":-1089,"num_write_kb":-41265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-533,"num_bytes_recovered":-66273570,"num_keys_recovered":-47,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.005373"},"pg_stats":[{"pgid":"3.1f","version":"151'847","reported_seq":4487,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.093056+0000","last_change":"2024-09-17T07:31:02.099976+0000","last_active":"2024-09-17T07:31:03.093056+0000","last_peered":"2024-09-17T07:31:03.093056+0000","last_clean":"2024-09-17T07:31:03.093056+0000","last_became_active":"2024-09-17T07:30:43.092456+0000","last_became_peered":"2024-09-17T07:30:43.092456+0000","last_unstale":"2024-09-17T07:31:03.093056+0000","last_undegraded":"2024-09-17T07:31:03.093056+0000","last_fullsized":"2024-09-17T07:31:03.093056+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:10.330347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064695639999999997,"stat_sum":{"num_bytes":4706440,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":25,"num_read":5779,"num_read_kb":34295,"num_write":985,"num_write_kb":41977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":26159662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.1e","version":"152'516","reported_seq":3066,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.097669+0000","last_change":"2024-09-17T07:31:02.098202+0000","last_active":"2024-09-17T07:31:02.097669+0000","last_peered":"2024-09-17T07:31:02.097669+0000","last_clean":"2024-09-17T07:31:02.097669+0000","last_became_active":"2024-09-17T07:30:19.087213+0000","last_became_peered":"2024-09-17T07:30:19.087213+0000","last_unstale":"2024-09-17T07:31:02.097669+0000","last_undegraded":"2024-09-17T07:31:02.097669+0000","last_fullsized":"2024-09-17T07:31:02.097669+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:22.624196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003050919,"stat_sum":{"num_bytes":3382661,"num_objects":49,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":40,"num_read":3519,"num_read_kb":30123,"num_write":508,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":15743718,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.1d","version":"153'534","reported_seq":2875,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092911+0000","last_change":"2024-09-17T07:31:04.093261+0000","last_active":"2024-09-17T07:31:04.092911+0000","last_peered":"2024-09-17T07:31:04.092911+0000","last_clean":"2024-09-17T07:31:04.092911+0000","last_became_active":"2024-09-17T07:30:30.096044+0000","last_became_peered":"2024-09-17T07:30:30.096044+0000","last_unstale":"2024-09-17T07:31:04.092911+0000","last_undegraded":"2024-09-17T07:31:04.092911+0000","last_fullsized":"2024-09-17T07:31:04.092911+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:23.270714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085854300000000002,"stat_sum":{"num_bytes":3117514,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":35,"num_read":1948,"num_read_kb":28759,"num_write":577,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":27967854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.1c","version":"153'787","reported_seq":6063,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092278+0000","last_change":"2024-09-17T07:31:04.092825+0000","last_active":"2024-09-17T07:31:04.092278+0000","last_peered":"2024-09-17T07:31:04.092278+0000","last_clean":"2024-09-17T07:31:04.092278+0000","last_became_active":"2024-09-17T07:30:31.098495+0000","last_became_peered":"2024-09-17T07:30:31.098495+0000","last_unstale":"2024-09-17T07:31:04.092278+0000","last_undegraded":"2024-09-17T07:31:04.092278+0000","last_fullsized":"2024-09-17T07:31:04.092278+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":787,"log_dups_size":0,"ondisk_log_size":787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:31:14.450056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078575200000000002,"stat_sum":{"num_bytes":3489822,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":17,"num_read":7466,"num_read_kb":42402,"num_write":1037,"num_write_kb":38721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":26661650,"num_keys_recovered":180,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.1b","version":"154'549","reported_seq":4811,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.176055+0000","last_change":"2024-09-17T07:31:03.107084+0000","last_active":"2024-09-17T07:31:03.176055+0000","last_peered":"2024-09-17T07:31:03.176055+0000","last_clean":"2024-09-17T07:31:03.176055+0000","last_became_active":"2024-09-17T07:31:03.094058+0000","last_became_peered":"2024-09-17T07:31:03.094058+0000","last_unstale":"2024-09-17T07:31:03.176055+0000","last_undegraded":"2024-09-17T07:31:03.176055+0000","last_fullsized":"2024-09-17T07:31:03.176055+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:16:41.269388+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0087697049999999992,"stat_sum":{"num_bytes":2644152,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":19,"num_read":5413,"num_read_kb":52124,"num_write":561,"num_write_kb":22867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":674,"num_bytes_recovered":91784745,"num_keys_recovered":126,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.1a","version":"152'907","reported_seq":14506,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.093543+0000","last_change":"2024-09-17T07:31:04.093859+0000","last_active":"2024-09-17T07:31:04.093543+0000","last_peered":"2024-09-17T07:31:04.093543+0000","last_clean":"2024-09-17T07:31:04.093543+0000","last_became_active":"2024-09-17T07:30:31.090258+0000","last_became_peered":"2024-09-17T07:30:31.090258+0000","last_unstale":"2024-09-17T07:31:04.093543+0000","last_undegraded":"2024-09-17T07:31:04.093543+0000","last_fullsized":"2024-09-17T07:31:04.093543+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:23.949344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000929053,"stat_sum":{"num_bytes":2580871,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":24,"num_read":16687,"num_read_kb":60954,"num_write":1187,"num_write_kb":20597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":19561990,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.19","version":"154'432","reported_seq":5582,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.101759+0000","last_change":"2024-09-17T07:31:04.103080+0000","last_active":"2024-09-17T07:31:04.101759+0000","last_peered":"2024-09-17T07:31:04.101759+0000","last_clean":"2024-09-17T07:31:04.101759+0000","last_became_active":"2024-09-17T07:30:18.083918+0000","last_became_peered":"2024-09-17T07:30:18.083918+0000","last_unstale":"2024-09-17T07:31:04.101759+0000","last_undegraded":"2024-09-17T07:31:04.101759+0000","last_fullsized":"2024-09-17T07:31:04.101759+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:07.424010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016198950000000001,"stat_sum":{"num_bytes":1916928,"num_objects":35,"num_object_clones":2,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":30,"num_read":4352,"num_read_kb":37354,"num_write":397,"num_write_kb":43089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":25536146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.18","version":"153'745","reported_seq":4519,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.134109+0000","last_change":"2024-09-17T07:31:02.097684+0000","last_active":"2024-09-17T07:31:02.134109+0000","last_peered":"2024-09-17T07:31:02.134109+0000","last_clean":"2024-09-17T07:31:02.134109+0000","last_became_active":"2024-09-17T07:30:43.091692+0000","last_became_peered":"2024-09-17T07:30:43.091692+0000","last_unstale":"2024-09-17T07:31:02.134109+0000","last_undegraded":"2024-09-17T07:31:02.134109+0000","last_fullsized":"2024-09-17T07:31:02.134109+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:53.805635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034934319999999999,"stat_sum":{"num_bytes":3145728,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":23,"num_read":5907,"num_read_kb":25272,"num_write":787,"num_write_kb":16603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":29456184,"num_keys_recovered":176,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.17","version":"153'427","reported_seq":4117,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.093084+0000","last_change":"2024-09-17T07:31:02.099536+0000","last_active":"2024-09-17T07:31:03.093084+0000","last_peered":"2024-09-17T07:31:03.093084+0000","last_clean":"2024-09-17T07:31:03.093084+0000","last_became_active":"2024-09-17T07:28:43.080428+0000","last_became_peered":"2024-09-17T07:28:43.080428+0000","last_unstale":"2024-09-17T07:31:03.093084+0000","last_undegraded":"2024-09-17T07:31:03.093084+0000","last_fullsized":"2024-09-17T07:31:03.093084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:13.757594+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.006196422,"stat_sum":{"num_bytes":2056192,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":29,"num_read":4961,"num_read_kb":22717,"num_write":364,"num_write_kb":13531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.16","version":"155'483","reported_seq":3993,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.105547+0000","last_change":"2024-09-17T07:31:04.105719+0000","last_active":"2024-09-17T07:31:04.105547+0000","last_peered":"2024-09-17T07:31:04.105547+0000","last_clean":"2024-09-17T07:31:04.105547+0000","last_became_active":"2024-09-17T07:28:43.082313+0000","last_became_peered":"2024-09-17T07:28:43.082313+0000","last_unstale":"2024-09-17T07:31:04.105547+0000","last_undegraded":"2024-09-17T07:31:04.105547+0000","last_fullsized":"2024-09-17T07:31:04.105547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:06.892513+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.011787407999999999,"stat_sum":{"num_bytes":10092586,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":5970,"num_read_kb":58212,"num_write":428,"num_write_kb":31849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"unknown","last_fresh":"2024-09-17T07:31:04.094787+0000","last_change":"2024-09-17T07:31:04.319298+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:31:04.094787+0000","last_undegraded":"2024-09-17T07:31:04.094787+0000","last_fullsized":"2024-09-17T07:31:04.094787+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"153'448","reported_seq":2013,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092836+0000","last_change":"2024-09-17T07:31:04.093567+0000","last_active":"2024-09-17T07:31:04.092836+0000","last_peered":"2024-09-17T07:31:04.092836+0000","last_clean":"2024-09-17T07:31:04.092836+0000","last_became_active":"2024-09-17T07:30:30.095364+0000","last_became_peered":"2024-09-17T07:30:30.095364+0000","last_unstale":"2024-09-17T07:31:04.092836+0000","last_undegraded":"2024-09-17T07:31:04.092836+0000","last_fullsized":"2024-09-17T07:31:04.092836+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:23.973898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001442175,"stat_sum":{"num_bytes":2726342,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":23,"num_read":1409,"num_read_kb":17972,"num_write":479,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":16518806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":290,"reported_epoch":154,"state":"peering","last_fresh":"2024-09-17T07:31:03.106174+0000","last_change":"2024-09-17T07:31:04.095221+0000","last_active":"2024-09-17T07:31:04.095221+0000","last_peered":"2024-09-17T07:31:03.106174+0000","last_clean":"2024-09-17T07:31:03.106174+0000","last_became_active":"2024-09-17T07:29:12.071738+0000","last_became_peered":"2024-09-17T07:29:12.071738+0000","last_unstale":"2024-09-17T07:31:03.106174+0000","last_undegraded":"2024-09-17T07:31:03.106174+0000","last_fullsized":"2024-09-17T07:31:03.106174+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"154'1001","reported_seq":2635,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.093066+0000","last_change":"2024-09-17T07:31:02.094427+0000","last_active":"2024-09-17T07:31:03.093066+0000","last_peered":"2024-09-17T07:31:03.093066+0000","last_clean":"2024-09-17T07:31:03.093066+0000","last_became_active":"2024-09-17T07:28:43.079229+0000","last_became_peered":"2024-09-17T07:28:43.079229+0000","last_unstale":"2024-09-17T07:31:03.093066+0000","last_undegraded":"2024-09-17T07:31:03.093066+0000","last_fullsized":"2024-09-17T07:31:03.093066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:31:22.953289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016067130000000001,"stat_sum":{"num_bytes":6126684,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":28,"num_read":4625,"num_read_kb":48744,"num_write":1477,"num_write_kb":29297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.101735+0000","last_change":"2024-09-17T07:31:01.268641+0000","last_active":"2024-09-17T07:31:04.101735+0000","last_peered":"2024-09-17T07:31:04.101735+0000","last_clean":"2024-09-17T07:31:04.101735+0000","last_became_active":"2024-09-17T07:31:01.268257+0000","last_became_peered":"2024-09-17T07:31:01.268257+0000","last_unstale":"2024-09-17T07:31:04.101735+0000","last_undegraded":"2024-09-17T07:31:04.101735+0000","last_fullsized":"2024-09-17T07:31:04.101735+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:04.188817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"149'562","reported_seq":3583,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.097552+0000","last_change":"2024-09-17T07:31:02.097885+0000","last_active":"2024-09-17T07:31:02.097552+0000","last_peered":"2024-09-17T07:31:02.097552+0000","last_clean":"2024-09-17T07:31:02.097552+0000","last_became_active":"2024-09-17T07:30:19.087787+0000","last_became_peered":"2024-09-17T07:30:19.087787+0000","last_unstale":"2024-09-17T07:31:02.097552+0000","last_undegraded":"2024-09-17T07:31:02.097552+0000","last_fullsized":"2024-09-17T07:31:02.097552+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:11.335423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003046024,"stat_sum":{"num_bytes":4497670,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":26,"num_read":3991,"num_read_kb":36993,"num_write":611,"num_write_kb":22622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":15169480,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.096042+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:31:04.096042+0000","last_peered":"2024-09-17T07:31:04.096042+0000","last_clean":"2024-09-17T07:31:04.096042+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:31:04.096042+0000","last_undegraded":"2024-09-17T07:31:04.096042+0000","last_fullsized":"2024-09-17T07:31:04.096042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"149'1050","reported_seq":3497,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.102001+0000","last_change":"2024-09-17T07:31:04.102896+0000","last_active":"2024-09-17T07:31:04.102001+0000","last_peered":"2024-09-17T07:31:04.102001+0000","last_clean":"2024-09-17T07:31:04.102001+0000","last_became_active":"2024-09-17T07:28:43.239387+0000","last_became_peered":"2024-09-17T07:28:43.239387+0000","last_unstale":"2024-09-17T07:31:04.102001+0000","last_undegraded":"2024-09-17T07:31:04.102001+0000","last_fullsized":"2024-09-17T07:31:04.102001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1050,"log_dups_size":0,"ondisk_log_size":1050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:43.520894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013651480000000001,"stat_sum":{"num_bytes":2228224,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":23,"num_read":6807,"num_read_kb":16873,"num_write":1443,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.091982+0000","last_change":"2024-09-17T07:30:15.200774+0000","last_active":"2024-09-17T07:31:04.091982+0000","last_peered":"2024-09-17T07:31:04.091982+0000","last_clean":"2024-09-17T07:31:04.091982+0000","last_became_active":"2024-09-17T07:30:15.200470+0000","last_became_peered":"2024-09-17T07:30:15.200470+0000","last_unstale":"2024-09-17T07:31:04.091982+0000","last_undegraded":"2024-09-17T07:31:04.091982+0000","last_fullsized":"2024-09-17T07:31:04.091982+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:58.984439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"155'544","reported_seq":3177,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.102250+0000","last_change":"2024-09-17T07:31:04.102812+0000","last_active":"2024-09-17T07:31:04.102250+0000","last_peered":"2024-09-17T07:31:04.102250+0000","last_clean":"2024-09-17T07:31:04.102250+0000","last_became_active":"2024-09-17T07:30:19.085184+0000","last_became_peered":"2024-09-17T07:30:19.085184+0000","last_unstale":"2024-09-17T07:31:04.102250+0000","last_undegraded":"2024-09-17T07:31:04.102250+0000","last_fullsized":"2024-09-17T07:31:04.102250+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:34.286856+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010218346999999999,"stat_sum":{"num_bytes":12113463,"num_objects":38,"num_object_clones":4,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":26,"num_read":2665,"num_read_kb":46282,"num_write":616,"num_write_kb":27531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":35691417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.3","version":"154'625","reported_seq":2747,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.093040+0000","last_change":"2024-09-17T07:31:04.093545+0000","last_active":"2024-09-17T07:31:04.093040+0000","last_peered":"2024-09-17T07:31:04.093040+0000","last_clean":"2024-09-17T07:31:04.093040+0000","last_became_active":"2024-09-17T07:30:31.091522+0000","last_became_peered":"2024-09-17T07:30:31.091522+0000","last_unstale":"2024-09-17T07:31:04.093040+0000","last_undegraded":"2024-09-17T07:31:04.093040+0000","last_fullsized":"2024-09-17T07:31:04.093040+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:09.847502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010078540000000001,"stat_sum":{"num_bytes":3824433,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":29,"num_read":2196,"num_read_kb":35335,"num_write":599,"num_write_kb":16663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":26727758,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.2","version":"22'2","reported_seq":289,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092020+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:31:04.092020+0000","last_peered":"2024-09-17T07:31:04.092020+0000","last_clean":"2024-09-17T07:31:04.092020+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:31:04.092020+0000","last_undegraded":"2024-09-17T07:31:04.092020+0000","last_fullsized":"2024-09-17T07:31:04.092020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"155'809","reported_seq":11233,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.105080+0000","last_change":"2024-09-17T07:31:04.105429+0000","last_active":"2024-09-17T07:31:04.105080+0000","last_peered":"2024-09-17T07:31:04.105080+0000","last_clean":"2024-09-17T07:31:04.105080+0000","last_became_active":"2024-09-17T07:28:43.239409+0000","last_became_peered":"2024-09-17T07:28:43.239409+0000","last_unstale":"2024-09-17T07:31:04.105080+0000","last_undegraded":"2024-09-17T07:31:04.105080+0000","last_fullsized":"2024-09-17T07:31:04.105080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:29.936462+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0036379070000000001,"stat_sum":{"num_bytes":4454000,"num_objects":46,"num_object_clones":5,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":35,"num_read":12614,"num_read_kb":91705,"num_write":1234,"num_write_kb":58366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.1","version":"0'0","reported_seq":296,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.089503+0000","last_change":"2024-09-17T07:30:15.201999+0000","last_active":"2024-09-17T07:31:02.089503+0000","last_peered":"2024-09-17T07:31:02.089503+0000","last_clean":"2024-09-17T07:31:02.089503+0000","last_became_active":"2024-09-17T07:30:15.201642+0000","last_became_peered":"2024-09-17T07:30:15.201642+0000","last_unstale":"2024-09-17T07:31:02.089503+0000","last_undegraded":"2024-09-17T07:31:02.089503+0000","last_fullsized":"2024-09-17T07:31:02.089503+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:47.871565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"152'414","reported_seq":4563,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.092991+0000","last_change":"2024-09-17T07:31:02.093665+0000","last_active":"2024-09-17T07:31:03.092991+0000","last_peered":"2024-09-17T07:31:03.092991+0000","last_clean":"2024-09-17T07:31:03.092991+0000","last_became_active":"2024-09-17T07:30:18.088918+0000","last_became_peered":"2024-09-17T07:30:18.088918+0000","last_unstale":"2024-09-17T07:31:03.092991+0000","last_undegraded":"2024-09-17T07:31:03.092991+0000","last_fullsized":"2024-09-17T07:31:03.092991+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:25:34.813695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089712099999999996,"stat_sum":{"num_bytes":3406686,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":20,"num_read":3786,"num_read_kb":51681,"num_write":421,"num_write_kb":28012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":36305471,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.0","version":"0'0","reported_seq":298,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.095344+0000","last_change":"2024-09-17T07:31:01.231085+0000","last_active":"2024-09-17T07:31:04.095344+0000","last_peered":"2024-09-17T07:31:04.095344+0000","last_clean":"2024-09-17T07:31:04.095344+0000","last_became_active":"2024-09-17T07:31:01.230794+0000","last_became_peered":"2024-09-17T07:31:01.230794+0000","last_unstale":"2024-09-17T07:31:04.095344+0000","last_undegraded":"2024-09-17T07:31:04.095344+0000","last_fullsized":"2024-09-17T07:31:04.095344+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:25.321726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"151'392","reported_seq":2379,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.096231+0000","last_change":"2024-09-17T07:31:02.096927+0000","last_active":"2024-09-17T07:31:02.096231+0000","last_peered":"2024-09-17T07:31:02.096231+0000","last_clean":"2024-09-17T07:31:02.096231+0000","last_became_active":"2024-09-17T07:30:18.084247+0000","last_became_peered":"2024-09-17T07:30:18.084247+0000","last_unstale":"2024-09-17T07:31:02.096231+0000","last_undegraded":"2024-09-17T07:31:02.096231+0000","last_fullsized":"2024-09-17T07:31:02.096231+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:35.872352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002613818,"stat_sum":{"num_bytes":7770197,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":18,"num_read":2955,"num_read_kb":9260,"num_write":357,"num_write_kb":6317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":6805602,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.3","version":"20'1","reported_seq":73,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.093338+0000","last_change":"2024-09-17T07:30:34.116563+0000","last_active":"2024-09-17T07:31:03.093338+0000","last_peered":"2024-09-17T07:31:03.093338+0000","last_clean":"2024-09-17T07:31:03.093338+0000","last_became_active":"2024-09-17T07:30:34.113598+0000","last_became_peered":"2024-09-17T07:30:34.113598+0000","last_unstale":"2024-09-17T07:31:03.093338+0000","last_undegraded":"2024-09-17T07:31:03.093338+0000","last_fullsized":"2024-09-17T07:31:03.093338+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:52.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":2,"reported_epoch":155,"state":"peering","last_fresh":"2024-09-17T07:31:03.092986+0000","last_change":"2024-09-17T07:31:04.319242+0000","last_active":"2024-09-17T07:31:04.319242+0000","last_peered":"2024-09-17T07:31:03.092986+0000","last_clean":"2024-09-17T07:31:03.092986+0000","last_became_active":"2024-09-17T07:30:41.141587+0000","last_became_peered":"2024-09-17T07:30:41.141587+0000","last_unstale":"2024-09-17T07:31:03.092986+0000","last_undegraded":"2024-09-17T07:31:03.092986+0000","last_fullsized":"2024-09-17T07:31:03.092986+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"151'507","reported_seq":2317,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.094825+0000","last_change":"2024-09-17T07:31:02.096246+0000","last_active":"2024-09-17T07:31:02.094825+0000","last_peered":"2024-09-17T07:31:02.094825+0000","last_clean":"2024-09-17T07:31:02.094825+0000","last_became_active":"2024-09-17T07:30:19.086777+0000","last_became_peered":"2024-09-17T07:30:19.086777+0000","last_unstale":"2024-09-17T07:31:02.094825+0000","last_undegraded":"2024-09-17T07:31:02.094825+0000","last_fullsized":"2024-09-17T07:31:02.094825+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:07.985140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018944280000000001,"stat_sum":{"num_bytes":14159547,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":21,"num_read":1637,"num_read_kb":24381,"num_write":524,"num_write_kb":33950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":24920122,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.9","version":"149'508","reported_seq":3462,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092384+0000","last_change":"2024-09-17T07:31:04.092939+0000","last_active":"2024-09-17T07:31:04.092384+0000","last_peered":"2024-09-17T07:31:04.092384+0000","last_clean":"2024-09-17T07:31:04.092384+0000","last_became_active":"2024-09-17T07:30:30.095518+0000","last_became_peered":"2024-09-17T07:30:30.095518+0000","last_unstale":"2024-09-17T07:31:04.092384+0000","last_undegraded":"2024-09-17T07:31:04.092384+0000","last_fullsized":"2024-09-17T07:31:04.092384+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:49.580571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000891394,"stat_sum":{"num_bytes":12335912,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":22,"num_read":4313,"num_read_kb":20899,"num_write":510,"num_write_kb":12190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":11434290,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.a","version":"155'708","reported_seq":8813,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.101507+0000","last_change":"2024-09-17T07:31:04.096673+0000","last_active":"2024-09-17T07:31:04.101507+0000","last_peered":"2024-09-17T07:31:04.101507+0000","last_clean":"2024-09-17T07:31:04.101507+0000","last_became_active":"2024-09-17T07:30:31.090321+0000","last_became_peered":"2024-09-17T07:30:31.090321+0000","last_unstale":"2024-09-17T07:31:04.101507+0000","last_undegraded":"2024-09-17T07:31:04.101507+0000","last_fullsized":"2024-09-17T07:31:04.101507+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:27:55.890852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001775523,"stat_sum":{"num_bytes":3309568,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":30,"num_read":9267,"num_read_kb":52833,"num_write":673,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":24254406,"num_keys_recovered":93,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.b","version":"149'546","reported_seq":2796,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.094380+0000","last_change":"2024-09-17T07:31:02.094751+0000","last_active":"2024-09-17T07:31:02.094380+0000","last_peered":"2024-09-17T07:31:02.094380+0000","last_clean":"2024-09-17T07:31:02.094380+0000","last_became_active":"2024-09-17T07:30:19.087515+0000","last_became_peered":"2024-09-17T07:30:19.087515+0000","last_unstale":"2024-09-17T07:31:02.094380+0000","last_undegraded":"2024-09-17T07:31:02.094380+0000","last_fullsized":"2024-09-17T07:31:02.094380+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:56.093547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000579428,"stat_sum":{"num_bytes":2266248,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":2408,"num_read_kb":29904,"num_write":664,"num_write_kb":15633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":11178824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.c","version":"149'854","reported_seq":5314,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092356+0000","last_change":"2024-09-17T07:31:04.093797+0000","last_active":"2024-09-17T07:31:04.092356+0000","last_peered":"2024-09-17T07:31:04.092356+0000","last_clean":"2024-09-17T07:31:04.092356+0000","last_became_active":"2024-09-17T07:30:19.085668+0000","last_became_peered":"2024-09-17T07:30:19.085668+0000","last_unstale":"2024-09-17T07:31:04.092356+0000","last_undegraded":"2024-09-17T07:31:04.092356+0000","last_fullsized":"2024-09-17T07:31:04.092356+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:27.109147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010420970000000001,"stat_sum":{"num_bytes":461193,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":20,"num_read":6039,"num_read_kb":27697,"num_write":1098,"num_write_kb":14822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":15619889,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.d","version":"155'714","reported_seq":4508,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.105410+0000","last_change":"2024-09-17T07:31:04.105658+0000","last_active":"2024-09-17T07:31:04.105410+0000","last_peered":"2024-09-17T07:31:04.105410+0000","last_clean":"2024-09-17T07:31:04.105410+0000","last_became_active":"2024-09-17T07:28:43.080816+0000","last_became_peered":"2024-09-17T07:28:43.080816+0000","last_unstale":"2024-09-17T07:31:04.105410+0000","last_undegraded":"2024-09-17T07:31:04.105410+0000","last_fullsized":"2024-09-17T07:31:04.105410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:21.986122+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0096653750000000004,"stat_sum":{"num_bytes":22293790,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":16,"num_read":6005,"num_read_kb":48957,"num_write":829,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.e","version":"155'446","reported_seq":7551,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.105424+0000","last_change":"2024-09-17T07:31:04.105712+0000","last_active":"2024-09-17T07:31:04.105424+0000","last_peered":"2024-09-17T07:31:04.105424+0000","last_clean":"2024-09-17T07:31:04.105424+0000","last_became_active":"2024-09-17T07:29:12.076559+0000","last_became_peered":"2024-09-17T07:29:12.076559+0000","last_unstale":"2024-09-17T07:31:04.105424+0000","last_undegraded":"2024-09-17T07:31:04.105424+0000","last_fullsized":"2024-09-17T07:31:04.105424+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:12.466155+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0095779550000000008,"stat_sum":{"num_bytes":16296498,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":20,"num_read":7081,"num_read_kb":55687,"num_write":388,"num_write_kb":41376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":12168265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"unknown","last_fresh":"2024-09-17T07:31:04.094787+0000","last_change":"2024-09-17T07:31:04.319313+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:31:04.094787+0000","last_undegraded":"2024-09-17T07:31:04.094787+0000","last_fullsized":"2024-09-17T07:31:04.094787+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"155'394","reported_seq":2666,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.102103+0000","last_change":"2024-09-17T07:31:04.102103+0000","last_active":"2024-09-17T07:31:04.102103+0000","last_peered":"2024-09-17T07:31:04.102103+0000","last_clean":"2024-09-17T07:31:04.102103+0000","last_became_active":"2024-09-17T07:28:43.077360+0000","last_became_peered":"2024-09-17T07:28:43.077360+0000","last_unstale":"2024-09-17T07:31:04.102103+0000","last_undegraded":"2024-09-17T07:31:04.102103+0000","last_fullsized":"2024-09-17T07:31:04.102103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:45.567912+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0061405280000000001,"stat_sum":{"num_bytes":3301376,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":19,"num_read":2841,"num_read_kb":26198,"num_write":447,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.11","version":"152'778","reported_seq":3729,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.096234+0000","last_change":"2024-09-17T07:31:04.096234+0000","last_active":"2024-09-17T07:31:04.096234+0000","last_peered":"2024-09-17T07:31:04.096234+0000","last_clean":"2024-09-17T07:31:04.096234+0000","last_became_active":"2024-09-17T07:30:31.091943+0000","last_became_peered":"2024-09-17T07:30:31.091943+0000","last_unstale":"2024-09-17T07:31:04.096234+0000","last_undegraded":"2024-09-17T07:31:04.096234+0000","last_fullsized":"2024-09-17T07:31:04.096234+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":778,"log_dups_size":0,"ondisk_log_size":778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:44.363378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071229300000000002,"stat_sum":{"num_bytes":2783085,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":18,"num_read":5279,"num_read_kb":45190,"num_write":921,"num_write_kb":26416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":31676674,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.12","version":"153'381","reported_seq":2265,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.147378+0000","last_change":"2024-09-17T07:31:02.100630+0000","last_active":"2024-09-17T07:31:03.147378+0000","last_peered":"2024-09-17T07:31:03.147378+0000","last_clean":"2024-09-17T07:31:03.147378+0000","last_became_active":"2024-09-17T07:30:44.090130+0000","last_became_peered":"2024-09-17T07:30:44.090130+0000","last_unstale":"2024-09-17T07:31:03.147378+0000","last_undegraded":"2024-09-17T07:31:03.147378+0000","last_fullsized":"2024-09-17T07:31:03.147378+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:52:06.468412+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0078669629999999994,"stat_sum":{"num_bytes":6851680,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":30,"num_read":2097,"num_read_kb":44445,"num_write":389,"num_write_kb":29142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":21202173,"num_keys_recovered":25,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.13","version":"150'750","reported_seq":3697,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.097904+0000","last_change":"2024-09-17T07:31:04.097904+0000","last_active":"2024-09-17T07:31:04.097904+0000","last_peered":"2024-09-17T07:31:04.097904+0000","last_clean":"2024-09-17T07:31:04.097904+0000","last_became_active":"2024-09-17T07:30:32.085258+0000","last_became_peered":"2024-09-17T07:30:32.085258+0000","last_unstale":"2024-09-17T07:31:04.097904+0000","last_undegraded":"2024-09-17T07:31:04.097904+0000","last_fullsized":"2024-09-17T07:31:04.097904+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:11:25.013233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015260479999999999,"stat_sum":{"num_bytes":9961805,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":19,"num_read":4480,"num_read_kb":28100,"num_write":1060,"num_write_kb":16610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":21539718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":178301295,"num_objects":984,"num_object_clones":159,"num_object_copies":1968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":984,"num_whiteouts":725,"num_read":154497,"num_read_kb":1151348,"num_write":21571,"num_write_kb":751366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5548,"num_bytes_recovered":574084844,"num_keys_recovered":811,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19644416,"data_stored":230081231,"data_compressed":12878306,"data_compressed_allocated":16281600,"data_compressed_original":227329858,"omap_allocated":7902,"internal_metadata":0},"log_size":18658,"ondisk_log_size":18658,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":66815,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542178,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63504,"kb_used_data":5948,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94308336,"statfs":{"total":96636764160,"available":96571736064,"internally_reserved":0,"allocated":6090752,"data_stored":80260548,"data_compressed":4209914,"data_compressed_allocated":3686400,"data_compressed_original":78905344,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29232,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342608,"statfs":{"total":96636764160,"available":96606830592,"internally_reserved":0,"allocated":2252800,"data_stored":5660692,"data_compressed":245339,"data_compressed_allocated":344064,"data_compressed_original":4726784,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542178,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59236,"kb_used_data":5832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94312604,"statfs":{"total":96636764160,"available":96576106496,"internally_reserved":0,"allocated":5971968,"data_stored":75965961,"data_compressed":4175066,"data_compressed_allocated":3559424,"data_compressed_original":74600556,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542178,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58348,"kb_used_data":4940,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313492,"statfs":{"total":96636764160,"available":96577015808,"internally_reserved":0,"allocated":5058560,"data_stored":36699417,"data_compressed":1921789,"data_compressed_allocated":2555904,"data_compressed_original":35250176,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056654,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47176,"kb_used_data":6128,"kb_used_omap":16,"kb_used_meta":41007,"kb_avail":94324664,"statfs":{"total":96636764160,"available":96588455936,"internally_reserved":0,"allocated":6275072,"data_stored":67262833,"data_compressed":3937380,"data_compressed_allocated":3731456,"data_compressed_original":65761850,"omap_allocated":17058,"internal_metadata":41991518},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542178,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51708,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320132,"statfs":{"total":96636764160,"available":96583815168,"internally_reserved":0,"allocated":2523136,"data_stored":7268310,"data_compressed":404211,"data_compressed_allocated":503808,"data_compressed_original":6238208,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79132,"kb_used_data":3528,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292708,"statfs":{"total":96636764160,"available":96555732992,"internally_reserved":0,"allocated":3612672,"data_stored":28890886,"data_compressed":1497046,"data_compressed_allocated":1388544,"data_compressed_original":27725824,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542178,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61920,"kb_used_data":4360,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94309920,"statfs":{"total":96636764160,"available":96573358080,"internally_reserved":0,"allocated":4464640,"data_stored":34115695,"data_compressed":2233175,"data_compressed_allocated":2117632,"data_compressed_original":32833536,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1806336,"data_stored":16105865,"data_compressed":987326,"data_compressed_allocated":1318912,"data_compressed_original":15724544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1622016,"data_stored":18506125,"data_compressed":997863,"data_compressed_allocated":1323008,"data_compressed_original":18288640,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":688128,"data_stored":6386309,"data_compressed":404211,"data_compressed_allocated":503808,"data_compressed_original":6238208,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4022272,"data_stored":52915066,"data_compressed":3116765,"data_compressed_allocated":3416064,"data_compressed_original":52396602,"omap_allocated":7902,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3211264,"data_stored":35811541,"data_compressed":1921789,"data_compressed_allocated":2555904,"data_compressed_original":35250176,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3932160,"data_stored":45930998,"data_compressed":2587233,"data_compressed_allocated":3375104,"data_compressed_original":45487368,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":4150695,"data_compressed":211122,"data_compressed_allocated":299008,"data_compressed_original":4136960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4042752,"data_stored":50274632,"data_compressed":2651997,"data_compressed_allocated":3489792,"data_compressed_original":49807360,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:31:06.463 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:31:06.501 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:06.501+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:06.548 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:06.549 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:06.558 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:31:06.577 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:06.575+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:06.677 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:06.676+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:06.678 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:06.679 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:06.687 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:31:06.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:31:06.778 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:06.777+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:06.803 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:06.803 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:06.813 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:31:06.819 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:06.819 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:31:06.832 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2024-09-17T07:31:06.781336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":178891682,"num_objects":988,"num_object_clones":159,"num_object_copies":1976,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":988,"num_whiteouts":725,"num_read":154563,"num_read_kb":1151403,"num_write":21674,"num_write_kb":752996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5551,"num_bytes_recovered":574675212,"num_keys_recovered":811,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18733,"ondisk_log_size":18733,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":450256,"kb_used_data":35400,"kb_used_omap":27,"kb_used_meta":414628,"kb_avail":754524464,"statfs":{"total":773094113280,"available":772633051136,"internally_reserved":0,"allocated":36249600,"data_stored":336124342,"data_compressed":18623920,"data_compressed_allocated":17887232,"data_compressed_original":326042278,"omap_allocated":28217,"internal_metadata":424579527},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-310724235,"num_objects":-132,"num_object_clones":-30,"num_object_copies":-264,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":-1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-132,"num_whiteouts":-60,"num_read":-9064,"num_read_kb":-54727,"num_write":-1134,"num_write_kb":-41277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-533,"num_bytes_recovered":-66273570,"num_keys_recovered":-47,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000797"},"pg_stats":[{"pgid":"3.1f","version":"151'847","reported_seq":4487,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.093056+0000","last_change":"2024-09-17T07:31:02.099976+0000","last_active":"2024-09-17T07:31:03.093056+0000","last_peered":"2024-09-17T07:31:03.093056+0000","last_clean":"2024-09-17T07:31:03.093056+0000","last_became_active":"2024-09-17T07:30:43.092456+0000","last_became_peered":"2024-09-17T07:30:43.092456+0000","last_unstale":"2024-09-17T07:31:03.093056+0000","last_undegraded":"2024-09-17T07:31:03.093056+0000","last_fullsized":"2024-09-17T07:31:03.093056+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:10.330347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064695639999999997,"stat_sum":{"num_bytes":4706440,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":25,"num_read":5779,"num_read_kb":34295,"num_write":985,"num_write_kb":41977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":26159662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.1e","version":"152'516","reported_seq":3066,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.097669+0000","last_change":"2024-09-17T07:31:02.098202+0000","last_active":"2024-09-17T07:31:02.097669+0000","last_peered":"2024-09-17T07:31:02.097669+0000","last_clean":"2024-09-17T07:31:02.097669+0000","last_became_active":"2024-09-17T07:30:19.087213+0000","last_became_peered":"2024-09-17T07:30:19.087213+0000","last_unstale":"2024-09-17T07:31:02.097669+0000","last_undegraded":"2024-09-17T07:31:02.097669+0000","last_fullsized":"2024-09-17T07:31:02.097669+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:22.624196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003050919,"stat_sum":{"num_bytes":3382661,"num_objects":49,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":40,"num_read":3519,"num_read_kb":30123,"num_write":508,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":15743718,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.1d","version":"153'534","reported_seq":2875,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092911+0000","last_change":"2024-09-17T07:31:04.093261+0000","last_active":"2024-09-17T07:31:04.092911+0000","last_peered":"2024-09-17T07:31:04.092911+0000","last_clean":"2024-09-17T07:31:04.092911+0000","last_became_active":"2024-09-17T07:30:30.096044+0000","last_became_peered":"2024-09-17T07:30:30.096044+0000","last_unstale":"2024-09-17T07:31:04.092911+0000","last_undegraded":"2024-09-17T07:31:04.092911+0000","last_fullsized":"2024-09-17T07:31:04.092911+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:23.270714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085854300000000002,"stat_sum":{"num_bytes":3117514,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":35,"num_read":1948,"num_read_kb":28759,"num_write":577,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":27967854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.1c","version":"153'787","reported_seq":6063,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092278+0000","last_change":"2024-09-17T07:31:04.092825+0000","last_active":"2024-09-17T07:31:04.092278+0000","last_peered":"2024-09-17T07:31:04.092278+0000","last_clean":"2024-09-17T07:31:04.092278+0000","last_became_active":"2024-09-17T07:30:31.098495+0000","last_became_peered":"2024-09-17T07:30:31.098495+0000","last_unstale":"2024-09-17T07:31:04.092278+0000","last_undegraded":"2024-09-17T07:31:04.092278+0000","last_fullsized":"2024-09-17T07:31:04.092278+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":787,"log_dups_size":0,"ondisk_log_size":787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:31:14.450056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078575200000000002,"stat_sum":{"num_bytes":3489822,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":17,"num_read":7466,"num_read_kb":42402,"num_write":1037,"num_write_kb":38721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":26661650,"num_keys_recovered":180,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.1b","version":"154'549","reported_seq":4811,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.176055+0000","last_change":"2024-09-17T07:31:03.107084+0000","last_active":"2024-09-17T07:31:03.176055+0000","last_peered":"2024-09-17T07:31:03.176055+0000","last_clean":"2024-09-17T07:31:03.176055+0000","last_became_active":"2024-09-17T07:31:03.094058+0000","last_became_peered":"2024-09-17T07:31:03.094058+0000","last_unstale":"2024-09-17T07:31:03.176055+0000","last_undegraded":"2024-09-17T07:31:03.176055+0000","last_fullsized":"2024-09-17T07:31:03.176055+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:16:41.269388+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0087697049999999992,"stat_sum":{"num_bytes":2644152,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":19,"num_read":5413,"num_read_kb":52124,"num_write":561,"num_write_kb":22867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":674,"num_bytes_recovered":91784745,"num_keys_recovered":126,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.1a","version":"152'907","reported_seq":14506,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.093543+0000","last_change":"2024-09-17T07:31:04.093859+0000","last_active":"2024-09-17T07:31:04.093543+0000","last_peered":"2024-09-17T07:31:04.093543+0000","last_clean":"2024-09-17T07:31:04.093543+0000","last_became_active":"2024-09-17T07:30:31.090258+0000","last_became_peered":"2024-09-17T07:30:31.090258+0000","last_unstale":"2024-09-17T07:31:04.093543+0000","last_undegraded":"2024-09-17T07:31:04.093543+0000","last_fullsized":"2024-09-17T07:31:04.093543+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:23.949344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000929053,"stat_sum":{"num_bytes":2580871,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":24,"num_read":16687,"num_read_kb":60954,"num_write":1187,"num_write_kb":20597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":19561990,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.19","version":"154'432","reported_seq":5582,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.101759+0000","last_change":"2024-09-17T07:31:04.103080+0000","last_active":"2024-09-17T07:31:04.101759+0000","last_peered":"2024-09-17T07:31:04.101759+0000","last_clean":"2024-09-17T07:31:04.101759+0000","last_became_active":"2024-09-17T07:30:18.083918+0000","last_became_peered":"2024-09-17T07:30:18.083918+0000","last_unstale":"2024-09-17T07:31:04.101759+0000","last_undegraded":"2024-09-17T07:31:04.101759+0000","last_fullsized":"2024-09-17T07:31:04.101759+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:07.424010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016198950000000001,"stat_sum":{"num_bytes":1916928,"num_objects":35,"num_object_clones":2,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":30,"num_read":4352,"num_read_kb":37354,"num_write":397,"num_write_kb":43089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":25536146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.18","version":"153'745","reported_seq":4519,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.134109+0000","last_change":"2024-09-17T07:31:02.097684+0000","last_active":"2024-09-17T07:31:02.134109+0000","last_peered":"2024-09-17T07:31:02.134109+0000","last_clean":"2024-09-17T07:31:02.134109+0000","last_became_active":"2024-09-17T07:30:43.091692+0000","last_became_peered":"2024-09-17T07:30:43.091692+0000","last_unstale":"2024-09-17T07:31:02.134109+0000","last_undegraded":"2024-09-17T07:31:02.134109+0000","last_fullsized":"2024-09-17T07:31:02.134109+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:53.805635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034934319999999999,"stat_sum":{"num_bytes":3145728,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":23,"num_read":5907,"num_read_kb":25272,"num_write":787,"num_write_kb":16603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":29456184,"num_keys_recovered":176,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.17","version":"153'427","reported_seq":4117,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.093084+0000","last_change":"2024-09-17T07:31:02.099536+0000","last_active":"2024-09-17T07:31:03.093084+0000","last_peered":"2024-09-17T07:31:03.093084+0000","last_clean":"2024-09-17T07:31:03.093084+0000","last_became_active":"2024-09-17T07:28:43.080428+0000","last_became_peered":"2024-09-17T07:28:43.080428+0000","last_unstale":"2024-09-17T07:31:03.093084+0000","last_undegraded":"2024-09-17T07:31:03.093084+0000","last_fullsized":"2024-09-17T07:31:03.093084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:13.757594+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.006196422,"stat_sum":{"num_bytes":2056192,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":29,"num_read":4961,"num_read_kb":22717,"num_write":364,"num_write_kb":13531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.16","version":"155'483","reported_seq":3993,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.105547+0000","last_change":"2024-09-17T07:31:04.105719+0000","last_active":"2024-09-17T07:31:04.105547+0000","last_peered":"2024-09-17T07:31:04.105547+0000","last_clean":"2024-09-17T07:31:04.105547+0000","last_became_active":"2024-09-17T07:28:43.082313+0000","last_became_peered":"2024-09-17T07:28:43.082313+0000","last_unstale":"2024-09-17T07:31:04.105547+0000","last_undegraded":"2024-09-17T07:31:04.105547+0000","last_fullsized":"2024-09-17T07:31:04.105547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:06.892513+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.011787407999999999,"stat_sum":{"num_bytes":10092586,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":5970,"num_read_kb":58212,"num_write":428,"num_write_kb":31849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"unknown","last_fresh":"2024-09-17T07:31:04.094787+0000","last_change":"2024-09-17T07:31:04.319298+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:31:04.094787+0000","last_undegraded":"2024-09-17T07:31:04.094787+0000","last_fullsized":"2024-09-17T07:31:04.094787+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"153'448","reported_seq":2013,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092836+0000","last_change":"2024-09-17T07:31:04.093567+0000","last_active":"2024-09-17T07:31:04.092836+0000","last_peered":"2024-09-17T07:31:04.092836+0000","last_clean":"2024-09-17T07:31:04.092836+0000","last_became_active":"2024-09-17T07:30:30.095364+0000","last_became_peered":"2024-09-17T07:30:30.095364+0000","last_unstale":"2024-09-17T07:31:04.092836+0000","last_undegraded":"2024-09-17T07:31:04.092836+0000","last_fullsized":"2024-09-17T07:31:04.092836+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:23.973898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001442175,"stat_sum":{"num_bytes":2726342,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":23,"num_read":1409,"num_read_kb":17972,"num_write":479,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":16518806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":290,"reported_epoch":154,"state":"peering","last_fresh":"2024-09-17T07:31:03.106174+0000","last_change":"2024-09-17T07:31:04.095221+0000","last_active":"2024-09-17T07:31:04.095221+0000","last_peered":"2024-09-17T07:31:03.106174+0000","last_clean":"2024-09-17T07:31:03.106174+0000","last_became_active":"2024-09-17T07:29:12.071738+0000","last_became_peered":"2024-09-17T07:29:12.071738+0000","last_unstale":"2024-09-17T07:31:03.106174+0000","last_undegraded":"2024-09-17T07:31:03.106174+0000","last_fullsized":"2024-09-17T07:31:03.106174+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"154'1001","reported_seq":2635,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.093066+0000","last_change":"2024-09-17T07:31:02.094427+0000","last_active":"2024-09-17T07:31:03.093066+0000","last_peered":"2024-09-17T07:31:03.093066+0000","last_clean":"2024-09-17T07:31:03.093066+0000","last_became_active":"2024-09-17T07:28:43.079229+0000","last_became_peered":"2024-09-17T07:28:43.079229+0000","last_unstale":"2024-09-17T07:31:03.093066+0000","last_undegraded":"2024-09-17T07:31:03.093066+0000","last_fullsized":"2024-09-17T07:31:03.093066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:31:22.953289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016067130000000001,"stat_sum":{"num_bytes":6126684,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":28,"num_read":4625,"num_read_kb":48744,"num_write":1477,"num_write_kb":29297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.101735+0000","last_change":"2024-09-17T07:31:01.268641+0000","last_active":"2024-09-17T07:31:04.101735+0000","last_peered":"2024-09-17T07:31:04.101735+0000","last_clean":"2024-09-17T07:31:04.101735+0000","last_became_active":"2024-09-17T07:31:01.268257+0000","last_became_peered":"2024-09-17T07:31:01.268257+0000","last_unstale":"2024-09-17T07:31:04.101735+0000","last_undegraded":"2024-09-17T07:31:04.101735+0000","last_fullsized":"2024-09-17T07:31:04.101735+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:04.188817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"149'562","reported_seq":3583,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.097552+0000","last_change":"2024-09-17T07:31:02.097885+0000","last_active":"2024-09-17T07:31:02.097552+0000","last_peered":"2024-09-17T07:31:02.097552+0000","last_clean":"2024-09-17T07:31:02.097552+0000","last_became_active":"2024-09-17T07:30:19.087787+0000","last_became_peered":"2024-09-17T07:30:19.087787+0000","last_unstale":"2024-09-17T07:31:02.097552+0000","last_undegraded":"2024-09-17T07:31:02.097552+0000","last_fullsized":"2024-09-17T07:31:02.097552+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:11.335423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003046024,"stat_sum":{"num_bytes":4497670,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":26,"num_read":3991,"num_read_kb":36993,"num_write":611,"num_write_kb":22622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":15169480,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.096042+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:31:04.096042+0000","last_peered":"2024-09-17T07:31:04.096042+0000","last_clean":"2024-09-17T07:31:04.096042+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:31:04.096042+0000","last_undegraded":"2024-09-17T07:31:04.096042+0000","last_fullsized":"2024-09-17T07:31:04.096042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"149'1050","reported_seq":3497,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.102001+0000","last_change":"2024-09-17T07:31:04.102896+0000","last_active":"2024-09-17T07:31:04.102001+0000","last_peered":"2024-09-17T07:31:04.102001+0000","last_clean":"2024-09-17T07:31:04.102001+0000","last_became_active":"2024-09-17T07:28:43.239387+0000","last_became_peered":"2024-09-17T07:28:43.239387+0000","last_unstale":"2024-09-17T07:31:04.102001+0000","last_undegraded":"2024-09-17T07:31:04.102001+0000","last_fullsized":"2024-09-17T07:31:04.102001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1050,"log_dups_size":0,"ondisk_log_size":1050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:43.520894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013651480000000001,"stat_sum":{"num_bytes":2228224,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":23,"num_read":6807,"num_read_kb":16873,"num_write":1443,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.091982+0000","last_change":"2024-09-17T07:30:15.200774+0000","last_active":"2024-09-17T07:31:04.091982+0000","last_peered":"2024-09-17T07:31:04.091982+0000","last_clean":"2024-09-17T07:31:04.091982+0000","last_became_active":"2024-09-17T07:30:15.200470+0000","last_became_peered":"2024-09-17T07:30:15.200470+0000","last_unstale":"2024-09-17T07:31:04.091982+0000","last_undegraded":"2024-09-17T07:31:04.091982+0000","last_fullsized":"2024-09-17T07:31:04.091982+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:58.984439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"155'544","reported_seq":3177,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.102250+0000","last_change":"2024-09-17T07:31:04.102812+0000","last_active":"2024-09-17T07:31:04.102250+0000","last_peered":"2024-09-17T07:31:04.102250+0000","last_clean":"2024-09-17T07:31:04.102250+0000","last_became_active":"2024-09-17T07:30:19.085184+0000","last_became_peered":"2024-09-17T07:30:19.085184+0000","last_unstale":"2024-09-17T07:31:04.102250+0000","last_undegraded":"2024-09-17T07:31:04.102250+0000","last_fullsized":"2024-09-17T07:31:04.102250+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:34.286856+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010218346999999999,"stat_sum":{"num_bytes":12113463,"num_objects":38,"num_object_clones":4,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":26,"num_read":2665,"num_read_kb":46282,"num_write":616,"num_write_kb":27531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":35691417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.3","version":"154'625","reported_seq":2747,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.093040+0000","last_change":"2024-09-17T07:31:04.093545+0000","last_active":"2024-09-17T07:31:04.093040+0000","last_peered":"2024-09-17T07:31:04.093040+0000","last_clean":"2024-09-17T07:31:04.093040+0000","last_became_active":"2024-09-17T07:30:31.091522+0000","last_became_peered":"2024-09-17T07:30:31.091522+0000","last_unstale":"2024-09-17T07:31:04.093040+0000","last_undegraded":"2024-09-17T07:31:04.093040+0000","last_fullsized":"2024-09-17T07:31:04.093040+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:09.847502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010078540000000001,"stat_sum":{"num_bytes":3824433,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":29,"num_read":2196,"num_read_kb":35335,"num_write":599,"num_write_kb":16663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":26727758,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.2","version":"22'2","reported_seq":289,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092020+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:31:04.092020+0000","last_peered":"2024-09-17T07:31:04.092020+0000","last_clean":"2024-09-17T07:31:04.092020+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:31:04.092020+0000","last_undegraded":"2024-09-17T07:31:04.092020+0000","last_fullsized":"2024-09-17T07:31:04.092020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"155'809","reported_seq":11233,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.105080+0000","last_change":"2024-09-17T07:31:04.105429+0000","last_active":"2024-09-17T07:31:04.105080+0000","last_peered":"2024-09-17T07:31:04.105080+0000","last_clean":"2024-09-17T07:31:04.105080+0000","last_became_active":"2024-09-17T07:28:43.239409+0000","last_became_peered":"2024-09-17T07:28:43.239409+0000","last_unstale":"2024-09-17T07:31:04.105080+0000","last_undegraded":"2024-09-17T07:31:04.105080+0000","last_fullsized":"2024-09-17T07:31:04.105080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:29.936462+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0036379070000000001,"stat_sum":{"num_bytes":4454000,"num_objects":46,"num_object_clones":5,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":35,"num_read":12614,"num_read_kb":91705,"num_write":1234,"num_write_kb":58366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.1","version":"0'0","reported_seq":296,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.089503+0000","last_change":"2024-09-17T07:30:15.201999+0000","last_active":"2024-09-17T07:31:02.089503+0000","last_peered":"2024-09-17T07:31:02.089503+0000","last_clean":"2024-09-17T07:31:02.089503+0000","last_became_active":"2024-09-17T07:30:15.201642+0000","last_became_peered":"2024-09-17T07:30:15.201642+0000","last_unstale":"2024-09-17T07:31:02.089503+0000","last_undegraded":"2024-09-17T07:31:02.089503+0000","last_fullsized":"2024-09-17T07:31:02.089503+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:47.871565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"152'414","reported_seq":4563,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.092991+0000","last_change":"2024-09-17T07:31:02.093665+0000","last_active":"2024-09-17T07:31:03.092991+0000","last_peered":"2024-09-17T07:31:03.092991+0000","last_clean":"2024-09-17T07:31:03.092991+0000","last_became_active":"2024-09-17T07:30:18.088918+0000","last_became_peered":"2024-09-17T07:30:18.088918+0000","last_unstale":"2024-09-17T07:31:03.092991+0000","last_undegraded":"2024-09-17T07:31:03.092991+0000","last_fullsized":"2024-09-17T07:31:03.092991+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:25:34.813695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089712099999999996,"stat_sum":{"num_bytes":3406686,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":20,"num_read":3786,"num_read_kb":51681,"num_write":421,"num_write_kb":28012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":36305471,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.0","version":"0'0","reported_seq":298,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.095344+0000","last_change":"2024-09-17T07:31:01.231085+0000","last_active":"2024-09-17T07:31:04.095344+0000","last_peered":"2024-09-17T07:31:04.095344+0000","last_clean":"2024-09-17T07:31:04.095344+0000","last_became_active":"2024-09-17T07:31:01.230794+0000","last_became_peered":"2024-09-17T07:31:01.230794+0000","last_unstale":"2024-09-17T07:31:04.095344+0000","last_undegraded":"2024-09-17T07:31:04.095344+0000","last_fullsized":"2024-09-17T07:31:04.095344+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:25.321726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"151'392","reported_seq":2379,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.096231+0000","last_change":"2024-09-17T07:31:02.096927+0000","last_active":"2024-09-17T07:31:02.096231+0000","last_peered":"2024-09-17T07:31:02.096231+0000","last_clean":"2024-09-17T07:31:02.096231+0000","last_became_active":"2024-09-17T07:30:18.084247+0000","last_became_peered":"2024-09-17T07:30:18.084247+0000","last_unstale":"2024-09-17T07:31:02.096231+0000","last_undegraded":"2024-09-17T07:31:02.096231+0000","last_fullsized":"2024-09-17T07:31:02.096231+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:35.872352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002613818,"stat_sum":{"num_bytes":7770197,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":18,"num_read":2955,"num_read_kb":9260,"num_write":357,"num_write_kb":6317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":6805602,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.3","version":"20'1","reported_seq":73,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.093338+0000","last_change":"2024-09-17T07:30:34.116563+0000","last_active":"2024-09-17T07:31:03.093338+0000","last_peered":"2024-09-17T07:31:03.093338+0000","last_clean":"2024-09-17T07:31:03.093338+0000","last_became_active":"2024-09-17T07:30:34.113598+0000","last_became_peered":"2024-09-17T07:30:34.113598+0000","last_unstale":"2024-09-17T07:31:03.093338+0000","last_undegraded":"2024-09-17T07:31:03.093338+0000","last_fullsized":"2024-09-17T07:31:03.093338+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:52.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":2,"reported_epoch":155,"state":"peering","last_fresh":"2024-09-17T07:31:03.092986+0000","last_change":"2024-09-17T07:31:04.319242+0000","last_active":"2024-09-17T07:31:04.319242+0000","last_peered":"2024-09-17T07:31:03.092986+0000","last_clean":"2024-09-17T07:31:03.092986+0000","last_became_active":"2024-09-17T07:30:41.141587+0000","last_became_peered":"2024-09-17T07:30:41.141587+0000","last_unstale":"2024-09-17T07:31:03.092986+0000","last_undegraded":"2024-09-17T07:31:03.092986+0000","last_fullsized":"2024-09-17T07:31:03.092986+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"151'507","reported_seq":2317,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.094825+0000","last_change":"2024-09-17T07:31:02.096246+0000","last_active":"2024-09-17T07:31:02.094825+0000","last_peered":"2024-09-17T07:31:02.094825+0000","last_clean":"2024-09-17T07:31:02.094825+0000","last_became_active":"2024-09-17T07:30:19.086777+0000","last_became_peered":"2024-09-17T07:30:19.086777+0000","last_unstale":"2024-09-17T07:31:02.094825+0000","last_undegraded":"2024-09-17T07:31:02.094825+0000","last_fullsized":"2024-09-17T07:31:02.094825+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:07.985140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018944280000000001,"stat_sum":{"num_bytes":14159547,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":21,"num_read":1637,"num_read_kb":24381,"num_write":524,"num_write_kb":33950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":24920122,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.9","version":"149'508","reported_seq":3462,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092384+0000","last_change":"2024-09-17T07:31:04.092939+0000","last_active":"2024-09-17T07:31:04.092384+0000","last_peered":"2024-09-17T07:31:04.092384+0000","last_clean":"2024-09-17T07:31:04.092384+0000","last_became_active":"2024-09-17T07:30:30.095518+0000","last_became_peered":"2024-09-17T07:30:30.095518+0000","last_unstale":"2024-09-17T07:31:04.092384+0000","last_undegraded":"2024-09-17T07:31:04.092384+0000","last_fullsized":"2024-09-17T07:31:04.092384+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:49.580571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000891394,"stat_sum":{"num_bytes":12335912,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":22,"num_read":4313,"num_read_kb":20899,"num_write":510,"num_write_kb":12190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":11434290,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.a","version":"155'708","reported_seq":8813,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.101507+0000","last_change":"2024-09-17T07:31:04.096673+0000","last_active":"2024-09-17T07:31:04.101507+0000","last_peered":"2024-09-17T07:31:04.101507+0000","last_clean":"2024-09-17T07:31:04.101507+0000","last_became_active":"2024-09-17T07:30:31.090321+0000","last_became_peered":"2024-09-17T07:30:31.090321+0000","last_unstale":"2024-09-17T07:31:04.101507+0000","last_undegraded":"2024-09-17T07:31:04.101507+0000","last_fullsized":"2024-09-17T07:31:04.101507+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:27:55.890852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001775523,"stat_sum":{"num_bytes":3309568,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":30,"num_read":9267,"num_read_kb":52833,"num_write":673,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":24254406,"num_keys_recovered":93,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.b","version":"149'546","reported_seq":2796,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.094380+0000","last_change":"2024-09-17T07:31:02.094751+0000","last_active":"2024-09-17T07:31:02.094380+0000","last_peered":"2024-09-17T07:31:02.094380+0000","last_clean":"2024-09-17T07:31:02.094380+0000","last_became_active":"2024-09-17T07:30:19.087515+0000","last_became_peered":"2024-09-17T07:30:19.087515+0000","last_unstale":"2024-09-17T07:31:02.094380+0000","last_undegraded":"2024-09-17T07:31:02.094380+0000","last_fullsized":"2024-09-17T07:31:02.094380+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:56.093547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000579428,"stat_sum":{"num_bytes":2266248,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":2408,"num_read_kb":29904,"num_write":664,"num_write_kb":15633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":11178824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.c","version":"149'854","reported_seq":5314,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092356+0000","last_change":"2024-09-17T07:31:04.093797+0000","last_active":"2024-09-17T07:31:04.092356+0000","last_peered":"2024-09-17T07:31:04.092356+0000","last_clean":"2024-09-17T07:31:04.092356+0000","last_became_active":"2024-09-17T07:30:19.085668+0000","last_became_peered":"2024-09-17T07:30:19.085668+0000","last_unstale":"2024-09-17T07:31:04.092356+0000","last_undegraded":"2024-09-17T07:31:04.092356+0000","last_fullsized":"2024-09-17T07:31:04.092356+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:27.109147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010420970000000001,"stat_sum":{"num_bytes":461193,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":20,"num_read":6039,"num_read_kb":27697,"num_write":1098,"num_write_kb":14822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":15619889,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.d","version":"155'714","reported_seq":4508,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.105410+0000","last_change":"2024-09-17T07:31:04.105658+0000","last_active":"2024-09-17T07:31:04.105410+0000","last_peered":"2024-09-17T07:31:04.105410+0000","last_clean":"2024-09-17T07:31:04.105410+0000","last_became_active":"2024-09-17T07:28:43.080816+0000","last_became_peered":"2024-09-17T07:28:43.080816+0000","last_unstale":"2024-09-17T07:31:04.105410+0000","last_undegraded":"2024-09-17T07:31:04.105410+0000","last_fullsized":"2024-09-17T07:31:04.105410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:21.986122+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0096653750000000004,"stat_sum":{"num_bytes":22293790,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":16,"num_read":6005,"num_read_kb":48957,"num_write":829,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.e","version":"155'446","reported_seq":7551,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.105424+0000","last_change":"2024-09-17T07:31:04.105712+0000","last_active":"2024-09-17T07:31:04.105424+0000","last_peered":"2024-09-17T07:31:04.105424+0000","last_clean":"2024-09-17T07:31:04.105424+0000","last_became_active":"2024-09-17T07:29:12.076559+0000","last_became_peered":"2024-09-17T07:29:12.076559+0000","last_unstale":"2024-09-17T07:31:04.105424+0000","last_undegraded":"2024-09-17T07:31:04.105424+0000","last_fullsized":"2024-09-17T07:31:04.105424+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:12.466155+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0095779550000000008,"stat_sum":{"num_bytes":16296498,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":20,"num_read":7081,"num_read_kb":55687,"num_write":388,"num_write_kb":41376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":12168265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"unknown","last_fresh":"2024-09-17T07:31:04.094787+0000","last_change":"2024-09-17T07:31:04.319313+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:31:04.094787+0000","last_undegraded":"2024-09-17T07:31:04.094787+0000","last_fullsized":"2024-09-17T07:31:04.094787+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"155'394","reported_seq":2666,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.102103+0000","last_change":"2024-09-17T07:31:04.102103+0000","last_active":"2024-09-17T07:31:04.102103+0000","last_peered":"2024-09-17T07:31:04.102103+0000","last_clean":"2024-09-17T07:31:04.102103+0000","last_became_active":"2024-09-17T07:28:43.077360+0000","last_became_peered":"2024-09-17T07:28:43.077360+0000","last_unstale":"2024-09-17T07:31:04.102103+0000","last_undegraded":"2024-09-17T07:31:04.102103+0000","last_fullsized":"2024-09-17T07:31:04.102103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:45.567912+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0061405280000000001,"stat_sum":{"num_bytes":3301376,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":19,"num_read":2841,"num_read_kb":26198,"num_write":447,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.11","version":"152'778","reported_seq":3729,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.096234+0000","last_change":"2024-09-17T07:31:04.096234+0000","last_active":"2024-09-17T07:31:04.096234+0000","last_peered":"2024-09-17T07:31:04.096234+0000","last_clean":"2024-09-17T07:31:04.096234+0000","last_became_active":"2024-09-17T07:30:31.091943+0000","last_became_peered":"2024-09-17T07:30:31.091943+0000","last_unstale":"2024-09-17T07:31:04.096234+0000","last_undegraded":"2024-09-17T07:31:04.096234+0000","last_fullsized":"2024-09-17T07:31:04.096234+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":778,"log_dups_size":0,"ondisk_log_size":778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:44.363378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071229300000000002,"stat_sum":{"num_bytes":2783085,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":18,"num_read":5279,"num_read_kb":45190,"num_write":921,"num_write_kb":26416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":31676674,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.12","version":"153'381","reported_seq":2265,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.147378+0000","last_change":"2024-09-17T07:31:02.100630+0000","last_active":"2024-09-17T07:31:03.147378+0000","last_peered":"2024-09-17T07:31:03.147378+0000","last_clean":"2024-09-17T07:31:03.147378+0000","last_became_active":"2024-09-17T07:30:44.090130+0000","last_became_peered":"2024-09-17T07:30:44.090130+0000","last_unstale":"2024-09-17T07:31:03.147378+0000","last_undegraded":"2024-09-17T07:31:03.147378+0000","last_fullsized":"2024-09-17T07:31:03.147378+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:52:06.468412+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0078669629999999994,"stat_sum":{"num_bytes":6851680,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":30,"num_read":2097,"num_read_kb":44445,"num_write":389,"num_write_kb":29142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":21202173,"num_keys_recovered":25,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.13","version":"150'750","reported_seq":3697,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.097904+0000","last_change":"2024-09-17T07:31:04.097904+0000","last_active":"2024-09-17T07:31:04.097904+0000","last_peered":"2024-09-17T07:31:04.097904+0000","last_clean":"2024-09-17T07:31:04.097904+0000","last_became_active":"2024-09-17T07:30:32.085258+0000","last_became_peered":"2024-09-17T07:30:32.085258+0000","last_unstale":"2024-09-17T07:31:04.097904+0000","last_undegraded":"2024-09-17T07:31:04.097904+0000","last_fullsized":"2024-09-17T07:31:04.097904+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:11:25.013233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015260479999999999,"stat_sum":{"num_bytes":9961805,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":19,"num_read":4480,"num_read_kb":28100,"num_write":1060,"num_write_kb":16610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":21539718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":178301295,"num_objects":984,"num_object_clones":159,"num_object_copies":1968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":984,"num_whiteouts":725,"num_read":154497,"num_read_kb":1151348,"num_write":21571,"num_write_kb":751366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5548,"num_bytes_recovered":574084844,"num_keys_recovered":811,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19644416,"data_stored":230081231,"data_compressed":12878306,"data_compressed_allocated":16281600,"data_compressed_original":227329858,"omap_allocated":7902,"internal_metadata":0},"log_size":18658,"ondisk_log_size":18658,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":66815,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542178,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63504,"kb_used_data":5948,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94308336,"statfs":{"total":96636764160,"available":96571736064,"internally_reserved":0,"allocated":6090752,"data_stored":80260548,"data_compressed":4209914,"data_compressed_allocated":3686400,"data_compressed_original":78905344,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29232,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342608,"statfs":{"total":96636764160,"available":96606830592,"internally_reserved":0,"allocated":2252800,"data_stored":5660692,"data_compressed":245339,"data_compressed_allocated":344064,"data_compressed_original":4726784,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542178,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59236,"kb_used_data":5832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94312604,"statfs":{"total":96636764160,"available":96576106496,"internally_reserved":0,"allocated":5971968,"data_stored":75965961,"data_compressed":4175066,"data_compressed_allocated":3559424,"data_compressed_original":74600556,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542178,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58348,"kb_used_data":4940,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313492,"statfs":{"total":96636764160,"available":96577015808,"internally_reserved":0,"allocated":5058560,"data_stored":36699417,"data_compressed":1921789,"data_compressed_allocated":2555904,"data_compressed_original":35250176,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056654,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47176,"kb_used_data":6128,"kb_used_omap":16,"kb_used_meta":41007,"kb_avail":94324664,"statfs":{"total":96636764160,"available":96588455936,"internally_reserved":0,"allocated":6275072,"data_stored":67262833,"data_compressed":3937380,"data_compressed_allocated":3731456,"data_compressed_original":65761850,"omap_allocated":17058,"internal_metadata":41991518},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542178,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51708,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320132,"statfs":{"total":96636764160,"available":96583815168,"internally_reserved":0,"allocated":2523136,"data_stored":7268310,"data_compressed":404211,"data_compressed_allocated":503808,"data_compressed_original":6238208,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79132,"kb_used_data":3528,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292708,"statfs":{"total":96636764160,"available":96555732992,"internally_reserved":0,"allocated":3612672,"data_stored":28890886,"data_compressed":1497046,"data_compressed_allocated":1388544,"data_compressed_original":27725824,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542178,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61920,"kb_used_data":4360,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94309920,"statfs":{"total":96636764160,"available":96573358080,"internally_reserved":0,"allocated":4464640,"data_stored":34115695,"data_compressed":2233175,"data_compressed_allocated":2117632,"data_compressed_original":32833536,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1806336,"data_stored":16105865,"data_compressed":987326,"data_compressed_allocated":1318912,"data_compressed_original":15724544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1622016,"data_stored":18506125,"data_compressed":997863,"data_compressed_allocated":1323008,"data_compressed_original":18288640,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":688128,"data_stored":6386309,"data_compressed":404211,"data_compressed_allocated":503808,"data_compressed_original":6238208,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4022272,"data_stored":52915066,"data_compressed":3116765,"data_compressed_allocated":3416064,"data_compressed_original":52396602,"omap_allocated":7902,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3211264,"data_stored":35811541,"data_compressed":1921789,"data_compressed_allocated":2555904,"data_compressed_original":35250176,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3932160,"data_stored":45930998,"data_compressed":2587233,"data_compressed_allocated":3375104,"data_compressed_original":45487368,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":4150695,"data_compressed":211122,"data_compressed_allocated":299008,"data_compressed_original":4136960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4042752,"data_stored":50274632,"data_compressed":2651997,"data_compressed_allocated":3489792,"data_compressed_original":49807360,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:31:06.835 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T07:31:06.879 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:06.877+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:06.901 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:31:06.901 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:31:06.935 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:06.935 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:06.945 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:31:06.980 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:06.979+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:07.081 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:07.079+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:07.088 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is set 2024-09-17T07:31:07.104 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:07.104 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:07.114 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:31:07.122 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:31:07.122 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:31:07.140 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:31:07.140 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:31:07.156 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:31:07.156 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:31:07.172 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:31:07.173 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:31:07.182 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:07.181+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:07.190 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:31:07.191 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:31:07.207 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:31:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:31:07.220 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:07.221 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:31:07.221 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:31:07.221 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:07.224 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:31:07.224 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:31:07.229 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:31:07.270 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:07.270 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"b283e6d5-0f4a-4271-a443-15e695e35364","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub flag(s) set","count":7},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":174,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":158,"num_osds":8,"num_up_osds":8,"osd_up_since":1726558198,"num_in_osds":8,"osd_in_since":1726558226,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":37},{"state_name":"peering","count":2},{"state_name":"unknown","count":2}],"num_pgs":41,"num_pools":3,"num_objects":988,"data_bytes":178891682,"bytes_used":461062144,"bytes_avail":772633051136,"bytes_total":773094113280,"unknown_pgs_ratio":0.048780485987663269,"inactive_pgs_ratio":0.048780485987663269},"fsmap":{"epoch":1,"btime":"2024-09-17T07:28:12:268963+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-17T07:30:58.779932+0000","services":{}},"progress_events":{}} 2024-09-17T07:31:07.284 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T07:31:07.284 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:31:07.313 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:31:07.312+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:31:07.373 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:07.373 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:31:07.373 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:31:07.373 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:31:07.373 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:07.381 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:31:07.384 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:31:07.383+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:31:07.485 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:07.484+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:07.526 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:31:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:31:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:31:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:30525 3.1b 3:def7c8f0:::rbd_data.118381ffbbab.00000000000000b0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T07:31:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:31:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 114.002367815, 2024-09-17T07:31:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55515850300000003, 2024-09-17T07:31:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 30525 2024-09-17T07:31:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:31:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522299+0000", 2024-09-17T07:31:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522300+0000", 2024-09-17T07:31:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0189999999999999e-06 2024-09-17T07:31:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522305+0000", 2024-09-17T07:31:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3639999999999998e-06 2024-09-17T07:31:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522318+0000", 2024-09-17T07:31:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3030999999999999e-05 2024-09-17T07:31:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522326+0000", 2024-09-17T07:31:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4200000000000001e-06 2024-09-17T07:31:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522385+0000", 2024-09-17T07:31:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9005000000000002e-05 2024-09-17T07:31:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:31:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:13.522393+0000", 2024-09-17T07:31:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5480000000000002e-06 2024-09-17T07:31:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075347+0000", 2024-09-17T07:31:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.55295417800000002 2024-09-17T07:31:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075502+0000", 2024-09-17T07:31:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154272 2024-09-17T07:31:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 7", 2024-09-17T07:31:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.075630+0000", 2024-09-17T07:31:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012816300000000001 2024-09-17T07:31:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:31:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077221+0000", 2024-09-17T07:31:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015912929999999999 2024-09-17T07:31:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077353+0000", 2024-09-17T07:31:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013215699999999999 2024-09-17T07:31:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:31:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077413+0000", 2024-09-17T07:31:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0034999999999997e-05 2024-09-17T07:31:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:14.077457+0000", 2024-09-17T07:31:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4017999999999998e-05 2024-09-17T07:31:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44181 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13922304~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:31:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.887189167, 2024-09-17T07:31:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046010502000000002, 2024-09-17T07:31:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44181 2024-09-17T07:31:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:31:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637477+0000", 2024-09-17T07:31:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637478+0000", 2024-09-17T07:31:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2899999999999999e-07 2024-09-17T07:31:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637480+0000", 2024-09-17T07:31:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.843e-06 2024-09-17T07:31:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637487+0000", 2024-09-17T07:31:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6159999999999996e-06 2024-09-17T07:31:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637492+0000", 2024-09-17T07:31:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5360000000000003e-06 2024-09-17T07:31:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683351+0000", 2024-09-17T07:31:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045858758999999999 2024-09-17T07:31:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683422+0000", 2024-09-17T07:31:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1705999999999994e-05 2024-09-17T07:31:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683488+0000", 2024-09-17T07:31:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5612999999999997e-05 2024-09-17T07:31:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44182 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13938688~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:31:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.887168496, 2024-09-17T07:31:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046153118999999999, 2024-09-17T07:31:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44182 2024-09-17T07:31:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:31:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:31:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637498+0000", 2024-09-17T07:31:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9099999999999999e-07 2024-09-17T07:31:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637506+0000", 2024-09-17T07:31:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1520000000000003e-06 2024-09-17T07:31:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637512+0000", 2024-09-17T07:31:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2870000000000001e-06 2024-09-17T07:31:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637517+0000", 2024-09-17T07:31:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7790000000000003e-06 2024-09-17T07:31:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683514+0000", 2024-09-17T07:31:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045997616999999998 2024-09-17T07:31:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683584+0000", 2024-09-17T07:31:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9942000000000002e-05 2024-09-17T07:31:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683651+0000", 2024-09-17T07:31:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6951000000000003e-05 2024-09-17T07:31:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44183 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 13955072~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:31:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.887143264, 2024-09-17T07:31:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046339793999999997, 2024-09-17T07:31:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44183 2024-09-17T07:31:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:31:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.637523+0000", 2024-09-17T07:31:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639583+0000", 2024-09-17T07:31:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002059412 2024-09-17T07:31:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639597+0000", 2024-09-17T07:31:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3832000000000001e-05 2024-09-17T07:31:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639607+0000", 2024-09-17T07:31:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0485e-05 2024-09-17T07:31:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.639614+0000", 2024-09-17T07:31:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0849999999999999e-06 2024-09-17T07:31:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683705+0000", 2024-09-17T07:31:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044091261 2024-09-17T07:31:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683792+0000", 2024-09-17T07:31:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6197000000000002e-05 2024-09-17T07:31:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.683863+0000", 2024-09-17T07:31:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1521999999999997e-05 2024-09-17T07:31:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44199 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14217216~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:31:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.883625712, 2024-09-17T07:31:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045995623999999999, 2024-09-17T07:31:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44199 2024-09-17T07:31:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:31:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:31:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641041+0000", 2024-09-17T07:31:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3500000000000002e-07 2024-09-17T07:31:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641043+0000", 2024-09-17T07:31:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.624e-06 2024-09-17T07:31:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641048+0000", 2024-09-17T07:31:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4029999999999998e-06 2024-09-17T07:31:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641053+0000", 2024-09-17T07:31:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6670000000000004e-06 2024-09-17T07:31:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686918+0000", 2024-09-17T07:31:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045865131000000003 2024-09-17T07:31:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.686986+0000", 2024-09-17T07:31:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8299000000000005e-05 2024-09-17T07:31:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687036+0000", 2024-09-17T07:31:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0164999999999998e-05 2024-09-17T07:31:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44200 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14233600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:31:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.883607883, 2024-09-17T07:31:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046183984999999997, 2024-09-17T07:31:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44200 2024-09-17T07:31:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:31:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:31:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641059+0000", 2024-09-17T07:31:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:31:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641061+0000", 2024-09-17T07:31:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.767e-06 2024-09-17T07:31:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641067+0000", 2024-09-17T07:31:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7379999999999999e-06 2024-09-17T07:31:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641071+0000", 2024-09-17T07:31:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4089999999999996e-06 2024-09-17T07:31:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687078+0000", 2024-09-17T07:31:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046007095999999997 2024-09-17T07:31:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687158+0000", 2024-09-17T07:31:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0374e-05 2024-09-17T07:31:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687243+0000", 2024-09-17T07:31:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4221999999999994e-05 2024-09-17T07:31:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44201 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14249984~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:31:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.88359023699999, 2024-09-17T07:31:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046303182999999998, 2024-09-17T07:31:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.557 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44201 2024-09-17T07:31:07.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.557 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:31:07.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641076+0000", 2024-09-17T07:31:07.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641120+0000", 2024-09-17T07:31:07.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3933000000000001e-05 2024-09-17T07:31:07.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641123+0000", 2024-09-17T07:31:07.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.191e-06 2024-09-17T07:31:07.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641133+0000", 2024-09-17T07:31:07.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4020000000000008e-06 2024-09-17T07:31:07.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641138+0000", 2024-09-17T07:31:07.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6559999999999997e-06 2024-09-17T07:31:07.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687279+0000", 2024-09-17T07:31:07.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046140278999999999 2024-09-17T07:31:07.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687338+0000", 2024-09-17T07:31:07.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9268999999999998e-05 2024-09-17T07:31:07.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687379+0000", 2024-09-17T07:31:07.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1452999999999998e-05 2024-09-17T07:31:07.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.560 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.561 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44202 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14266368~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.561 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:31:07.561 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.883519471, 2024-09-17T07:31:07.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046484299, 2024-09-17T07:31:07.561 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.561 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.561 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.561 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44202 2024-09-17T07:31:07.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.561 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:31:07.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.641147+0000", 2024-09-17T07:31:07.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642810+0000", 2024-09-17T07:31:07.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016627980000000001 2024-09-17T07:31:07.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642815+0000", 2024-09-17T07:31:07.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.091e-06 2024-09-17T07:31:07.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642829+0000", 2024-09-17T07:31:07.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4047e-05 2024-09-17T07:31:07.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.642844+0000", 2024-09-17T07:31:07.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4515000000000001e-05 2024-09-17T07:31:07.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687426+0000", 2024-09-17T07:31:07.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044582201000000002 2024-09-17T07:31:07.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687529+0000", 2024-09-17T07:31:07.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000103078 2024-09-17T07:31:07.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.687631+0000", 2024-09-17T07:31:07.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000102569 2024-09-17T07:31:07.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.565 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.565 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44209 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14381056~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.565 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:31:07.565 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.881621319, 2024-09-17T07:31:07.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045944444000000001, 2024-09-17T07:31:07.565 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.565 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.566 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.566 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44209 2024-09-17T07:31:07.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.566 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:31:07.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643045+0000", 2024-09-17T07:31:07.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643046+0000", 2024-09-17T07:31:07.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8400000000000005e-07 2024-09-17T07:31:07.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643048+0000", 2024-09-17T07:31:07.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.827e-06 2024-09-17T07:31:07.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643054+0000", 2024-09-17T07:31:07.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.032e-06 2024-09-17T07:31:07.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643065+0000", 2024-09-17T07:31:07.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1861999999999999e-05 2024-09-17T07:31:07.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688806+0000", 2024-09-17T07:31:07.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045740617999999997 2024-09-17T07:31:07.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688927+0000", 2024-09-17T07:31:07.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000121327 2024-09-17T07:31:07.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.688990+0000", 2024-09-17T07:31:07.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2293999999999997e-05 2024-09-17T07:31:07.569 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.569 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.569 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.570 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44210 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14397440~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.570 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:31:07.570 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.881594393, 2024-09-17T07:31:07.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046104635999999997, 2024-09-17T07:31:07.570 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.570 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.570 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.570 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.570 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44210 2024-09-17T07:31:07.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.571 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:31:07.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643072+0000", 2024-09-17T07:31:07.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643073+0000", 2024-09-17T07:31:07.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.77e-07 2024-09-17T07:31:07.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643075+0000", 2024-09-17T07:31:07.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0820000000000001e-06 2024-09-17T07:31:07.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643082+0000", 2024-09-17T07:31:07.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7789999999999996e-06 2024-09-17T07:31:07.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643087+0000", 2024-09-17T07:31:07.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.967e-06 2024-09-17T07:31:07.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689022+0000", 2024-09-17T07:31:07.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.045934349999999999 2024-09-17T07:31:07.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689112+0000", 2024-09-17T07:31:07.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0039999999999999e-05 2024-09-17T07:31:07.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689177+0000", 2024-09-17T07:31:07.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5041000000000003e-05 2024-09-17T07:31:07.574 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.574 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.574 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.574 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44211 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14413824~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.574 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:31:07.574 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.88157326, 2024-09-17T07:31:07.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046309003000000001, 2024-09-17T07:31:07.574 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.574 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.575 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.575 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44211 2024-09-17T07:31:07.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.575 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:31:07.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643093+0000", 2024-09-17T07:31:07.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643094+0000", 2024-09-17T07:31:07.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4200000000000002e-07 2024-09-17T07:31:07.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643095+0000", 2024-09-17T07:31:07.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8109999999999999e-06 2024-09-17T07:31:07.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643101+0000", 2024-09-17T07:31:07.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8730000000000004e-06 2024-09-17T07:31:07.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643106+0000", 2024-09-17T07:31:07.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6859999999999997e-06 2024-09-17T07:31:07.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689220+0000", 2024-09-17T07:31:07.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046113945000000003 2024-09-17T07:31:07.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689309+0000", 2024-09-17T07:31:07.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9560000000000003e-05 2024-09-17T07:31:07.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689402+0000", 2024-09-17T07:31:07.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2786000000000002e-05 2024-09-17T07:31:07.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.578 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.579 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44212 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14430208~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.579 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:31:07.579 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.881528698, 2024-09-17T07:31:07.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046448912000000002, 2024-09-17T07:31:07.579 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.579 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.579 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.579 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44212 2024-09-17T07:31:07.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.580 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:31:07.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:31:07.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643138+0000", 2024-09-17T07:31:07.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2200000000000004e-07 2024-09-17T07:31:07.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643141+0000", 2024-09-17T07:31:07.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5210000000000001e-06 2024-09-17T07:31:07.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643149+0000", 2024-09-17T07:31:07.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6490000000000008e-06 2024-09-17T07:31:07.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643161+0000", 2024-09-17T07:31:07.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2241e-05 2024-09-17T07:31:07.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689423+0000", 2024-09-17T07:31:07.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046262323000000001 2024-09-17T07:31:07.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689505+0000", 2024-09-17T07:31:07.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2103999999999999e-05 2024-09-17T07:31:07.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689587+0000", 2024-09-17T07:31:07.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1452000000000006e-05 2024-09-17T07:31:07.583 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.583 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.583 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.583 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44213 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14446592~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.583 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:31:07.583 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.88149719899999, 2024-09-17T07:31:07.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046601780000000002, 2024-09-17T07:31:07.583 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.584 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.584 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.584 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.584 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44213 2024-09-17T07:31:07.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.584 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:31:07.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643169+0000", 2024-09-17T07:31:07.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643170+0000", 2024-09-17T07:31:07.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1399999999999997e-07 2024-09-17T07:31:07.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643172+0000", 2024-09-17T07:31:07.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1660000000000001e-06 2024-09-17T07:31:07.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643180+0000", 2024-09-17T07:31:07.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0020000000000006e-06 2024-09-17T07:31:07.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643185+0000", 2024-09-17T07:31:07.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2480000000000004e-06 2024-09-17T07:31:07.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689622+0000", 2024-09-17T07:31:07.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046436742000000003 2024-09-17T07:31:07.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689697+0000", 2024-09-17T07:31:07.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5179999999999995e-05 2024-09-17T07:31:07.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689771+0000", 2024-09-17T07:31:07.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3928000000000001e-05 2024-09-17T07:31:07.587 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.587 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.588 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.588 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44214 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14462976~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.588 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:31:07.588 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.88147413900001, 2024-09-17T07:31:07.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046736038000000001, 2024-09-17T07:31:07.588 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.588 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.588 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.588 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44214 2024-09-17T07:31:07.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.589 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:31:07.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643192+0000", 2024-09-17T07:31:07.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643193+0000", 2024-09-17T07:31:07.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:31:07.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643195+0000", 2024-09-17T07:31:07.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0640000000000001e-06 2024-09-17T07:31:07.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643201+0000", 2024-09-17T07:31:07.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2859999999999996e-06 2024-09-17T07:31:07.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643206+0000", 2024-09-17T07:31:07.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7140000000000001e-06 2024-09-17T07:31:07.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689814+0000", 2024-09-17T07:31:07.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046607742000000001 2024-09-17T07:31:07.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689886+0000", 2024-09-17T07:31:07.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2020999999999994e-05 2024-09-17T07:31:07.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689928+0000", 2024-09-17T07:31:07.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2771000000000001e-05 2024-09-17T07:31:07.592 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.592 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.592 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.592 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44215 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14479360~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.592 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:31:07.592 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.88143024199999, 2024-09-17T07:31:07.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046868543999999998, 2024-09-17T07:31:07.593 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.593 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.593 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.593 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44215 2024-09-17T07:31:07.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.593 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:31:07.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643236+0000", 2024-09-17T07:31:07.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643237+0000", 2024-09-17T07:31:07.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0399999999999996e-07 2024-09-17T07:31:07.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643245+0000", 2024-09-17T07:31:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3270000000000001e-06 2024-09-17T07:31:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643254+0000", 2024-09-17T07:31:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6049999999999998e-06 2024-09-17T07:31:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643260+0000", 2024-09-17T07:31:07.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7849999999999997e-06 2024-09-17T07:31:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.689961+0000", 2024-09-17T07:31:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046701196 2024-09-17T07:31:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690036+0000", 2024-09-17T07:31:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5153000000000004e-05 2024-09-17T07:31:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690105+0000", 2024-09-17T07:31:07.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8874000000000005e-05 2024-09-17T07:31:07.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.599 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.599 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44216 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14495744~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.599 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:31:07.599 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.88139852899999, 2024-09-17T07:31:07.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047003047999999999, 2024-09-17T07:31:07.599 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.599 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.599 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.600 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44216 2024-09-17T07:31:07.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.600 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:31:07.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643268+0000", 2024-09-17T07:31:07.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643269+0000", 2024-09-17T07:31:07.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7100000000000002e-07 2024-09-17T07:31:07.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643271+0000", 2024-09-17T07:31:07.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2060000000000001e-06 2024-09-17T07:31:07.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643278+0000", 2024-09-17T07:31:07.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8850000000000001e-06 2024-09-17T07:31:07.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643282+0000", 2024-09-17T07:31:07.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5979999999999999e-06 2024-09-17T07:31:07.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690133+0000", 2024-09-17T07:31:07.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.046850470999999998 2024-09-17T07:31:07.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690205+0000", 2024-09-17T07:31:07.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2442999999999997e-05 2024-09-17T07:31:07.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690271+0000", 2024-09-17T07:31:07.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5874e-05 2024-09-17T07:31:07.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.603 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.603 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44217 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14512128~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.604 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:31:07.604 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.881377517, 2024-09-17T07:31:07.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047166485000000001, 2024-09-17T07:31:07.604 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.604 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.604 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.604 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44217 2024-09-17T07:31:07.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.604 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:31:07.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:31:07.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643289+0000", 2024-09-17T07:31:07.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.27e-07 2024-09-17T07:31:07.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643291+0000", 2024-09-17T07:31:07.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.962e-06 2024-09-17T07:31:07.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643297+0000", 2024-09-17T07:31:07.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0700000000000003e-06 2024-09-17T07:31:07.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643302+0000", 2024-09-17T07:31:07.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:31:07.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690308+0000", 2024-09-17T07:31:07.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047005354999999999 2024-09-17T07:31:07.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690385+0000", 2024-09-17T07:31:07.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7541999999999997e-05 2024-09-17T07:31:07.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690455+0000", 2024-09-17T07:31:07.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0328999999999997e-05 2024-09-17T07:31:07.608 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.608 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.608 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.608 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44218 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14528512~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.608 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:31:07.608 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.88135784000001, 2024-09-17T07:31:07.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047321015000000001, 2024-09-17T07:31:07.608 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.608 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.608 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.609 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44218 2024-09-17T07:31:07.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.609 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:31:07.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:31:07.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643309+0000", 2024-09-17T07:31:07.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2699999999999999e-07 2024-09-17T07:31:07.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643311+0000", 2024-09-17T07:31:07.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.933e-06 2024-09-17T07:31:07.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643318+0000", 2024-09-17T07:31:07.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3980000000000004e-06 2024-09-17T07:31:07.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643322+0000", 2024-09-17T07:31:07.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7609999999999998e-06 2024-09-17T07:31:07.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690484+0000", 2024-09-17T07:31:07.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047161428999999998 2024-09-17T07:31:07.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690564+0000", 2024-09-17T07:31:07.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9845000000000005e-05 2024-09-17T07:31:07.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690630+0000", 2024-09-17T07:31:07.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6021999999999999e-05 2024-09-17T07:31:07.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.612 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.613 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.613 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44219 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14544896~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.613 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:31:07.613 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.881337473, 2024-09-17T07:31:07.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048128708999999999, 2024-09-17T07:31:07.613 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.613 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.613 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.613 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44219 2024-09-17T07:31:07.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.614 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:31:07.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:31:07.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643329+0000", 2024-09-17T07:31:07.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3500000000000002e-07 2024-09-17T07:31:07.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643337+0000", 2024-09-17T07:31:07.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.109e-06 2024-09-17T07:31:07.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643350+0000", 2024-09-17T07:31:07.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3046e-05 2024-09-17T07:31:07.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643356+0000", 2024-09-17T07:31:07.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8930000000000002e-06 2024-09-17T07:31:07.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.690660+0000", 2024-09-17T07:31:07.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047304061000000001 2024-09-17T07:31:07.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691372+0000", 2024-09-17T07:31:07.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00071281299999999997 2024-09-17T07:31:07.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691458+0000", 2024-09-17T07:31:07.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5352000000000005e-05 2024-09-17T07:31:07.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.617 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.617 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:44220 3.1a 3:5e73cfcb:::rbd_data.11835e2dca9d.0000000000000001:15 [read 14561280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:07.617 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:31:07.617 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.881303425, 2024-09-17T07:31:07.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048309849000000002, 2024-09-17T07:31:07.618 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:07.618 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:07.618 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:07.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:07.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:07.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:07.618 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44220 2024-09-17T07:31:07.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.618 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:07.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:07.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:31:07.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:07.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.643363+0000", 2024-09-17T07:31:07.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:07.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:07.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646869+0000", 2024-09-17T07:31:07.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003506267 2024-09-17T07:31:07.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:07.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646874+0000", 2024-09-17T07:31:07.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7500000000000003e-06 2024-09-17T07:31:07.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:07.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646887+0000", 2024-09-17T07:31:07.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3115999999999999e-05 2024-09-17T07:31:07.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:07.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.646894+0000", 2024-09-17T07:31:07.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5289999999999997e-06 2024-09-17T07:31:07.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:07.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691547+0000", 2024-09-17T07:31:07.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.044652743000000002 2024-09-17T07:31:07.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:07.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691622+0000", 2024-09-17T07:31:07.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5037999999999995e-05 2024-09-17T07:31:07.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:07.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:07.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:07.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.691673+0000", 2024-09-17T07:31:07.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-17T07:31:07.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.622 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:07.622 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:07.622 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:07.625 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:31:07.625 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:07.622+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:07.625 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:07.626 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:31:07.633 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-17T07:31:07.088612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":178891682,"num_objects":988,"num_object_clones":159,"num_object_copies":1976,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":988,"num_whiteouts":725,"num_read":154563,"num_read_kb":1151403,"num_write":21674,"num_write_kb":752996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5551,"num_bytes_recovered":574675212,"num_keys_recovered":811,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18733,"ondisk_log_size":18733,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":450256,"kb_used_data":35400,"kb_used_omap":27,"kb_used_meta":414628,"kb_avail":754524464,"statfs":{"total":773094113280,"available":772633051136,"internally_reserved":0,"allocated":36249600,"data_stored":336124342,"data_compressed":18623920,"data_compressed_allocated":17887232,"data_compressed_original":326042278,"omap_allocated":28217,"internal_metadata":424579527},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-310724235,"num_objects":-132,"num_object_clones":-30,"num_object_copies":-264,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":-1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-132,"num_whiteouts":-60,"num_read":-9064,"num_read_kb":-54727,"num_write":-1134,"num_write_kb":-41277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-533,"num_bytes_recovered":-66273570,"num_keys_recovered":-47,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.999439"},"pg_stats":[{"pgid":"3.1f","version":"151'847","reported_seq":4487,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.093056+0000","last_change":"2024-09-17T07:31:02.099976+0000","last_active":"2024-09-17T07:31:03.093056+0000","last_peered":"2024-09-17T07:31:03.093056+0000","last_clean":"2024-09-17T07:31:03.093056+0000","last_became_active":"2024-09-17T07:30:43.092456+0000","last_became_peered":"2024-09-17T07:30:43.092456+0000","last_unstale":"2024-09-17T07:31:03.093056+0000","last_undegraded":"2024-09-17T07:31:03.093056+0000","last_fullsized":"2024-09-17T07:31:03.093056+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:10.330347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064695639999999997,"stat_sum":{"num_bytes":4706440,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":25,"num_read":5779,"num_read_kb":34295,"num_write":985,"num_write_kb":41977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":26159662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.1e","version":"152'516","reported_seq":3066,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.097669+0000","last_change":"2024-09-17T07:31:02.098202+0000","last_active":"2024-09-17T07:31:02.097669+0000","last_peered":"2024-09-17T07:31:02.097669+0000","last_clean":"2024-09-17T07:31:02.097669+0000","last_became_active":"2024-09-17T07:30:19.087213+0000","last_became_peered":"2024-09-17T07:30:19.087213+0000","last_unstale":"2024-09-17T07:31:02.097669+0000","last_undegraded":"2024-09-17T07:31:02.097669+0000","last_fullsized":"2024-09-17T07:31:02.097669+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:22.624196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003050919,"stat_sum":{"num_bytes":3382661,"num_objects":49,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":40,"num_read":3519,"num_read_kb":30123,"num_write":508,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":15743718,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.1d","version":"153'534","reported_seq":2875,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092911+0000","last_change":"2024-09-17T07:31:04.093261+0000","last_active":"2024-09-17T07:31:04.092911+0000","last_peered":"2024-09-17T07:31:04.092911+0000","last_clean":"2024-09-17T07:31:04.092911+0000","last_became_active":"2024-09-17T07:30:30.096044+0000","last_became_peered":"2024-09-17T07:30:30.096044+0000","last_unstale":"2024-09-17T07:31:04.092911+0000","last_undegraded":"2024-09-17T07:31:04.092911+0000","last_fullsized":"2024-09-17T07:31:04.092911+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:23.270714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085854300000000002,"stat_sum":{"num_bytes":3117514,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":35,"num_read":1948,"num_read_kb":28759,"num_write":577,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":27967854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.1c","version":"153'787","reported_seq":6063,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092278+0000","last_change":"2024-09-17T07:31:04.092825+0000","last_active":"2024-09-17T07:31:04.092278+0000","last_peered":"2024-09-17T07:31:04.092278+0000","last_clean":"2024-09-17T07:31:04.092278+0000","last_became_active":"2024-09-17T07:30:31.098495+0000","last_became_peered":"2024-09-17T07:30:31.098495+0000","last_unstale":"2024-09-17T07:31:04.092278+0000","last_undegraded":"2024-09-17T07:31:04.092278+0000","last_fullsized":"2024-09-17T07:31:04.092278+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":787,"log_dups_size":0,"ondisk_log_size":787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:31:14.450056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078575200000000002,"stat_sum":{"num_bytes":3489822,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":17,"num_read":7466,"num_read_kb":42402,"num_write":1037,"num_write_kb":38721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":26661650,"num_keys_recovered":180,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.1b","version":"154'549","reported_seq":4811,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.176055+0000","last_change":"2024-09-17T07:31:03.107084+0000","last_active":"2024-09-17T07:31:03.176055+0000","last_peered":"2024-09-17T07:31:03.176055+0000","last_clean":"2024-09-17T07:31:03.176055+0000","last_became_active":"2024-09-17T07:31:03.094058+0000","last_became_peered":"2024-09-17T07:31:03.094058+0000","last_unstale":"2024-09-17T07:31:03.176055+0000","last_undegraded":"2024-09-17T07:31:03.176055+0000","last_fullsized":"2024-09-17T07:31:03.176055+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:16:41.269388+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0087697049999999992,"stat_sum":{"num_bytes":2644152,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":19,"num_read":5413,"num_read_kb":52124,"num_write":561,"num_write_kb":22867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":674,"num_bytes_recovered":91784745,"num_keys_recovered":126,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.1a","version":"152'907","reported_seq":14506,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.093543+0000","last_change":"2024-09-17T07:31:04.093859+0000","last_active":"2024-09-17T07:31:04.093543+0000","last_peered":"2024-09-17T07:31:04.093543+0000","last_clean":"2024-09-17T07:31:04.093543+0000","last_became_active":"2024-09-17T07:30:31.090258+0000","last_became_peered":"2024-09-17T07:30:31.090258+0000","last_unstale":"2024-09-17T07:31:04.093543+0000","last_undegraded":"2024-09-17T07:31:04.093543+0000","last_fullsized":"2024-09-17T07:31:04.093543+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:23.949344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000929053,"stat_sum":{"num_bytes":2580871,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":24,"num_read":16687,"num_read_kb":60954,"num_write":1187,"num_write_kb":20597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":19561990,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.19","version":"154'432","reported_seq":5582,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.101759+0000","last_change":"2024-09-17T07:31:04.103080+0000","last_active":"2024-09-17T07:31:04.101759+0000","last_peered":"2024-09-17T07:31:04.101759+0000","last_clean":"2024-09-17T07:31:04.101759+0000","last_became_active":"2024-09-17T07:30:18.083918+0000","last_became_peered":"2024-09-17T07:30:18.083918+0000","last_unstale":"2024-09-17T07:31:04.101759+0000","last_undegraded":"2024-09-17T07:31:04.101759+0000","last_fullsized":"2024-09-17T07:31:04.101759+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:07.424010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016198950000000001,"stat_sum":{"num_bytes":1916928,"num_objects":35,"num_object_clones":2,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":30,"num_read":4352,"num_read_kb":37354,"num_write":397,"num_write_kb":43089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":25536146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.18","version":"153'745","reported_seq":4519,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.134109+0000","last_change":"2024-09-17T07:31:02.097684+0000","last_active":"2024-09-17T07:31:02.134109+0000","last_peered":"2024-09-17T07:31:02.134109+0000","last_clean":"2024-09-17T07:31:02.134109+0000","last_became_active":"2024-09-17T07:30:43.091692+0000","last_became_peered":"2024-09-17T07:30:43.091692+0000","last_unstale":"2024-09-17T07:31:02.134109+0000","last_undegraded":"2024-09-17T07:31:02.134109+0000","last_fullsized":"2024-09-17T07:31:02.134109+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:53.805635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034934319999999999,"stat_sum":{"num_bytes":3145728,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":23,"num_read":5907,"num_read_kb":25272,"num_write":787,"num_write_kb":16603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":29456184,"num_keys_recovered":176,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.17","version":"153'427","reported_seq":4117,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.093084+0000","last_change":"2024-09-17T07:31:02.099536+0000","last_active":"2024-09-17T07:31:03.093084+0000","last_peered":"2024-09-17T07:31:03.093084+0000","last_clean":"2024-09-17T07:31:03.093084+0000","last_became_active":"2024-09-17T07:28:43.080428+0000","last_became_peered":"2024-09-17T07:28:43.080428+0000","last_unstale":"2024-09-17T07:31:03.093084+0000","last_undegraded":"2024-09-17T07:31:03.093084+0000","last_fullsized":"2024-09-17T07:31:03.093084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:13.757594+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.006196422,"stat_sum":{"num_bytes":2056192,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":29,"num_read":4961,"num_read_kb":22717,"num_write":364,"num_write_kb":13531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.16","version":"155'483","reported_seq":3993,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.105547+0000","last_change":"2024-09-17T07:31:04.105719+0000","last_active":"2024-09-17T07:31:04.105547+0000","last_peered":"2024-09-17T07:31:04.105547+0000","last_clean":"2024-09-17T07:31:04.105547+0000","last_became_active":"2024-09-17T07:28:43.082313+0000","last_became_peered":"2024-09-17T07:28:43.082313+0000","last_unstale":"2024-09-17T07:31:04.105547+0000","last_undegraded":"2024-09-17T07:31:04.105547+0000","last_fullsized":"2024-09-17T07:31:04.105547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:06.892513+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.011787407999999999,"stat_sum":{"num_bytes":10092586,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":5970,"num_read_kb":58212,"num_write":428,"num_write_kb":31849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"unknown","last_fresh":"2024-09-17T07:31:04.094787+0000","last_change":"2024-09-17T07:31:04.319298+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:31:04.094787+0000","last_undegraded":"2024-09-17T07:31:04.094787+0000","last_fullsized":"2024-09-17T07:31:04.094787+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"153'448","reported_seq":2013,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092836+0000","last_change":"2024-09-17T07:31:04.093567+0000","last_active":"2024-09-17T07:31:04.092836+0000","last_peered":"2024-09-17T07:31:04.092836+0000","last_clean":"2024-09-17T07:31:04.092836+0000","last_became_active":"2024-09-17T07:30:30.095364+0000","last_became_peered":"2024-09-17T07:30:30.095364+0000","last_unstale":"2024-09-17T07:31:04.092836+0000","last_undegraded":"2024-09-17T07:31:04.092836+0000","last_fullsized":"2024-09-17T07:31:04.092836+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:23.973898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001442175,"stat_sum":{"num_bytes":2726342,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":23,"num_read":1409,"num_read_kb":17972,"num_write":479,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":16518806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.7","version":"0'0","reported_seq":290,"reported_epoch":154,"state":"peering","last_fresh":"2024-09-17T07:31:03.106174+0000","last_change":"2024-09-17T07:31:04.095221+0000","last_active":"2024-09-17T07:31:04.095221+0000","last_peered":"2024-09-17T07:31:03.106174+0000","last_clean":"2024-09-17T07:31:03.106174+0000","last_became_active":"2024-09-17T07:29:12.071738+0000","last_became_peered":"2024-09-17T07:29:12.071738+0000","last_unstale":"2024-09-17T07:31:03.106174+0000","last_undegraded":"2024-09-17T07:31:03.106174+0000","last_fullsized":"2024-09-17T07:31:03.106174+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"154'1001","reported_seq":2635,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.093066+0000","last_change":"2024-09-17T07:31:02.094427+0000","last_active":"2024-09-17T07:31:03.093066+0000","last_peered":"2024-09-17T07:31:03.093066+0000","last_clean":"2024-09-17T07:31:03.093066+0000","last_became_active":"2024-09-17T07:28:43.079229+0000","last_became_peered":"2024-09-17T07:28:43.079229+0000","last_unstale":"2024-09-17T07:31:03.093066+0000","last_undegraded":"2024-09-17T07:31:03.093066+0000","last_fullsized":"2024-09-17T07:31:03.093066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:31:22.953289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016067130000000001,"stat_sum":{"num_bytes":6126684,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":28,"num_read":4625,"num_read_kb":48744,"num_write":1477,"num_write_kb":29297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.101735+0000","last_change":"2024-09-17T07:31:01.268641+0000","last_active":"2024-09-17T07:31:04.101735+0000","last_peered":"2024-09-17T07:31:04.101735+0000","last_clean":"2024-09-17T07:31:04.101735+0000","last_became_active":"2024-09-17T07:31:01.268257+0000","last_became_peered":"2024-09-17T07:31:01.268257+0000","last_unstale":"2024-09-17T07:31:04.101735+0000","last_undegraded":"2024-09-17T07:31:04.101735+0000","last_fullsized":"2024-09-17T07:31:04.101735+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:04.188817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"149'562","reported_seq":3583,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.097552+0000","last_change":"2024-09-17T07:31:02.097885+0000","last_active":"2024-09-17T07:31:02.097552+0000","last_peered":"2024-09-17T07:31:02.097552+0000","last_clean":"2024-09-17T07:31:02.097552+0000","last_became_active":"2024-09-17T07:30:19.087787+0000","last_became_peered":"2024-09-17T07:30:19.087787+0000","last_unstale":"2024-09-17T07:31:02.097552+0000","last_undegraded":"2024-09-17T07:31:02.097552+0000","last_fullsized":"2024-09-17T07:31:02.097552+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:11.335423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003046024,"stat_sum":{"num_bytes":4497670,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":26,"num_read":3991,"num_read_kb":36993,"num_write":611,"num_write_kb":22622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":15169480,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.5","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.096042+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:31:04.096042+0000","last_peered":"2024-09-17T07:31:04.096042+0000","last_clean":"2024-09-17T07:31:04.096042+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:31:04.096042+0000","last_undegraded":"2024-09-17T07:31:04.096042+0000","last_fullsized":"2024-09-17T07:31:04.096042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"149'1050","reported_seq":3497,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.102001+0000","last_change":"2024-09-17T07:31:04.102896+0000","last_active":"2024-09-17T07:31:04.102001+0000","last_peered":"2024-09-17T07:31:04.102001+0000","last_clean":"2024-09-17T07:31:04.102001+0000","last_became_active":"2024-09-17T07:28:43.239387+0000","last_became_peered":"2024-09-17T07:28:43.239387+0000","last_unstale":"2024-09-17T07:31:04.102001+0000","last_undegraded":"2024-09-17T07:31:04.102001+0000","last_fullsized":"2024-09-17T07:31:04.102001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1050,"log_dups_size":0,"ondisk_log_size":1050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:43.520894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013651480000000001,"stat_sum":{"num_bytes":2228224,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":23,"num_read":6807,"num_read_kb":16873,"num_write":1443,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.091982+0000","last_change":"2024-09-17T07:30:15.200774+0000","last_active":"2024-09-17T07:31:04.091982+0000","last_peered":"2024-09-17T07:31:04.091982+0000","last_clean":"2024-09-17T07:31:04.091982+0000","last_became_active":"2024-09-17T07:30:15.200470+0000","last_became_peered":"2024-09-17T07:30:15.200470+0000","last_unstale":"2024-09-17T07:31:04.091982+0000","last_undegraded":"2024-09-17T07:31:04.091982+0000","last_fullsized":"2024-09-17T07:31:04.091982+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:58.984439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"155'544","reported_seq":3177,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.102250+0000","last_change":"2024-09-17T07:31:04.102812+0000","last_active":"2024-09-17T07:31:04.102250+0000","last_peered":"2024-09-17T07:31:04.102250+0000","last_clean":"2024-09-17T07:31:04.102250+0000","last_became_active":"2024-09-17T07:30:19.085184+0000","last_became_peered":"2024-09-17T07:30:19.085184+0000","last_unstale":"2024-09-17T07:31:04.102250+0000","last_undegraded":"2024-09-17T07:31:04.102250+0000","last_fullsized":"2024-09-17T07:31:04.102250+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:34.286856+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010218346999999999,"stat_sum":{"num_bytes":12113463,"num_objects":38,"num_object_clones":4,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":26,"num_read":2665,"num_read_kb":46282,"num_write":616,"num_write_kb":27531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":35691417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.3","version":"154'625","reported_seq":2747,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.093040+0000","last_change":"2024-09-17T07:31:04.093545+0000","last_active":"2024-09-17T07:31:04.093040+0000","last_peered":"2024-09-17T07:31:04.093040+0000","last_clean":"2024-09-17T07:31:04.093040+0000","last_became_active":"2024-09-17T07:30:31.091522+0000","last_became_peered":"2024-09-17T07:30:31.091522+0000","last_unstale":"2024-09-17T07:31:04.093040+0000","last_undegraded":"2024-09-17T07:31:04.093040+0000","last_fullsized":"2024-09-17T07:31:04.093040+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:09.847502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010078540000000001,"stat_sum":{"num_bytes":3824433,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":29,"num_read":2196,"num_read_kb":35335,"num_write":599,"num_write_kb":16663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":26727758,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.2","version":"22'2","reported_seq":289,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092020+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:31:04.092020+0000","last_peered":"2024-09-17T07:31:04.092020+0000","last_clean":"2024-09-17T07:31:04.092020+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:31:04.092020+0000","last_undegraded":"2024-09-17T07:31:04.092020+0000","last_fullsized":"2024-09-17T07:31:04.092020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"155'809","reported_seq":11233,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.105080+0000","last_change":"2024-09-17T07:31:04.105429+0000","last_active":"2024-09-17T07:31:04.105080+0000","last_peered":"2024-09-17T07:31:04.105080+0000","last_clean":"2024-09-17T07:31:04.105080+0000","last_became_active":"2024-09-17T07:28:43.239409+0000","last_became_peered":"2024-09-17T07:28:43.239409+0000","last_unstale":"2024-09-17T07:31:04.105080+0000","last_undegraded":"2024-09-17T07:31:04.105080+0000","last_fullsized":"2024-09-17T07:31:04.105080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:06:29.936462+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0036379070000000001,"stat_sum":{"num_bytes":4454000,"num_objects":46,"num_object_clones":5,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":35,"num_read":12614,"num_read_kb":91705,"num_write":1234,"num_write_kb":58366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.1","version":"0'0","reported_seq":296,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.089503+0000","last_change":"2024-09-17T07:30:15.201999+0000","last_active":"2024-09-17T07:31:02.089503+0000","last_peered":"2024-09-17T07:31:02.089503+0000","last_clean":"2024-09-17T07:31:02.089503+0000","last_became_active":"2024-09-17T07:30:15.201642+0000","last_became_peered":"2024-09-17T07:30:15.201642+0000","last_unstale":"2024-09-17T07:31:02.089503+0000","last_undegraded":"2024-09-17T07:31:02.089503+0000","last_fullsized":"2024-09-17T07:31:02.089503+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:47.871565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"152'414","reported_seq":4563,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.092991+0000","last_change":"2024-09-17T07:31:02.093665+0000","last_active":"2024-09-17T07:31:03.092991+0000","last_peered":"2024-09-17T07:31:03.092991+0000","last_clean":"2024-09-17T07:31:03.092991+0000","last_became_active":"2024-09-17T07:30:18.088918+0000","last_became_peered":"2024-09-17T07:30:18.088918+0000","last_unstale":"2024-09-17T07:31:03.092991+0000","last_undegraded":"2024-09-17T07:31:03.092991+0000","last_fullsized":"2024-09-17T07:31:03.092991+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:25:34.813695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089712099999999996,"stat_sum":{"num_bytes":3406686,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":20,"num_read":3786,"num_read_kb":51681,"num_write":421,"num_write_kb":28012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":36305471,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.0","version":"0'0","reported_seq":298,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.095344+0000","last_change":"2024-09-17T07:31:01.231085+0000","last_active":"2024-09-17T07:31:04.095344+0000","last_peered":"2024-09-17T07:31:04.095344+0000","last_clean":"2024-09-17T07:31:04.095344+0000","last_became_active":"2024-09-17T07:31:01.230794+0000","last_became_peered":"2024-09-17T07:31:01.230794+0000","last_unstale":"2024-09-17T07:31:04.095344+0000","last_undegraded":"2024-09-17T07:31:04.095344+0000","last_fullsized":"2024-09-17T07:31:04.095344+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:25.321726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"151'392","reported_seq":2379,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.096231+0000","last_change":"2024-09-17T07:31:02.096927+0000","last_active":"2024-09-17T07:31:02.096231+0000","last_peered":"2024-09-17T07:31:02.096231+0000","last_clean":"2024-09-17T07:31:02.096231+0000","last_became_active":"2024-09-17T07:30:18.084247+0000","last_became_peered":"2024-09-17T07:30:18.084247+0000","last_unstale":"2024-09-17T07:31:02.096231+0000","last_undegraded":"2024-09-17T07:31:02.096231+0000","last_fullsized":"2024-09-17T07:31:02.096231+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:35.872352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002613818,"stat_sum":{"num_bytes":7770197,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":18,"num_read":2955,"num_read_kb":9260,"num_write":357,"num_write_kb":6317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":6805602,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"2.3","version":"20'1","reported_seq":73,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.093338+0000","last_change":"2024-09-17T07:30:34.116563+0000","last_active":"2024-09-17T07:31:03.093338+0000","last_peered":"2024-09-17T07:31:03.093338+0000","last_clean":"2024-09-17T07:31:03.093338+0000","last_became_active":"2024-09-17T07:30:34.113598+0000","last_became_peered":"2024-09-17T07:30:34.113598+0000","last_unstale":"2024-09-17T07:31:03.093338+0000","last_undegraded":"2024-09-17T07:31:03.093338+0000","last_fullsized":"2024-09-17T07:31:03.093338+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:52.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":2,"reported_epoch":155,"state":"peering","last_fresh":"2024-09-17T07:31:03.092986+0000","last_change":"2024-09-17T07:31:04.319242+0000","last_active":"2024-09-17T07:31:04.319242+0000","last_peered":"2024-09-17T07:31:03.092986+0000","last_clean":"2024-09-17T07:31:03.092986+0000","last_became_active":"2024-09-17T07:30:41.141587+0000","last_became_peered":"2024-09-17T07:30:41.141587+0000","last_unstale":"2024-09-17T07:31:03.092986+0000","last_undegraded":"2024-09-17T07:31:03.092986+0000","last_fullsized":"2024-09-17T07:31:03.092986+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"151'507","reported_seq":2317,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.094825+0000","last_change":"2024-09-17T07:31:02.096246+0000","last_active":"2024-09-17T07:31:02.094825+0000","last_peered":"2024-09-17T07:31:02.094825+0000","last_clean":"2024-09-17T07:31:02.094825+0000","last_became_active":"2024-09-17T07:30:19.086777+0000","last_became_peered":"2024-09-17T07:30:19.086777+0000","last_unstale":"2024-09-17T07:31:02.094825+0000","last_undegraded":"2024-09-17T07:31:02.094825+0000","last_fullsized":"2024-09-17T07:31:02.094825+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:07.985140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018944280000000001,"stat_sum":{"num_bytes":14159547,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":21,"num_read":1637,"num_read_kb":24381,"num_write":524,"num_write_kb":33950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":24920122,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.9","version":"149'508","reported_seq":3462,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092384+0000","last_change":"2024-09-17T07:31:04.092939+0000","last_active":"2024-09-17T07:31:04.092384+0000","last_peered":"2024-09-17T07:31:04.092384+0000","last_clean":"2024-09-17T07:31:04.092384+0000","last_became_active":"2024-09-17T07:30:30.095518+0000","last_became_peered":"2024-09-17T07:30:30.095518+0000","last_unstale":"2024-09-17T07:31:04.092384+0000","last_undegraded":"2024-09-17T07:31:04.092384+0000","last_fullsized":"2024-09-17T07:31:04.092384+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:49.580571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000891394,"stat_sum":{"num_bytes":12335912,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":22,"num_read":4313,"num_read_kb":20899,"num_write":510,"num_write_kb":12190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":11434290,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.a","version":"155'708","reported_seq":8813,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.101507+0000","last_change":"2024-09-17T07:31:04.096673+0000","last_active":"2024-09-17T07:31:04.101507+0000","last_peered":"2024-09-17T07:31:04.101507+0000","last_clean":"2024-09-17T07:31:04.101507+0000","last_became_active":"2024-09-17T07:30:31.090321+0000","last_became_peered":"2024-09-17T07:30:31.090321+0000","last_unstale":"2024-09-17T07:31:04.101507+0000","last_undegraded":"2024-09-17T07:31:04.101507+0000","last_fullsized":"2024-09-17T07:31:04.101507+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:27:55.890852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001775523,"stat_sum":{"num_bytes":3309568,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":30,"num_read":9267,"num_read_kb":52833,"num_write":673,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":24254406,"num_keys_recovered":93,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.b","version":"149'546","reported_seq":2796,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-17T07:31:02.094380+0000","last_change":"2024-09-17T07:31:02.094751+0000","last_active":"2024-09-17T07:31:02.094380+0000","last_peered":"2024-09-17T07:31:02.094380+0000","last_clean":"2024-09-17T07:31:02.094380+0000","last_became_active":"2024-09-17T07:30:19.087515+0000","last_became_peered":"2024-09-17T07:30:19.087515+0000","last_unstale":"2024-09-17T07:31:02.094380+0000","last_undegraded":"2024-09-17T07:31:02.094380+0000","last_fullsized":"2024-09-17T07:31:02.094380+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:56.093547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000579428,"stat_sum":{"num_bytes":2266248,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":2408,"num_read_kb":29904,"num_write":664,"num_write_kb":15633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":11178824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.c","version":"149'854","reported_seq":5314,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.092356+0000","last_change":"2024-09-17T07:31:04.093797+0000","last_active":"2024-09-17T07:31:04.092356+0000","last_peered":"2024-09-17T07:31:04.092356+0000","last_clean":"2024-09-17T07:31:04.092356+0000","last_became_active":"2024-09-17T07:30:19.085668+0000","last_became_peered":"2024-09-17T07:30:19.085668+0000","last_unstale":"2024-09-17T07:31:04.092356+0000","last_undegraded":"2024-09-17T07:31:04.092356+0000","last_fullsized":"2024-09-17T07:31:04.092356+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:27.109147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010420970000000001,"stat_sum":{"num_bytes":461193,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":20,"num_read":6039,"num_read_kb":27697,"num_write":1098,"num_write_kb":14822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":15619889,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.d","version":"155'714","reported_seq":4508,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.105410+0000","last_change":"2024-09-17T07:31:04.105658+0000","last_active":"2024-09-17T07:31:04.105410+0000","last_peered":"2024-09-17T07:31:04.105410+0000","last_clean":"2024-09-17T07:31:04.105410+0000","last_became_active":"2024-09-17T07:28:43.080816+0000","last_became_peered":"2024-09-17T07:28:43.080816+0000","last_unstale":"2024-09-17T07:31:04.105410+0000","last_undegraded":"2024-09-17T07:31:04.105410+0000","last_fullsized":"2024-09-17T07:31:04.105410+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:21.986122+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0096653750000000004,"stat_sum":{"num_bytes":22293790,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":16,"num_read":6005,"num_read_kb":48957,"num_write":829,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.e","version":"155'446","reported_seq":7551,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.105424+0000","last_change":"2024-09-17T07:31:04.105712+0000","last_active":"2024-09-17T07:31:04.105424+0000","last_peered":"2024-09-17T07:31:04.105424+0000","last_clean":"2024-09-17T07:31:04.105424+0000","last_became_active":"2024-09-17T07:29:12.076559+0000","last_became_peered":"2024-09-17T07:29:12.076559+0000","last_unstale":"2024-09-17T07:31:04.105424+0000","last_undegraded":"2024-09-17T07:31:04.105424+0000","last_fullsized":"2024-09-17T07:31:04.105424+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:12.466155+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0095779550000000008,"stat_sum":{"num_bytes":16296498,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":20,"num_read":7081,"num_read_kb":55687,"num_write":388,"num_write_kb":41376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":12168265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"unknown","last_fresh":"2024-09-17T07:31:04.094787+0000","last_change":"2024-09-17T07:31:04.319313+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T07:31:04.094787+0000","last_undegraded":"2024-09-17T07:31:04.094787+0000","last_fullsized":"2024-09-17T07:31:04.094787+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"155'394","reported_seq":2666,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.102103+0000","last_change":"2024-09-17T07:31:04.102103+0000","last_active":"2024-09-17T07:31:04.102103+0000","last_peered":"2024-09-17T07:31:04.102103+0000","last_clean":"2024-09-17T07:31:04.102103+0000","last_became_active":"2024-09-17T07:28:43.077360+0000","last_became_peered":"2024-09-17T07:28:43.077360+0000","last_unstale":"2024-09-17T07:31:04.102103+0000","last_undegraded":"2024-09-17T07:31:04.102103+0000","last_fullsized":"2024-09-17T07:31:04.102103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:45.567912+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0061405280000000001,"stat_sum":{"num_bytes":3301376,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":19,"num_read":2841,"num_read_kb":26198,"num_write":447,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.11","version":"152'778","reported_seq":3729,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.096234+0000","last_change":"2024-09-17T07:31:04.096234+0000","last_active":"2024-09-17T07:31:04.096234+0000","last_peered":"2024-09-17T07:31:04.096234+0000","last_clean":"2024-09-17T07:31:04.096234+0000","last_became_active":"2024-09-17T07:30:31.091943+0000","last_became_peered":"2024-09-17T07:30:31.091943+0000","last_unstale":"2024-09-17T07:31:04.096234+0000","last_undegraded":"2024-09-17T07:31:04.096234+0000","last_fullsized":"2024-09-17T07:31:04.096234+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":778,"log_dups_size":0,"ondisk_log_size":778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:44.363378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071229300000000002,"stat_sum":{"num_bytes":2783085,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":18,"num_read":5279,"num_read_kb":45190,"num_write":921,"num_write_kb":26416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":31676674,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.12","version":"153'381","reported_seq":2265,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T07:31:03.147378+0000","last_change":"2024-09-17T07:31:02.100630+0000","last_active":"2024-09-17T07:31:03.147378+0000","last_peered":"2024-09-17T07:31:03.147378+0000","last_clean":"2024-09-17T07:31:03.147378+0000","last_became_active":"2024-09-17T07:30:44.090130+0000","last_became_peered":"2024-09-17T07:30:44.090130+0000","last_unstale":"2024-09-17T07:31:03.147378+0000","last_undegraded":"2024-09-17T07:31:03.147378+0000","last_fullsized":"2024-09-17T07:31:03.147378+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:52:06.468412+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0078669629999999994,"stat_sum":{"num_bytes":6851680,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":30,"num_read":2097,"num_read_kb":44445,"num_write":389,"num_write_kb":29142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":21202173,"num_keys_recovered":25,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"9"}]},{"pgid":"3.13","version":"150'750","reported_seq":3697,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T07:31:04.097904+0000","last_change":"2024-09-17T07:31:04.097904+0000","last_active":"2024-09-17T07:31:04.097904+0000","last_peered":"2024-09-17T07:31:04.097904+0000","last_clean":"2024-09-17T07:31:04.097904+0000","last_became_active":"2024-09-17T07:30:32.085258+0000","last_became_peered":"2024-09-17T07:30:32.085258+0000","last_unstale":"2024-09-17T07:31:04.097904+0000","last_undegraded":"2024-09-17T07:31:04.097904+0000","last_fullsized":"2024-09-17T07:31:04.097904+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:11:25.013233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015260479999999999,"stat_sum":{"num_bytes":9961805,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":19,"num_read":4480,"num_read_kb":28100,"num_write":1060,"num_write_kb":16610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":21539718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":178301295,"num_objects":984,"num_object_clones":159,"num_object_copies":1968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":984,"num_whiteouts":725,"num_read":154497,"num_read_kb":1151348,"num_write":21571,"num_write_kb":751366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5548,"num_bytes_recovered":574084844,"num_keys_recovered":811,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19644416,"data_stored":230081231,"data_compressed":12878306,"data_compressed_allocated":16281600,"data_compressed_original":227329858,"omap_allocated":7902,"internal_metadata":0},"log_size":18658,"ondisk_log_size":18658,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":66815,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542178,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63504,"kb_used_data":5948,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94308336,"statfs":{"total":96636764160,"available":96571736064,"internally_reserved":0,"allocated":6090752,"data_stored":80260548,"data_compressed":4209914,"data_compressed_allocated":3686400,"data_compressed_original":78905344,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29232,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342608,"statfs":{"total":96636764160,"available":96606830592,"internally_reserved":0,"allocated":2252800,"data_stored":5660692,"data_compressed":245339,"data_compressed_allocated":344064,"data_compressed_original":4726784,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542178,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59236,"kb_used_data":5832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94312604,"statfs":{"total":96636764160,"available":96576106496,"internally_reserved":0,"allocated":5971968,"data_stored":75965961,"data_compressed":4175066,"data_compressed_allocated":3559424,"data_compressed_original":74600556,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542178,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58348,"kb_used_data":4940,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313492,"statfs":{"total":96636764160,"available":96577015808,"internally_reserved":0,"allocated":5058560,"data_stored":36699417,"data_compressed":1921789,"data_compressed_allocated":2555904,"data_compressed_original":35250176,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056654,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47176,"kb_used_data":6128,"kb_used_omap":16,"kb_used_meta":41007,"kb_avail":94324664,"statfs":{"total":96636764160,"available":96588455936,"internally_reserved":0,"allocated":6275072,"data_stored":67262833,"data_compressed":3937380,"data_compressed_allocated":3731456,"data_compressed_original":65761850,"omap_allocated":17058,"internal_metadata":41991518},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542178,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51708,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320132,"statfs":{"total":96636764160,"available":96583815168,"internally_reserved":0,"allocated":2523136,"data_stored":7268310,"data_compressed":404211,"data_compressed_allocated":503808,"data_compressed_original":6238208,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79132,"kb_used_data":3528,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292708,"statfs":{"total":96636764160,"available":96555732992,"internally_reserved":0,"allocated":3612672,"data_stored":28890886,"data_compressed":1497046,"data_compressed_allocated":1388544,"data_compressed_original":27725824,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542178,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61920,"kb_used_data":4360,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94309920,"statfs":{"total":96636764160,"available":96573358080,"internally_reserved":0,"allocated":4464640,"data_stored":34115695,"data_compressed":2233175,"data_compressed_allocated":2117632,"data_compressed_original":32833536,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1806336,"data_stored":16105865,"data_compressed":987326,"data_compressed_allocated":1318912,"data_compressed_original":15724544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1622016,"data_stored":18506125,"data_compressed":997863,"data_compressed_allocated":1323008,"data_compressed_original":18288640,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":688128,"data_stored":6386309,"data_compressed":404211,"data_compressed_allocated":503808,"data_compressed_original":6238208,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4022272,"data_stored":52915066,"data_compressed":3116765,"data_compressed_allocated":3416064,"data_compressed_original":52396602,"omap_allocated":7902,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3211264,"data_stored":35811541,"data_compressed":1921789,"data_compressed_allocated":2555904,"data_compressed_original":35250176,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3932160,"data_stored":45930998,"data_compressed":2587233,"data_compressed_allocated":3375104,"data_compressed_original":45487368,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":4150695,"data_compressed":211122,"data_compressed_allocated":299008,"data_compressed_original":4136960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4042752,"data_stored":50274632,"data_compressed":2651997,"data_compressed_allocated":3489792,"data_compressed_original":49807360,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:31:07.724 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:07.723+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:07.769 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:07.769 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:31:07.769 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:31:07.770 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:07.778 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:31:07.862 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:07.861+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:07.922 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:07.923 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:31:07.923 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:31:07.923 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:31:07.923 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:07.926 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:31:07.924+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:31:07.931 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:31:08.027 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:08.026+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:08.077 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:08.077 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:31:08.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:31:08.078 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:31:08.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.078 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19985 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5275648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.078 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:31:08.078 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.823895648, 2024-09-17T07:31:08.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071819998999999995, 2024-09-17T07:31:08.078 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.078 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.078 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.078 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.078 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.078 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.078 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19985 2024-09-17T07:31:08.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.079 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:31:08.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:31:08.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252492+0000", 2024-09-17T07:31:08.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:31:08.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252493+0000", 2024-09-17T07:31:08.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6300000000000004e-07 2024-09-17T07:31:08.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252496+0000", 2024-09-17T07:31:08.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.05e-06 2024-09-17T07:31:08.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252499+0000", 2024-09-17T07:31:08.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.241e-06 2024-09-17T07:31:08.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324036+0000", 2024-09-17T07:31:08.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071536587999999998 2024-09-17T07:31:08.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324118+0000", 2024-09-17T07:31:08.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2336999999999995e-05 2024-09-17T07:31:08.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324312+0000", 2024-09-17T07:31:08.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000193709 2024-09-17T07:31:08.082 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.082 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.082 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.082 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19986 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5505024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.082 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:31:08.082 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.82388449600001, 2024-09-17T07:31:08.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072083761999999996, 2024-09-17T07:31:08.083 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.083 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.083 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.083 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19986 2024-09-17T07:31:08.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.083 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:31:08.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:31:08.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252503+0000", 2024-09-17T07:31:08.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1e-07 2024-09-17T07:31:08.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252508+0000", 2024-09-17T07:31:08.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.668e-06 2024-09-17T07:31:08.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252511+0000", 2024-09-17T07:31:08.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.145e-06 2024-09-17T07:31:08.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252514+0000", 2024-09-17T07:31:08.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5569999999999998e-06 2024-09-17T07:31:08.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324326+0000", 2024-09-17T07:31:08.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071812681000000003 2024-09-17T07:31:08.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324372+0000", 2024-09-17T07:31:08.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5893000000000003e-05 2024-09-17T07:31:08.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324587+0000", 2024-09-17T07:31:08.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214608 2024-09-17T07:31:08.086 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.087 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.087 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.087 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19987 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5734400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.087 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:31:08.087 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.82387087799999, 2024-09-17T07:31:08.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072276676999999998, 2024-09-17T07:31:08.087 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.087 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.087 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.087 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.087 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.087 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.088 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19987 2024-09-17T07:31:08.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.088 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:31:08.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:31:08.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252517+0000", 2024-09-17T07:31:08.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:31:08.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252518+0000", 2024-09-17T07:31:08.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-17T07:31:08.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252522+0000", 2024-09-17T07:31:08.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2440000000000004e-06 2024-09-17T07:31:08.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252525+0000", 2024-09-17T07:31:08.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5919999999999999e-06 2024-09-17T07:31:08.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324621+0000", 2024-09-17T07:31:08.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072096442999999996 2024-09-17T07:31:08.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324699+0000", 2024-09-17T07:31:08.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8061999999999999e-05 2024-09-17T07:31:08.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324793+0000", 2024-09-17T07:31:08.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4207000000000001e-05 2024-09-17T07:31:08.091 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.091 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.091 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.091 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19988 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 5963776~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.092 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:31:08.092 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.823859907, 2024-09-17T07:31:08.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072585798000000007, 2024-09-17T07:31:08.092 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.092 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.092 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.092 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19988 2024-09-17T07:31:08.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.092 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:31:08.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:31:08.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252528+0000", 2024-09-17T07:31:08.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:31:08.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252529+0000", 2024-09-17T07:31:08.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0999999999999997e-07 2024-09-17T07:31:08.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252532+0000", 2024-09-17T07:31:08.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9579999999999999e-06 2024-09-17T07:31:08.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252534+0000", 2024-09-17T07:31:08.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5730000000000002e-06 2024-09-17T07:31:08.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324825+0000", 2024-09-17T07:31:08.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072290757999999997 2024-09-17T07:31:08.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.324874+0000", 2024-09-17T07:31:08.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9360000000000002e-05 2024-09-17T07:31:08.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325113+0000", 2024-09-17T07:31:08.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000239025 2024-09-17T07:31:08.096 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.096 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.096 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.096 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19989 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.096 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:31:08.096 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.823850286, 2024-09-17T07:31:08.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072963494000000004, 2024-09-17T07:31:08.096 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.096 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.096 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.097 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.097 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19989 2024-09-17T07:31:08.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.097 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:31:08.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252537+0000", 2024-09-17T07:31:08.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:31:08.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:31:08.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252538+0000", 2024-09-17T07:31:08.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2900000000000002e-07 2024-09-17T07:31:08.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252542+0000", 2024-09-17T07:31:08.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8600000000000003e-06 2024-09-17T07:31:08.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252545+0000", 2024-09-17T07:31:08.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.605e-06 2024-09-17T07:31:08.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325144+0000", 2024-09-17T07:31:08.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072599460000000005 2024-09-17T07:31:08.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325223+0000", 2024-09-17T07:31:08.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8473999999999994e-05 2024-09-17T07:31:08.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325501+0000", 2024-09-17T07:31:08.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000278063 2024-09-17T07:31:08.100 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.100 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.100 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.100 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19990 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6422528~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.101 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:31:08.101 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.823839815, 2024-09-17T07:31:08.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073241127000000003, 2024-09-17T07:31:08.101 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.101 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.101 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.101 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19990 2024-09-17T07:31:08.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.101 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:31:08.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:31:08.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252548+0000", 2024-09-17T07:31:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:31:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252552+0000", 2024-09-17T07:31:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4719999999999997e-06 2024-09-17T07:31:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252557+0000", 2024-09-17T07:31:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3039999999999998e-06 2024-09-17T07:31:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252560+0000", 2024-09-17T07:31:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8830000000000002e-06 2024-09-17T07:31:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325529+0000", 2024-09-17T07:31:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072968880999999999 2024-09-17T07:31:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325574+0000", 2024-09-17T07:31:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5466000000000001e-05 2024-09-17T07:31:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325789+0000", 2024-09-17T07:31:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021490999999999999 2024-09-17T07:31:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19991 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6651904~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:31:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.823820513, 2024-09-17T07:31:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073720552999999994, 2024-09-17T07:31:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19991 2024-09-17T07:31:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:31:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:31:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252567+0000", 2024-09-17T07:31:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4699999999999998e-07 2024-09-17T07:31:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252568+0000", 2024-09-17T07:31:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5300000000000002e-07 2024-09-17T07:31:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252572+0000", 2024-09-17T07:31:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9160000000000003e-06 2024-09-17T07:31:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252575+0000", 2024-09-17T07:31:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4569999999999999e-06 2024-09-17T07:31:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325820+0000", 2024-09-17T07:31:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073245597999999995 2024-09-17T07:31:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.325901+0000", 2024-09-17T07:31:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0913000000000003e-05 2024-09-17T07:31:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326288+0000", 2024-09-17T07:31:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00038646900000000002 2024-09-17T07:31:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19992 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 6881280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:31:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.82381015999999, 2024-09-17T07:31:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074054683999999996, 2024-09-17T07:31:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19992 2024-09-17T07:31:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:31:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252577+0000", 2024-09-17T07:31:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252578+0000", 2024-09-17T07:31:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0599999999999999e-07 2024-09-17T07:31:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252579+0000", 2024-09-17T07:31:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2500000000000004e-07 2024-09-17T07:31:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252582+0000", 2024-09-17T07:31:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-17T07:31:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252585+0000", 2024-09-17T07:31:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.542e-06 2024-09-17T07:31:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326315+0000", 2024-09-17T07:31:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073729921000000004 2024-09-17T07:31:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326359+0000", 2024-09-17T07:31:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4095999999999999e-05 2024-09-17T07:31:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326632+0000", 2024-09-17T07:31:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027314499999999997 2024-09-17T07:31:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19993 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7110656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:31:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.82379985599999, 2024-09-17T07:31:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074425142999999999, 2024-09-17T07:31:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19993 2024-09-17T07:31:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:31:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:31:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252588+0000", 2024-09-17T07:31:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:31:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252589+0000", 2024-09-17T07:31:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.33e-07 2024-09-17T07:31:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252593+0000", 2024-09-17T07:31:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:31:08.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252595+0000", 2024-09-17T07:31:08.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.621e-06 2024-09-17T07:31:08.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326666+0000", 2024-09-17T07:31:08.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074070787999999999 2024-09-17T07:31:08.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.326752+0000", 2024-09-17T07:31:08.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5748000000000006e-05 2024-09-17T07:31:08.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327013+0000", 2024-09-17T07:31:08.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026098400000000002 2024-09-17T07:31:08.119 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.119 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.119 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.119 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19994 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7340032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.120 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:31:08.120 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.823789397, 2024-09-17T07:31:08.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074765920999999999, 2024-09-17T07:31:08.120 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.120 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.120 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.120 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.120 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.120 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.120 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19994 2024-09-17T07:31:08.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.120 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:31:08.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:31:08.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252598+0000", 2024-09-17T07:31:08.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.17e-07 2024-09-17T07:31:08.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252599+0000", 2024-09-17T07:31:08.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2099999999999995e-07 2024-09-17T07:31:08.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252603+0000", 2024-09-17T07:31:08.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8430000000000003e-06 2024-09-17T07:31:08.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252606+0000", 2024-09-17T07:31:08.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4499999999999998e-06 2024-09-17T07:31:08.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327047+0000", 2024-09-17T07:31:08.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074441050999999994 2024-09-17T07:31:08.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327121+0000", 2024-09-17T07:31:08.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4283999999999996e-05 2024-09-17T07:31:08.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327364+0000", 2024-09-17T07:31:08.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000243155 2024-09-17T07:31:08.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.124 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.124 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19995 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7569408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.124 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:31:08.124 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.82377870000001, 2024-09-17T07:31:08.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075172694999999998, 2024-09-17T07:31:08.124 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.124 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.125 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.125 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19995 2024-09-17T07:31:08.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.125 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:31:08.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:31:08.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252609+0000", 2024-09-17T07:31:08.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:31:08.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252610+0000", 2024-09-17T07:31:08.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6799999999999999e-07 2024-09-17T07:31:08.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252614+0000", 2024-09-17T07:31:08.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8700000000000002e-06 2024-09-17T07:31:08.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252616+0000", 2024-09-17T07:31:08.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.419e-06 2024-09-17T07:31:08.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327407+0000", 2024-09-17T07:31:08.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074790617000000004 2024-09-17T07:31:08.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327498+0000", 2024-09-17T07:31:08.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0758000000000001e-05 2024-09-17T07:31:08.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327782+0000", 2024-09-17T07:31:08.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028395599999999998 2024-09-17T07:31:08.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.128 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.128 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.129 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19996 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7798784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.129 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:31:08.129 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.823768384, 2024-09-17T07:31:08.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075605832999999997, 2024-09-17T07:31:08.129 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.129 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.129 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.129 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.129 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19996 2024-09-17T07:31:08.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.129 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:31:08.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:31:08.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252619+0000", 2024-09-17T07:31:08.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0699999999999999e-07 2024-09-17T07:31:08.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252620+0000", 2024-09-17T07:31:08.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4099999999999997e-07 2024-09-17T07:31:08.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252624+0000", 2024-09-17T07:31:08.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8419999999999998e-06 2024-09-17T07:31:08.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252627+0000", 2024-09-17T07:31:08.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4980000000000001e-06 2024-09-17T07:31:08.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327844+0000", 2024-09-17T07:31:08.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075217709999999993 2024-09-17T07:31:08.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.327927+0000", 2024-09-17T07:31:08.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2628999999999998e-05 2024-09-17T07:31:08.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328225+0000", 2024-09-17T07:31:08.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00029800600000000002 2024-09-17T07:31:08.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.133 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.133 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.133 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19997 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8028160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.133 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:31:08.133 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.823758031, 2024-09-17T07:31:08.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076061237000000004, 2024-09-17T07:31:08.133 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.133 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.134 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.134 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.134 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19997 2024-09-17T07:31:08.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.134 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:31:08.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:31:08.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252630+0000", 2024-09-17T07:31:08.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:31:08.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252635+0000", 2024-09-17T07:31:08.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8269999999999997e-06 2024-09-17T07:31:08.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252639+0000", 2024-09-17T07:31:08.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.527e-06 2024-09-17T07:31:08.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252642+0000", 2024-09-17T07:31:08.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6129999999999998e-06 2024-09-17T07:31:08.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328268+0000", 2024-09-17T07:31:08.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.075626323999999995 2024-09-17T07:31:08.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328352+0000", 2024-09-17T07:31:08.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3578999999999994e-05 2024-09-17T07:31:08.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328691+0000", 2024-09-17T07:31:08.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033915099999999999 2024-09-17T07:31:08.137 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.137 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.137 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.137 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19998 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8257536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.138 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:31:08.138 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.82374258, 2024-09-17T07:31:08.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076441903000000005, 2024-09-17T07:31:08.138 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.138 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.138 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.138 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19998 2024-09-17T07:31:08.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.138 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:31:08.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:31:08.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252645+0000", 2024-09-17T07:31:08.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.34e-07 2024-09-17T07:31:08.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252646+0000", 2024-09-17T07:31:08.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1900000000000001e-07 2024-09-17T07:31:08.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252651+0000", 2024-09-17T07:31:08.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4710000000000001e-06 2024-09-17T07:31:08.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.252653+0000", 2024-09-17T07:31:08.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4899999999999999e-06 2024-09-17T07:31:08.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328747+0000", 2024-09-17T07:31:08.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.076094192000000005 2024-09-17T07:31:08.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.328824+0000", 2024-09-17T07:31:08.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7096999999999997e-05 2024-09-17T07:31:08.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.329087+0000", 2024-09-17T07:31:08.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00026249999999999998 2024-09-17T07:31:08.142 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.142 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.142 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.142 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20030 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7356416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.142 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:31:08.142 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.813626421, 2024-09-17T07:31:08.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072172616999999994, 2024-09-17T07:31:08.142 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.142 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.142 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.142 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.142 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20030 2024-09-17T07:31:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:31:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262761+0000", 2024-09-17T07:31:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262762+0000", 2024-09-17T07:31:08.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6800000000000001e-07 2024-09-17T07:31:08.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262764+0000", 2024-09-17T07:31:08.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.181e-06 2024-09-17T07:31:08.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262771+0000", 2024-09-17T07:31:08.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2729999999999997e-06 2024-09-17T07:31:08.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262777+0000", 2024-09-17T07:31:08.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9100000000000002e-06 2024-09-17T07:31:08.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334558+0000", 2024-09-17T07:31:08.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071780790999999997 2024-09-17T07:31:08.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334647+0000", 2024-09-17T07:31:08.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9448000000000001e-05 2024-09-17T07:31:08.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.334934+0000", 2024-09-17T07:31:08.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00028654599999999999 2024-09-17T07:31:08.146 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.146 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.146 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.146 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20031 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7585792~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.146 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:31:08.146 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.813604709, 2024-09-17T07:31:08.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072606135000000002, 2024-09-17T07:31:08.147 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.147 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.147 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.147 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.147 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20031 2024-09-17T07:31:08.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.147 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:31:08.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:31:08.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262783+0000", 2024-09-17T07:31:08.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9499999999999998e-07 2024-09-17T07:31:08.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262785+0000", 2024-09-17T07:31:08.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4529999999999999e-06 2024-09-17T07:31:08.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262789+0000", 2024-09-17T07:31:08.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6330000000000004e-06 2024-09-17T07:31:08.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262792+0000", 2024-09-17T07:31:08.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6309999999999999e-06 2024-09-17T07:31:08.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335116+0000", 2024-09-17T07:31:08.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072323897999999998 2024-09-17T07:31:08.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335166+0000", 2024-09-17T07:31:08.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9882000000000001e-05 2024-09-17T07:31:08.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335389+0000", 2024-09-17T07:31:08.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022334300000000001 2024-09-17T07:31:08.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.150 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.151 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20032 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 7815168~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.151 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:31:08.151 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.81359251000001, 2024-09-17T07:31:08.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073051230999999994, 2024-09-17T07:31:08.151 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.151 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.151 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.151 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20032 2024-09-17T07:31:08.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.152 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:31:08.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:31:08.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262795+0000", 2024-09-17T07:31:08.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.16e-07 2024-09-17T07:31:08.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262796+0000", 2024-09-17T07:31:08.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0809999999999999e-06 2024-09-17T07:31:08.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262800+0000", 2024-09-17T07:31:08.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8299999999999998e-06 2024-09-17T07:31:08.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262803+0000", 2024-09-17T07:31:08.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7240000000000001e-06 2024-09-17T07:31:08.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335431+0000", 2024-09-17T07:31:08.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072627852000000007 2024-09-17T07:31:08.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335525+0000", 2024-09-17T07:31:08.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3708000000000005e-05 2024-09-17T07:31:08.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335846+0000", 2024-09-17T07:31:08.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032182000000000001 2024-09-17T07:31:08.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.155 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.155 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.155 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20033 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8044544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.155 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:31:08.155 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.813581652, 2024-09-17T07:31:08.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073284199999999994, 2024-09-17T07:31:08.155 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.156 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.156 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.156 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.156 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20033 2024-09-17T07:31:08.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.156 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:31:08.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:31:08.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262806+0000", 2024-09-17T07:31:08.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.03e-07 2024-09-17T07:31:08.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262807+0000", 2024-09-17T07:31:08.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0070000000000001e-06 2024-09-17T07:31:08.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262811+0000", 2024-09-17T07:31:08.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7840000000000001e-06 2024-09-17T07:31:08.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262814+0000", 2024-09-17T07:31:08.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9529999999999999e-06 2024-09-17T07:31:08.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335863+0000", 2024-09-17T07:31:08.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073048921000000003 2024-09-17T07:31:08.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.335904+0000", 2024-09-17T07:31:08.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1545000000000003e-05 2024-09-17T07:31:08.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336090+0000", 2024-09-17T07:31:08.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185787 2024-09-17T07:31:08.159 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.159 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.159 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.160 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:20034 3.1c 3:38b18595:::rbd_data.11838dd88fc3.0000000000000002:d [read 8273920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-17T07:31:08.160 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:31:08.160 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.813567392, 2024-09-17T07:31:08.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073670478999999997, 2024-09-17T07:31:08.160 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.160 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.160 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.160 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 20034 2024-09-17T07:31:08.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.161 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:31:08.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:31:08.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262820+0000", 2024-09-17T07:31:08.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8299999999999998e-07 2024-09-17T07:31:08.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262822+0000", 2024-09-17T07:31:08.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3510000000000001e-06 2024-09-17T07:31:08.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262826+0000", 2024-09-17T07:31:08.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1699999999999999e-06 2024-09-17T07:31:08.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.262829+0000", 2024-09-17T07:31:08.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8990000000000001e-06 2024-09-17T07:31:08.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336126+0000", 2024-09-17T07:31:08.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073297254000000006 2024-09-17T07:31:08.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336211+0000", 2024-09-17T07:31:08.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4728000000000006e-05 2024-09-17T07:31:08.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:05.336491+0000", 2024-09-17T07:31:08.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00027979399999999998 2024-09-17T07:31:08.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.164 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.164 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.164 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:116163 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-17T07:31:08.164 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:31:08.164 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.811232523999998, 2024-09-17T07:31:08.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8549867870000001, 2024-09-17T07:31:08.164 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.165 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.165 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.165 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.165 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116163 2024-09-17T07:31:08.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.165 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:31:08.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265155+0000", 2024-09-17T07:31:08.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265157+0000", 2024-09-17T07:31:08.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-06 2024-09-17T07:31:08.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265168+0000", 2024-09-17T07:31:08.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0827e-05 2024-09-17T07:31:08.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265198+0000", 2024-09-17T07:31:08.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0383e-05 2024-09-17T07:31:08.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265212+0000", 2024-09-17T07:31:08.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3799e-05 2024-09-17T07:31:08.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265338+0000", 2024-09-17T07:31:08.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012576800000000001 2024-09-17T07:31:08.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:31:08.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:26.265357+0000", 2024-09-17T07:31:08.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9417000000000001e-05 2024-09-17T07:31:08.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:27.083102+0000", 2024-09-17T07:31:08.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.81774456299999998 2024-09-17T07:31:08.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:31:08.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:27.083113+0000", 2024-09-17T07:31:08.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1154e-05 2024-09-17T07:31:08.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:28.119793+0000", 2024-09-17T07:31:08.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.036680365 2024-09-17T07:31:08.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:28.119943+0000", 2024-09-17T07:31:08.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014940999999999999 2024-09-17T07:31:08.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:28.120142+0000", 2024-09-17T07:31:08.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00019897099999999999 2024-09-17T07:31:08.170 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.170 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.170 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.170 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.170 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.170 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:08.172 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:31:08.172 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:08.170+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:08.273 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:31:08.271+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:31:08.278 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:08.278 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:31:08.278 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:31:08.278 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:08.287 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:31:08.374 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:08.373+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:08.398 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:31:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:31:08.398 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:31:08.398 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:08.407 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:31:08.474 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:08.473+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:08.517 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:31:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:31:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:31:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:31:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.610459373, 2024-09-17T07:31:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:31:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:31:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:31:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:31:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:31:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:31:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:31:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:31:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:31:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:31:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:31:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:31:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:31:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:31:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:31:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:31:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:31:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:31:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:31:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.610440696, 2024-09-17T07:31:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:31:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:31:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:31:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:31:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:31:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:31:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:31:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:31:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:31:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:31:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:31:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:31:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:31:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:31:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:31:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:31:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:31:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:31:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:31:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.610422343, 2024-09-17T07:31:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:31:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:31:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:31:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:31:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:31:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:31:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:31:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:31:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:31:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:31:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:31:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:31:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:31:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:31:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:31:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:31:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:31:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:31:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:31:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.610402745, 2024-09-17T07:31:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:31:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:31:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:31:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:31:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:31:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:31:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:31:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:31:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:31:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:31:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:31:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:31:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:31:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:31:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:31:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:31:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:31:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:31:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:31:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.61038279, 2024-09-17T07:31:08.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:31:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.547 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:31:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:31:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:31:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:31:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:31:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:31:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:31:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:31:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:31:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:31:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:31:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:31:08.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:31:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:31:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:31:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:31:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:31:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:31:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.610364135, 2024-09-17T07:31:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:31:08.552 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:31:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:31:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:31:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:31:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:31:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:31:08.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:31:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:31:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:31:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:31:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:31:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:31:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:31:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:31:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:31:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:31:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:31:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.556 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:31:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.610345659, 2024-09-17T07:31:08.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:31:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.558 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:31:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:31:08.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:31:08.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:31:08.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:31:08.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:31:08.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:31:08.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:31:08.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:31:08.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:31:08.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:31:08.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:31:08.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:31:08.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:31:08.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:31:08.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:31:08.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:31:08.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.565 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:31:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.61032606000001, 2024-09-17T07:31:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:31:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:31:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:31:08.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:31:08.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:31:08.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:31:08.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:31:08.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:31:08.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:31:08.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:31:08.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:31:08.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:31:08.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:31:08.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:31:08.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.570 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.570 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:08.570 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:31:08.570 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.61030748500001, 2024-09-17T07:31:08.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:31:08.570 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.571 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.571 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.571 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:31:08.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.571 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:31:08.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:31:08.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:31:08.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:31:08.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:31:08.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:31:08.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:31:08.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:31:08.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:31:08.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:31:08.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:31:08.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:31:08.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:31:08.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:31:08.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:31:08.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:31:08.574 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.574 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.575 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.575 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:08.575 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:31:08.575 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.610285058, 2024-09-17T07:31:08.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:31:08.575 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.575 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.575 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.576 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.576 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:31:08.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.576 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:31:08.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:31:08.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:31:08.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:31:08.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:31:08.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:31:08.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:31:08.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:31:08.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:31:08.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:31:08.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:31:08.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:31:08.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:31:08.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:31:08.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:31:08.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:31:08.579 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.579 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.579 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.579 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:08.580 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:31:08.580 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.610257241, 2024-09-17T07:31:08.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:31:08.580 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.580 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.580 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.580 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.580 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.580 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.580 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:31:08.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.580 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:31:08.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:31:08.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:31:08.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:31:08.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:31:08.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:31:08.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:31:08.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:31:08.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:31:08.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:31:08.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:31:08.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:31:08.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:31:08.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:31:08.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:31:08.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:31:08.584 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.584 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.584 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.584 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:08.584 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:31:08.585 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.610235297, 2024-09-17T07:31:08.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:31:08.585 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.585 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.585 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.585 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:31:08.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.585 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:31:08.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:31:08.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:31:08.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:31:08.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:31:08.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:31:08.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:31:08.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:31:08.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:31:08.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:31:08.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:31:08.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:31:08.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:31:08.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:31:08.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:31:08.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:31:08.591 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.591 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.591 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.592 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:08.592 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:31:08.592 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.610214193, 2024-09-17T07:31:08.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:31:08.592 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.592 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.592 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.593 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:31:08.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.593 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:31:08.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:31:08.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:31:08.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:31:08.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:31:08.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:31:08.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:31:08.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:31:08.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:31:08.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:31:08.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:31:08.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:31:08.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:31:08.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:31:08.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:31:08.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:31:08.598 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.598 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.598 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.598 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:08.599 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:31:08.599 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.61019459000001, 2024-09-17T07:31:08.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:31:08.599 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.599 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.599 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.599 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:31:08.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.599 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:31:08.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:31:08.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:31:08.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:31:08.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:31:08.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:31:08.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:31:08.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:31:08.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:31:08.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:31:08.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:31:08.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:31:08.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:31:08.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:31:08.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:31:08.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:31:08.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.603 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.603 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:08.603 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:31:08.603 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.657253477, 2024-09-17T07:31:08.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:31:08.603 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.603 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.603 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.604 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:31:08.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.604 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:31:08.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:31:08.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:31:08.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:31:08.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:31:08.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:31:08.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:31:08.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:31:08.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:31:08.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:31:08.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:31:08.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:31:08.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:31:08.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:31:08.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:31:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:31:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:08.611 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:31:08.611 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.657239817, 2024-09-17T07:31:08.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:31:08.611 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.611 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:31:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:31:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:31:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:31:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:31:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:31:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:31:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:31:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:31:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:31:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:31:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:31:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:31:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:31:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:31:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:31:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:31:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:31:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.657225035, 2024-09-17T07:31:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:31:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:31:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:31:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:31:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:31:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:31:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:31:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:31:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:31:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:31:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:31:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:31:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:31:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:31:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:31:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:31:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:31:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:31:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:31:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.65721414399999, 2024-09-17T07:31:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:31:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:31:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:31:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:31:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:31:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:31:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:31:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:31:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:31:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:31:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:31:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:31:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:31:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:31:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:31:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:31:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:31:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:31:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:31:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.657204584, 2024-09-17T07:31:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:31:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:31:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:31:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:31:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:31:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:31:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:31:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:31:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:31:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:31:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:31:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:31:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:31:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:31:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:31:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:31:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:31:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:31:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:31:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.65719402400001, 2024-09-17T07:31:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:31:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:31:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:31:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:31:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:31:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:31:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:31:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:31:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:31:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:31:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:31:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:31:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:31:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:31:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:31:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:31:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:31:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:31:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:08.636 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:08.638 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:31:08.638 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:08.636+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:08.739 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:31:08.737+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:31:08.741 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:08.741 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:31:08.741 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:31:08.741 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:08.750 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:31:08.840 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:31:08.838+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:31:08.894 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:31:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:31:08.894 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:31:08.894 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:08.903 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:31:08.944 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:08.944+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:09.042 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:09.040+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:09.047 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:09.047 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:31:09.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:31:09.047 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:31:09.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.047 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:31:09.047 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:31:09.047 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 162.71115826299999, 2024-09-17T07:31:09.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:31:09.047 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.048 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.048 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:09.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:31:09.048 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:31:09.048 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:31:09.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.048 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:31:09.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:31:09.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:31:09.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:31:09.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:31:09.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:31:09.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:31:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:31:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:31:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:31:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:31:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:31:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:31:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:31:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:31:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:31:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:31:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:31:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:31:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:31:09.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:31:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:31:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:31:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:31:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:31:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:31:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:31:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.122106113, 2024-09-17T07:31:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:31:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:31:09.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:31:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:31:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:31:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:31:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:31:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:31:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:31:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:31:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:31:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:31:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:31:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:31:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:31:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:31:09.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:31:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:31:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:31:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.11682299700001, 2024-09-17T07:31:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:31:09.058 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:31:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:31:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:31:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:31:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:31:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:31:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:31:09.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:31:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:31:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:31:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:31:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:31:09.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:31:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:31:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:31:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:31:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:31:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:31:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.11679201699999, 2024-09-17T07:31:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:31:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:31:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:31:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:31:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:31:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:31:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:31:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:31:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:31:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:31:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:31:09.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:31:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:31:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:31:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:31:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:31:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:31:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:31:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:31:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.11676196799999, 2024-09-17T07:31:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:31:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.067 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:31:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:31:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:31:09.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:31:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:31:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:31:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:31:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:31:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:31:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:31:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:31:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:31:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:31:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:31:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:31:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:31:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:31:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:31:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.11673356099999, 2024-09-17T07:31:09.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:31:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:31:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:31:09.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:31:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:31:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:31:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:31:09.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:31:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:31:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:31:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:31:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:31:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:31:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:31:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:31:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:31:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:31:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:31:09.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:31:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.11671373999999, 2024-09-17T07:31:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:31:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.076 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:31:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:31:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:31:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:31:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:31:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:31:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:31:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:31:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:31:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:31:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:31:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:31:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:31:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:31:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:31:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:31:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:31:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:31:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.11668948799999, 2024-09-17T07:31:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:31:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.080 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:31:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:31:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:31:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:31:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:31:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:31:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:31:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:31:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:31:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:31:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:31:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:31:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:31:09.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:31:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:31:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:31:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:31:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:31:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.1112669, 2024-09-17T07:31:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:31:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:31:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:31:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:31:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:31:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:31:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:31:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:31:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:31:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:31:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:31:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:31:09.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:31:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:31:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:31:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:31:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:31:09.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:31:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:31:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.11122002499999, 2024-09-17T07:31:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:31:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.089 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:31:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:31:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:31:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:31:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:31:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:31:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:31:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:31:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:31:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:31:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:31:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:31:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:31:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:31:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:31:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:31:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:31:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.093 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:31:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.111205321, 2024-09-17T07:31:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:31:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:31:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:31:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:31:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:31:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:31:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:31:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:31:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:31:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:31:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:31:09.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:31:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:31:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:31:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:31:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:31:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:31:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:31:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:31:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.11118222600001, 2024-09-17T07:31:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:31:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:31:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:31:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:31:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:31:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:31:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:31:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:31:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:31:09.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:31:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:31:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:31:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:31:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:31:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:31:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:31:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:31:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:31:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:31:09.102 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.11115085099999, 2024-09-17T07:31:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:31:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:31:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:31:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:31:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:31:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:31:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:31:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:31:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:31:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:31:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:31:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:31:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:31:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:31:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:31:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:31:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:31:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:31:09.106 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:31:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.11113559099999, 2024-09-17T07:31:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:31:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.107 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:31:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:31:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:31:09.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:31:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:31:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:31:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:31:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:31:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:31:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:31:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:31:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:31:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:31:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:31:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:31:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:31:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:31:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:31:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.11111552099999, 2024-09-17T07:31:09.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:31:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:31:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:31:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:31:09.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:31:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:31:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:31:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:31:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:31:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:31:09.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:31:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:31:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:31:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:31:09.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:31:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.11110358100001, 2024-09-17T07:31:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:31:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.116 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:31:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:31:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:31:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:31:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:31:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:31:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:31:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:31:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:31:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:31:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:31:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:31:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:31:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:31:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:31:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:31:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:31:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.120 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:31:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.111080423, 2024-09-17T07:31:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:31:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:31:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:31:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:31:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:31:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:31:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:31:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:31:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:31:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:31:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:31:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:31:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:31:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:31:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:31:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:31:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:31:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:31:09.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:115644 3.15 3:af37e0f6:::rbd_header.118311e0d6f8:head [watch reconnect cookie 140227662450992 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T07:31:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:31:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.967923528999997, 2024-09-17T07:31:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.013658722, 2024-09-17T07:31:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.125 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115644 2024-09-17T07:31:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:31:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:31:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077907+0000", 2024-09-17T07:31:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.08e-06 2024-09-17T07:31:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077916+0000", 2024-09-17T07:31:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-09-17T07:31:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077938+0000", 2024-09-17T07:31:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1883999999999999e-05 2024-09-17T07:31:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077950+0000", 2024-09-17T07:31:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2298e-05 2024-09-17T07:31:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.078037+0000", 2024-09-17T07:31:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6726999999999998e-05 2024-09-17T07:31:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-17T07:31:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.078046+0000", 2024-09-17T07:31:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6010000000000001e-06 2024-09-17T07:31:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085705+0000", 2024-09-17T07:31:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007659084 2024-09-17T07:31:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:31:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085718+0000", 2024-09-17T07:31:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2081e-05 2024-09-17T07:31:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.089697+0000", 2024-09-17T07:31:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.003979373 2024-09-17T07:31:09.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.089875+0000", 2024-09-17T07:31:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017760000000000001 2024-09-17T07:31:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.091563+0000", 2024-09-17T07:31:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001688836 2024-09-17T07:31:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.130 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:115645 3.15 3:af37e0f6:::rbd_header.118311e0d6f8:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T07:31:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:31:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.965106009000003, 2024-09-17T07:31:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.011649171, 2024-09-17T07:31:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115645 2024-09-17T07:31:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:31:09.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:31:09.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080724+0000", 2024-09-17T07:31:09.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-17T07:31:09.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080735+0000", 2024-09-17T07:31:09.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1544e-05 2024-09-17T07:31:09.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080751+0000", 2024-09-17T07:31:09.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5244e-05 2024-09-17T07:31:09.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080761+0000", 2024-09-17T07:31:09.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0372e-05 2024-09-17T07:31:09.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080834+0000", 2024-09-17T07:31:09.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3263999999999995e-05 2024-09-17T07:31:09.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:09.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080842+0000", 2024-09-17T07:31:09.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2209999999999997e-06 2024-09-17T07:31:09.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085755+0000", 2024-09-17T07:31:09.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049130729999999996 2024-09-17T07:31:09.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:31:09.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085766+0000", 2024-09-17T07:31:09.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0288e-05 2024-09-17T07:31:09.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.090746+0000", 2024-09-17T07:31:09.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0049801190000001 2024-09-17T07:31:09.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.090798+0000", 2024-09-17T07:31:09.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2502e-05 2024-09-17T07:31:09.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-09-17T07:31:09.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.091037+0000", 2024-09-17T07:31:09.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0002385 2024-09-17T07:31:09.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:09.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092119+0000", 2024-09-17T07:31:09.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010820249999999999 2024-09-17T07:31:09.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:31:09.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092276+0000", 2024-09-17T07:31:09.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000156737 2024-09-17T07:31:09.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:31:09.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092330+0000", 2024-09-17T07:31:09.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4666999999999997e-05 2024-09-17T07:31:09.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092371+0000", 2024-09-17T07:31:09.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1043e-05 2024-09-17T07:31:09.137 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.137 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.137 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.138 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:116418 3.13 3:cdf2692d:::rbd_data.1183cb746bb8.0000000000000024:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T07:31:09.138 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:31:09.138 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.875689612000002, 2024-09-17T07:31:09.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92140365899999999, 2024-09-17T07:31:09.138 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:09.138 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:09.138 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:09.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:09.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:09.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:09.138 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116418 2024-09-17T07:31:09.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.138 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:09.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:09.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:31:09.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:09.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:31:09.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:09.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:09.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170140+0000", 2024-09-17T07:31:09.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.398e-06 2024-09-17T07:31:09.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:09.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170144+0000", 2024-09-17T07:31:09.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4399999999999998e-06 2024-09-17T07:31:09.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:09.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170157+0000", 2024-09-17T07:31:09.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2133999999999999e-05 2024-09-17T07:31:09.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:09.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170164+0000", 2024-09-17T07:31:09.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7200000000000006e-06 2024-09-17T07:31:09.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170227+0000", 2024-09-17T07:31:09.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3035999999999998e-05 2024-09-17T07:31:09.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:31:09.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170238+0000", 2024-09-17T07:31:09.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0397000000000001e-05 2024-09-17T07:31:09.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:09.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086367+0000", 2024-09-17T07:31:09.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.916128889 2024-09-17T07:31:09.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:09.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086523+0000", 2024-09-17T07:31:09.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015638800000000001 2024-09-17T07:31:09.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:31:09.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086652+0000", 2024-09-17T07:31:09.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000128806 2024-09-17T07:31:09.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:31:09.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091261+0000", 2024-09-17T07:31:09.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046091020000000003 2024-09-17T07:31:09.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:09.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091449+0000", 2024-09-17T07:31:09.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000188486 2024-09-17T07:31:09.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:31:09.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091503+0000", 2024-09-17T07:31:09.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3254999999999997e-05 2024-09-17T07:31:09.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:09.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:09.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:09.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091542+0000", 2024-09-17T07:31:09.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9607999999999998e-05 2024-09-17T07:31:09.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.144 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:09.144 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:09.144 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:09.144 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T07:31:09.146 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:31:09.146 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:09.144+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:09.236 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:31:09.268 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:09.268 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:31:09.268 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:31:09.268 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:09.268 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:09.267+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:09.278 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:31:09.362 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:09.361+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:09.372 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:31:09.372 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:31:09.390 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:31:09.390 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:31:09.407 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:31:09.407 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:31:09.423 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:31:09.423 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:31:09.430 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:09.430 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:09.439 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:31:09.440 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:31:09.441 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:31:09.448 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:09.446+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:09.458 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:31:09.458 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:31:09.475 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:31:09.475 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:31:09.492 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:31:09.493 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:31:09.549 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:09.549 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:09.550 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:09.548+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:09.558 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:31:09.650 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:31:09.649+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:31:09.665 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:09.665 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:09.674 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:31:09.751 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:09.750+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:09.781 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:09.781 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:09.790 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:31:09.873 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:31:09.872+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:31:09.937 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:09.937 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:09.945 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:31:09.986 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:09.985+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:10.055 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:10.054+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:10.092 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is set 2024-09-17T07:31:10.095 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:10.095 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:10.113 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:31:10.155 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:31:10.154+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:31:10.236 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:10.236 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:10.245 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:31:10.288 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:10.287+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:10.352 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:10.352 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:10.356 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:10.355+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:10.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:31:10.460 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:31:10.457+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:31:10.508 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:10.508 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:10.516 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:31:10.558 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:31:10.557+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:31:10.624 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:10.624 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:10.633 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:31:10.636 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:31:10.659 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:10.659+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:10.741 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:10.741 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:10.750 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:31:10.760 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:31:10.759+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:31:10.859 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:10.859 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:10.861 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:31:10.860+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:31:10.867 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:31:10.963 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:10.961+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:10.970 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:10.970 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:10.974 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:10.974 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:31:10.979 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:31:10.986 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2024-09-17T07:31:10.782023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":132149853,"num_objects":1014,"num_object_clones":156,"num_object_copies":2028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1014,"num_whiteouts":760,"num_read":164781,"num_read_kb":1207113,"num_write":23001,"num_write_kb":794324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6841,"num_bytes_recovered":667554936,"num_keys_recovered":891,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20008,"ondisk_log_size":20008,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":448856,"kb_used_data":29852,"kb_used_omap":27,"kb_used_meta":418788,"kb_avail":754525864,"statfs":{"total":773094113280,"available":772634484736,"internally_reserved":0,"allocated":30568448,"data_stored":167086339,"data_compressed":8953453,"data_compressed_allocated":11898880,"data_compressed_original":156991488,"omap_allocated":28217,"internal_metadata":428839367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-46741829,"num_objects":26,"num_object_clones":-3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":-2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":35,"num_read":10218,"num_read_kb":55710,"num_write":1327,"num_write_kb":41328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1290,"num_bytes_recovered":92879724,"num_keys_recovered":80,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000687"},"pg_stats":[{"pgid":"3.1f","version":"159'849","reported_seq":4505,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T07:31:08.150856+0000","last_change":"2024-09-17T07:31:04.100988+0000","last_active":"2024-09-17T07:31:08.150856+0000","last_peered":"2024-09-17T07:31:08.150856+0000","last_clean":"2024-09-17T07:31:08.150856+0000","last_became_active":"2024-09-17T07:30:43.092456+0000","last_became_peered":"2024-09-17T07:30:43.092456+0000","last_unstale":"2024-09-17T07:31:08.150856+0000","last_undegraded":"2024-09-17T07:31:08.150856+0000","last_fullsized":"2024-09-17T07:31:08.150856+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":849,"log_dups_size":0,"ondisk_log_size":849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:12.174954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012895070000000001,"stat_sum":{"num_bytes":4706440,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":25,"num_read":5785,"num_read_kb":34299,"num_write":986,"num_write_kb":41977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":26159662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.1e","version":"152'516","reported_seq":3091,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T07:31:07.095063+0000","last_change":"2024-09-17T07:31:04.102043+0000","last_active":"2024-09-17T07:31:07.095063+0000","last_peered":"2024-09-17T07:31:07.095063+0000","last_clean":"2024-09-17T07:31:07.095063+0000","last_became_active":"2024-09-17T07:30:19.087213+0000","last_became_peered":"2024-09-17T07:30:19.087213+0000","last_unstale":"2024-09-17T07:31:07.095063+0000","last_undegraded":"2024-09-17T07:31:07.095063+0000","last_fullsized":"2024-09-17T07:31:07.095063+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:54.591164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046235499999999997,"stat_sum":{"num_bytes":3382661,"num_objects":49,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":40,"num_read":3553,"num_read_kb":30152,"num_write":508,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":15743718,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.1d","version":"160'536","reported_seq":2892,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.113643+0000","last_change":"2024-09-17T07:31:09.113841+0000","last_active":"2024-09-17T07:31:09.113643+0000","last_peered":"2024-09-17T07:31:09.113643+0000","last_clean":"2024-09-17T07:31:09.113643+0000","last_became_active":"2024-09-17T07:30:30.096044+0000","last_became_peered":"2024-09-17T07:30:30.096044+0000","last_unstale":"2024-09-17T07:31:09.113643+0000","last_undegraded":"2024-09-17T07:31:09.113643+0000","last_fullsized":"2024-09-17T07:31:09.113643+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:41:28.043401+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0051523380000000002,"stat_sum":{"num_bytes":2068938,"num_objects":42,"num_object_clones":6,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":34,"num_read":1948,"num_read_kb":28759,"num_write":577,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":27967854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.1c","version":"160'793","reported_seq":6089,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.136905+0000","last_change":"2024-09-17T07:31:09.115906+0000","last_active":"2024-09-17T07:31:09.136905+0000","last_peered":"2024-09-17T07:31:09.136905+0000","last_clean":"2024-09-17T07:31:09.136905+0000","last_became_active":"2024-09-17T07:30:31.098495+0000","last_became_peered":"2024-09-17T07:30:31.098495+0000","last_unstale":"2024-09-17T07:31:09.136905+0000","last_undegraded":"2024-09-17T07:31:09.136905+0000","last_fullsized":"2024-09-17T07:31:09.136905+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:26.465626+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0070809139999999998,"stat_sum":{"num_bytes":1392670,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":15,"num_read":7476,"num_read_kb":42412,"num_write":1041,"num_write_kb":38721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":26661650,"num_keys_recovered":180,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.1b","version":"154'549","reported_seq":4826,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T07:31:08.094106+0000","last_change":"2024-09-17T07:31:04.099221+0000","last_active":"2024-09-17T07:31:08.094106+0000","last_peered":"2024-09-17T07:31:08.094106+0000","last_clean":"2024-09-17T07:31:08.094106+0000","last_became_active":"2024-09-17T07:31:03.094058+0000","last_became_peered":"2024-09-17T07:31:03.094058+0000","last_unstale":"2024-09-17T07:31:08.094106+0000","last_undegraded":"2024-09-17T07:31:08.094106+0000","last_fullsized":"2024-09-17T07:31:08.094106+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:55.445792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072087799999999999,"stat_sum":{"num_bytes":2644152,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":19,"num_read":5413,"num_read_kb":52124,"num_write":561,"num_write_kb":22867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":674,"num_bytes_recovered":91784745,"num_keys_recovered":126,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.1a","version":"160'909","reported_seq":14535,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.192814+0000","last_change":"2024-09-17T07:31:09.100208+0000","last_active":"2024-09-17T07:31:09.192814+0000","last_peered":"2024-09-17T07:31:09.192814+0000","last_clean":"2024-09-17T07:31:09.192814+0000","last_became_active":"2024-09-17T07:30:31.090258+0000","last_became_peered":"2024-09-17T07:30:31.090258+0000","last_unstale":"2024-09-17T07:31:09.192814+0000","last_undegraded":"2024-09-17T07:31:09.192814+0000","last_fullsized":"2024-09-17T07:31:09.192814+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:40.042064+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0069821989999999997,"stat_sum":{"num_bytes":1673332,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":23,"num_read":16699,"num_read_kb":60966,"num_write":1187,"num_write_kb":20597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":19561990,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.19","version":"154'432","reported_seq":5598,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.094297+0000","last_change":"2024-09-17T07:31:09.094297+0000","last_active":"2024-09-17T07:31:09.094297+0000","last_peered":"2024-09-17T07:31:09.094297+0000","last_clean":"2024-09-17T07:31:09.094297+0000","last_became_active":"2024-09-17T07:30:18.083918+0000","last_became_peered":"2024-09-17T07:30:18.083918+0000","last_unstale":"2024-09-17T07:31:09.094297+0000","last_undegraded":"2024-09-17T07:31:09.094297+0000","last_fullsized":"2024-09-17T07:31:09.094297+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:30.795752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086112700000000005,"stat_sum":{"num_bytes":1916928,"num_objects":35,"num_object_clones":2,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":30,"num_read":4352,"num_read_kb":37354,"num_write":397,"num_write_kb":43089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":25536146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.18","version":"154'755","reported_seq":4574,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T07:31:07.093343+0000","last_change":"2024-09-17T07:31:04.097094+0000","last_active":"2024-09-17T07:31:07.093343+0000","last_peered":"2024-09-17T07:31:07.093343+0000","last_clean":"2024-09-17T07:31:07.093343+0000","last_became_active":"2024-09-17T07:30:43.091692+0000","last_became_peered":"2024-09-17T07:30:43.091692+0000","last_unstale":"2024-09-17T07:31:07.093343+0000","last_undegraded":"2024-09-17T07:31:07.093343+0000","last_fullsized":"2024-09-17T07:31:07.093343+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:17.851207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085812200000000001,"stat_sum":{"num_bytes":3145728,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":23,"num_read":5969,"num_read_kb":25327,"num_write":792,"num_write_kb":16603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":29456184,"num_keys_recovered":176,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.17","version":"153'427","reported_seq":4133,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T07:31:08.094494+0000","last_change":"2024-09-17T07:31:04.101208+0000","last_active":"2024-09-17T07:31:08.094494+0000","last_peered":"2024-09-17T07:31:08.094494+0000","last_clean":"2024-09-17T07:31:08.094494+0000","last_became_active":"2024-09-17T07:28:43.080428+0000","last_became_peered":"2024-09-17T07:28:43.080428+0000","last_unstale":"2024-09-17T07:31:08.094494+0000","last_undegraded":"2024-09-17T07:31:08.094494+0000","last_fullsized":"2024-09-17T07:31:08.094494+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:49.410936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001240063,"stat_sum":{"num_bytes":2056192,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":29,"num_read":4961,"num_read_kb":22717,"num_write":364,"num_write_kb":13531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.16","version":"157'484","reported_seq":4010,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.111149+0000","last_change":"2024-09-17T07:31:09.111149+0000","last_active":"2024-09-17T07:31:09.111149+0000","last_peered":"2024-09-17T07:31:09.111149+0000","last_clean":"2024-09-17T07:31:09.111149+0000","last_became_active":"2024-09-17T07:28:43.082313+0000","last_became_peered":"2024-09-17T07:28:43.082313+0000","last_unstale":"2024-09-17T07:31:09.111149+0000","last_undegraded":"2024-09-17T07:31:09.111149+0000","last_fullsized":"2024-09-17T07:31:09.111149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:23.909158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025815529999999999,"stat_sum":{"num_bytes":1703978,"num_objects":26,"num_object_clones":2,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":20,"num_read":5970,"num_read_kb":58212,"num_write":429,"num_write_kb":31849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.15","version":"152'373","reported_seq":4411,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.100516+0000","last_change":"2024-09-17T07:31:09.102318+0000","last_active":"2024-09-17T07:31:09.100516+0000","last_peered":"2024-09-17T07:31:09.100516+0000","last_clean":"2024-09-17T07:31:09.100516+0000","last_became_active":"2024-09-17T07:31:08.094940+0000","last_became_peered":"2024-09-17T07:31:08.094940+0000","last_unstale":"2024-09-17T07:31:09.100516+0000","last_undegraded":"2024-09-17T07:31:09.100516+0000","last_fullsized":"2024-09-17T07:31:09.100516+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:32.821352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048355430000000003,"stat_sum":{"num_bytes":11730944,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":21,"num_read":3586,"num_read_kb":32352,"num_write":368,"num_write_kb":22781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":431,"num_bytes_recovered":60567103,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.14","version":"160'452","reported_seq":2035,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.208351+0000","last_change":"2024-09-17T07:31:09.100477+0000","last_active":"2024-09-17T07:31:09.208351+0000","last_peered":"2024-09-17T07:31:09.208351+0000","last_clean":"2024-09-17T07:31:09.208351+0000","last_became_active":"2024-09-17T07:30:30.095364+0000","last_became_peered":"2024-09-17T07:30:30.095364+0000","last_unstale":"2024-09-17T07:31:09.208351+0000","last_undegraded":"2024-09-17T07:31:09.208351+0000","last_fullsized":"2024-09-17T07:31:09.208351+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:24.023571+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0070866380000000001,"stat_sum":{"num_bytes":1677602,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":22,"num_read":1414,"num_read_kb":17974,"num_write":481,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":16518806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"2.7","version":"0'0","reported_seq":307,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.102866+0000","last_change":"2024-09-17T07:31:05.098106+0000","last_active":"2024-09-17T07:31:09.102866+0000","last_peered":"2024-09-17T07:31:09.102866+0000","last_clean":"2024-09-17T07:31:09.102866+0000","last_became_active":"2024-09-17T07:31:05.097821+0000","last_became_peered":"2024-09-17T07:31:05.097821+0000","last_unstale":"2024-09-17T07:31:09.102866+0000","last_undegraded":"2024-09-17T07:31:09.102866+0000","last_fullsized":"2024-09-17T07:31:09.102866+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:12.297890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"159'1002","reported_seq":2651,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T07:31:08.094340+0000","last_change":"2024-09-17T07:31:04.100390+0000","last_active":"2024-09-17T07:31:08.094340+0000","last_peered":"2024-09-17T07:31:08.094340+0000","last_clean":"2024-09-17T07:31:08.094340+0000","last_became_active":"2024-09-17T07:28:43.079229+0000","last_became_peered":"2024-09-17T07:28:43.079229+0000","last_unstale":"2024-09-17T07:31:08.094340+0000","last_undegraded":"2024-09-17T07:31:08.094340+0000","last_fullsized":"2024-09-17T07:31:08.094340+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1002,"log_dups_size":0,"ondisk_log_size":1002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:32.510405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002047138,"stat_sum":{"num_bytes":6126684,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":28,"num_read":4625,"num_read_kb":48744,"num_write":1477,"num_write_kb":29297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.093115+0000","last_change":"2024-09-17T07:31:01.268641+0000","last_active":"2024-09-17T07:31:09.093115+0000","last_peered":"2024-09-17T07:31:09.093115+0000","last_clean":"2024-09-17T07:31:09.093115+0000","last_became_active":"2024-09-17T07:31:01.268257+0000","last_became_peered":"2024-09-17T07:31:01.268257+0000","last_unstale":"2024-09-17T07:31:09.093115+0000","last_undegraded":"2024-09-17T07:31:09.093115+0000","last_fullsized":"2024-09-17T07:31:09.093115+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:04.188817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"149'562","reported_seq":3599,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T07:31:07.095063+0000","last_change":"2024-09-17T07:31:04.098106+0000","last_active":"2024-09-17T07:31:07.095063+0000","last_peered":"2024-09-17T07:31:07.095063+0000","last_clean":"2024-09-17T07:31:07.095063+0000","last_became_active":"2024-09-17T07:30:19.087787+0000","last_became_peered":"2024-09-17T07:30:19.087787+0000","last_unstale":"2024-09-17T07:31:07.095063+0000","last_undegraded":"2024-09-17T07:31:07.095063+0000","last_fullsized":"2024-09-17T07:31:07.095063+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:32.768656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00127473,"stat_sum":{"num_bytes":4497670,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":26,"num_read":3993,"num_read_kb":36994,"num_write":611,"num_write_kb":22622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":15169480,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":296,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.097659+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:31:09.097659+0000","last_peered":"2024-09-17T07:31:09.097659+0000","last_clean":"2024-09-17T07:31:09.097659+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:31:09.097659+0000","last_undegraded":"2024-09-17T07:31:09.097659+0000","last_fullsized":"2024-09-17T07:31:09.097659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"149'1050","reported_seq":3512,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.093639+0000","last_change":"2024-09-17T07:31:09.095226+0000","last_active":"2024-09-17T07:31:09.093639+0000","last_peered":"2024-09-17T07:31:09.093639+0000","last_clean":"2024-09-17T07:31:09.093639+0000","last_became_active":"2024-09-17T07:28:43.239387+0000","last_became_peered":"2024-09-17T07:28:43.239387+0000","last_unstale":"2024-09-17T07:31:09.093639+0000","last_undegraded":"2024-09-17T07:31:09.093639+0000","last_fullsized":"2024-09-17T07:31:09.093639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1050,"log_dups_size":0,"ondisk_log_size":1050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:55.931489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020015760000000001,"stat_sum":{"num_bytes":2228224,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":23,"num_read":6807,"num_read_kb":16873,"num_write":1443,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"2.4","version":"0'0","reported_seq":119,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.108398+0000","last_change":"2024-09-17T07:30:15.200774+0000","last_active":"2024-09-17T07:31:09.108398+0000","last_peered":"2024-09-17T07:31:09.108398+0000","last_clean":"2024-09-17T07:31:09.108398+0000","last_became_active":"2024-09-17T07:30:15.200470+0000","last_became_peered":"2024-09-17T07:30:15.200470+0000","last_unstale":"2024-09-17T07:31:09.108398+0000","last_undegraded":"2024-09-17T07:31:09.108398+0000","last_fullsized":"2024-09-17T07:31:09.108398+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:58.984439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"160'550","reported_seq":3199,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.205591+0000","last_change":"2024-09-17T07:31:09.116306+0000","last_active":"2024-09-17T07:31:09.205591+0000","last_peered":"2024-09-17T07:31:09.205591+0000","last_clean":"2024-09-17T07:31:09.205591+0000","last_became_active":"2024-09-17T07:30:19.085184+0000","last_became_peered":"2024-09-17T07:30:19.085184+0000","last_unstale":"2024-09-17T07:31:09.205591+0000","last_undegraded":"2024-09-17T07:31:09.205591+0000","last_fullsized":"2024-09-17T07:31:09.205591+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:25:42.165878+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0073725539999999999,"stat_sum":{"num_bytes":1633411,"num_objects":34,"num_object_clones":3,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":25,"num_read":2671,"num_read_kb":46286,"num_write":619,"num_write_kb":27531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":35691417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.3","version":"160'633","reported_seq":2778,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.185780+0000","last_change":"2024-09-17T07:31:09.094020+0000","last_active":"2024-09-17T07:31:09.185780+0000","last_peered":"2024-09-17T07:31:09.185780+0000","last_clean":"2024-09-17T07:31:09.185780+0000","last_became_active":"2024-09-17T07:30:31.091522+0000","last_became_peered":"2024-09-17T07:30:31.091522+0000","last_unstale":"2024-09-17T07:31:09.185780+0000","last_undegraded":"2024-09-17T07:31:09.185780+0000","last_fullsized":"2024-09-17T07:31:09.185780+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:44:34.682337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072082400000000001,"stat_sum":{"num_bytes":3824433,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":29,"num_read":2208,"num_read_kb":35347,"num_write":607,"num_write_kb":16669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":26727758,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"2.2","version":"22'2","reported_seq":299,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.108350+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:31:09.108350+0000","last_peered":"2024-09-17T07:31:09.108350+0000","last_clean":"2024-09-17T07:31:09.108350+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:31:09.108350+0000","last_undegraded":"2024-09-17T07:31:09.108350+0000","last_fullsized":"2024-09-17T07:31:09.108350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"160'811","reported_seq":11250,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.099147+0000","last_change":"2024-09-17T07:31:09.099621+0000","last_active":"2024-09-17T07:31:09.099147+0000","last_peered":"2024-09-17T07:31:09.099147+0000","last_clean":"2024-09-17T07:31:09.099147+0000","last_became_active":"2024-09-17T07:28:43.239409+0000","last_became_peered":"2024-09-17T07:28:43.239409+0000","last_unstale":"2024-09-17T07:31:09.099147+0000","last_undegraded":"2024-09-17T07:31:09.099147+0000","last_fullsized":"2024-09-17T07:31:09.099147+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:41.662067+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0058246280000000001,"stat_sum":{"num_bytes":3405424,"num_objects":44,"num_object_clones":4,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":34,"num_read":12614,"num_read_kb":91705,"num_write":1234,"num_write_kb":58366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"2.1","version":"0'0","reported_seq":306,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T07:31:07.090648+0000","last_change":"2024-09-17T07:30:15.201999+0000","last_active":"2024-09-17T07:31:07.090648+0000","last_peered":"2024-09-17T07:31:07.090648+0000","last_clean":"2024-09-17T07:31:07.090648+0000","last_became_active":"2024-09-17T07:30:15.201642+0000","last_became_peered":"2024-09-17T07:30:15.201642+0000","last_unstale":"2024-09-17T07:31:07.090648+0000","last_undegraded":"2024-09-17T07:31:07.090648+0000","last_fullsized":"2024-09-17T07:31:07.090648+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:47.871565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"159'415","reported_seq":4808,"reported_epoch":159,"state":"active+undersized+remapped","last_fresh":"2024-09-17T07:31:08.323101+0000","last_change":"2024-09-17T07:31:08.323260+0000","last_active":"2024-09-17T07:31:08.323101+0000","last_peered":"2024-09-17T07:31:08.323101+0000","last_clean":"2024-09-17T07:31:05.095225+0000","last_became_active":"2024-09-17T07:31:08.105821+0000","last_became_peered":"2024-09-17T07:31:08.105821+0000","last_unstale":"2024-09-17T07:31:08.323101+0000","last_undegraded":"2024-09-17T07:31:08.323101+0000","last_fullsized":"2024-09-17T07:31:08.093971+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010734379999999999,"stat_sum":{"num_bytes":3406614,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":3788,"num_read_kb":51682,"num_write":422,"num_write_kb":28012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":37615005,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0],"avail_no_missing":["0","3","4"],"object_location_counts":[{"shards":"0","objects":27}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.0","version":"0'0","reported_seq":308,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.097293+0000","last_change":"2024-09-17T07:31:01.231085+0000","last_active":"2024-09-17T07:31:09.097293+0000","last_peered":"2024-09-17T07:31:09.097293+0000","last_clean":"2024-09-17T07:31:09.097293+0000","last_became_active":"2024-09-17T07:31:01.230794+0000","last_became_peered":"2024-09-17T07:31:01.230794+0000","last_unstale":"2024-09-17T07:31:09.097293+0000","last_undegraded":"2024-09-17T07:31:09.097293+0000","last_fullsized":"2024-09-17T07:31:09.097293+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:25.321726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"151'392","reported_seq":2394,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T07:31:07.094614+0000","last_change":"2024-09-17T07:31:04.101308+0000","last_active":"2024-09-17T07:31:07.094614+0000","last_peered":"2024-09-17T07:31:07.094614+0000","last_clean":"2024-09-17T07:31:07.094614+0000","last_became_active":"2024-09-17T07:30:18.084247+0000","last_became_peered":"2024-09-17T07:30:18.084247+0000","last_unstale":"2024-09-17T07:31:07.094614+0000","last_undegraded":"2024-09-17T07:31:07.094614+0000","last_fullsized":"2024-09-17T07:31:07.094614+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:10.521316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038868459999999998,"stat_sum":{"num_bytes":7770197,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":18,"num_read":2955,"num_read_kb":9260,"num_write":357,"num_write_kb":6317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":6805602,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.3","version":"20'1","reported_seq":83,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T07:31:08.094066+0000","last_change":"2024-09-17T07:30:34.116563+0000","last_active":"2024-09-17T07:31:08.094066+0000","last_peered":"2024-09-17T07:31:08.094066+0000","last_clean":"2024-09-17T07:31:08.094066+0000","last_became_active":"2024-09-17T07:30:34.113598+0000","last_became_peered":"2024-09-17T07:30:34.113598+0000","last_unstale":"2024-09-17T07:31:08.094066+0000","last_undegraded":"2024-09-17T07:31:08.094066+0000","last_fullsized":"2024-09-17T07:31:08.094066+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:52.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":25,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.097284+0000","last_change":"2024-09-17T07:31:05.102195+0000","last_active":"2024-09-17T07:31:09.097284+0000","last_peered":"2024-09-17T07:31:09.097284+0000","last_clean":"2024-09-17T07:31:09.097284+0000","last_became_active":"2024-09-17T07:31:05.096988+0000","last_became_peered":"2024-09-17T07:31:05.096988+0000","last_unstale":"2024-09-17T07:31:09.097284+0000","last_undegraded":"2024-09-17T07:31:09.097284+0000","last_fullsized":"2024-09-17T07:31:09.097284+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:01.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"155'509","reported_seq":2334,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T07:31:07.093459+0000","last_change":"2024-09-17T07:31:04.104378+0000","last_active":"2024-09-17T07:31:07.093459+0000","last_peered":"2024-09-17T07:31:07.093459+0000","last_clean":"2024-09-17T07:31:07.093459+0000","last_became_active":"2024-09-17T07:30:19.086777+0000","last_became_peered":"2024-09-17T07:30:19.086777+0000","last_unstale":"2024-09-17T07:31:07.093459+0000","last_undegraded":"2024-09-17T07:31:07.093459+0000","last_fullsized":"2024-09-17T07:31:07.093459+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:01:45.454678+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0078155919999999997,"stat_sum":{"num_bytes":1533740,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":1637,"num_read_kb":24381,"num_write":524,"num_write_kb":33950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":24920122,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.9","version":"160'510","reported_seq":3479,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.105690+0000","last_change":"2024-09-17T07:31:09.106055+0000","last_active":"2024-09-17T07:31:09.105690+0000","last_peered":"2024-09-17T07:31:09.105690+0000","last_clean":"2024-09-17T07:31:09.105690+0000","last_became_active":"2024-09-17T07:30:30.095518+0000","last_became_peered":"2024-09-17T07:30:30.095518+0000","last_unstale":"2024-09-17T07:31:09.105690+0000","last_undegraded":"2024-09-17T07:31:09.105690+0000","last_fullsized":"2024-09-17T07:31:09.105690+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:01.990905+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012542207999999999,"stat_sum":{"num_bytes":11287336,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":21,"num_read":4313,"num_read_kb":20899,"num_write":510,"num_write_kb":12190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":11434290,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.a","version":"160'728","reported_seq":8897,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.108386+0000","last_change":"2024-09-17T07:31:09.108386+0000","last_active":"2024-09-17T07:31:09.108386+0000","last_peered":"2024-09-17T07:31:09.108386+0000","last_clean":"2024-09-17T07:31:09.108386+0000","last_became_active":"2024-09-17T07:30:31.090321+0000","last_became_peered":"2024-09-17T07:30:31.090321+0000","last_unstale":"2024-09-17T07:31:09.108386+0000","last_undegraded":"2024-09-17T07:31:09.108386+0000","last_fullsized":"2024-09-17T07:31:09.108386+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:10.319521+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.005464076,"stat_sum":{"num_bytes":2260992,"num_objects":34,"num_object_clones":3,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":29,"num_read":9364,"num_read_kb":52919,"num_write":683,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":24254406,"num_keys_recovered":93,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.b","version":"153'547","reported_seq":2812,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T07:31:07.094746+0000","last_change":"2024-09-17T07:31:04.097086+0000","last_active":"2024-09-17T07:31:07.094746+0000","last_peered":"2024-09-17T07:31:07.094746+0000","last_clean":"2024-09-17T07:31:07.094746+0000","last_became_active":"2024-09-17T07:30:19.087515+0000","last_became_peered":"2024-09-17T07:30:19.087515+0000","last_unstale":"2024-09-17T07:31:07.094746+0000","last_undegraded":"2024-09-17T07:31:07.094746+0000","last_fullsized":"2024-09-17T07:31:07.094746+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:27:30.824856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069721599999999996,"stat_sum":{"num_bytes":2266232,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":20,"num_read":2408,"num_read_kb":29904,"num_write":665,"num_write_kb":15633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":11178824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.c","version":"149'854","reported_seq":5329,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.108887+0000","last_change":"2024-09-17T07:31:09.111198+0000","last_active":"2024-09-17T07:31:09.108887+0000","last_peered":"2024-09-17T07:31:09.108887+0000","last_clean":"2024-09-17T07:31:09.108887+0000","last_became_active":"2024-09-17T07:30:19.085668+0000","last_became_peered":"2024-09-17T07:30:19.085668+0000","last_unstale":"2024-09-17T07:31:09.108887+0000","last_undegraded":"2024-09-17T07:31:09.108887+0000","last_fullsized":"2024-09-17T07:31:09.108887+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:38.251102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026294579999999999,"stat_sum":{"num_bytes":461193,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":20,"num_read":6039,"num_read_kb":27697,"num_write":1098,"num_write_kb":14822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":15619889,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.d","version":"160'717","reported_seq":4526,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.111950+0000","last_change":"2024-09-17T07:31:09.112181+0000","last_active":"2024-09-17T07:31:09.111950+0000","last_peered":"2024-09-17T07:31:09.111950+0000","last_clean":"2024-09-17T07:31:09.111950+0000","last_became_active":"2024-09-17T07:28:43.080816+0000","last_became_peered":"2024-09-17T07:28:43.080816+0000","last_unstale":"2024-09-17T07:31:09.111950+0000","last_undegraded":"2024-09-17T07:31:09.111950+0000","last_fullsized":"2024-09-17T07:31:09.111950+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:02.677029+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014645498999999999,"stat_sum":{"num_bytes":11808030,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":15,"num_read":6005,"num_read_kb":48957,"num_write":830,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.e","version":"159'447","reported_seq":7811,"reported_epoch":159,"state":"peering","last_fresh":"2024-09-17T07:31:08.589644+0000","last_change":"2024-09-17T07:31:09.097834+0000","last_active":"2024-09-17T07:31:09.097834+0000","last_peered":"2024-09-17T07:31:08.589644+0000","last_clean":"2024-09-17T07:31:05.095539+0000","last_became_active":"2024-09-17T07:31:08.104933+0000","last_became_peered":"2024-09-17T07:31:08.104933+0000","last_unstale":"2024-09-17T07:31:08.589644+0000","last_undegraded":"2024-09-17T07:31:08.589644+0000","last_fullsized":"2024-09-17T07:31:08.093302+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0095779550000000008,"stat_sum":{"num_bytes":7907890,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":7081,"num_read_kb":55687,"num_write":389,"num_write_kb":41376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":13797349,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.f","version":"160'824","reported_seq":5736,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.112670+0000","last_change":"2024-09-17T07:31:09.112900+0000","last_active":"2024-09-17T07:31:09.112670+0000","last_peered":"2024-09-17T07:31:09.112670+0000","last_clean":"2024-09-17T07:31:09.112670+0000","last_became_active":"2024-09-17T07:31:08.095644+0000","last_became_peered":"2024-09-17T07:31:08.095644+0000","last_unstale":"2024-09-17T07:31:09.112670+0000","last_undegraded":"2024-09-17T07:31:09.112670+0000","last_fullsized":"2024-09-17T07:31:09.112670+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:15.091885+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015345062,"stat_sum":{"num_bytes":1409501,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":27,"num_read":6418,"num_read_kb":23171,"num_write":921,"num_write_kb":18541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":25525458,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.10","version":"155'394","reported_seq":2681,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.103156+0000","last_change":"2024-09-17T07:31:09.103717+0000","last_active":"2024-09-17T07:31:09.103156+0000","last_peered":"2024-09-17T07:31:09.103156+0000","last_clean":"2024-09-17T07:31:09.103156+0000","last_became_active":"2024-09-17T07:28:43.077360+0000","last_became_peered":"2024-09-17T07:28:43.077360+0000","last_unstale":"2024-09-17T07:31:09.103156+0000","last_undegraded":"2024-09-17T07:31:09.103156+0000","last_fullsized":"2024-09-17T07:31:09.103156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:49:16.285601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079397599999999995,"stat_sum":{"num_bytes":3301376,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":19,"num_read":2841,"num_read_kb":26198,"num_write":447,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.11","version":"160'782","reported_seq":3747,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.106076+0000","last_change":"2024-09-17T07:31:09.108759+0000","last_active":"2024-09-17T07:31:09.106076+0000","last_peered":"2024-09-17T07:31:09.106076+0000","last_clean":"2024-09-17T07:31:09.106076+0000","last_became_active":"2024-09-17T07:30:31.091943+0000","last_became_peered":"2024-09-17T07:30:31.091943+0000","last_unstale":"2024-09-17T07:31:09.106076+0000","last_undegraded":"2024-09-17T07:31:09.106076+0000","last_fullsized":"2024-09-17T07:31:09.106076+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:37:28.427138+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.011363589,"stat_sum":{"num_bytes":1517469,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":16,"num_read":5279,"num_read_kb":45190,"num_write":921,"num_write_kb":26416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":31676674,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.12","version":"153'381","reported_seq":2541,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T07:31:08.097984+0000","last_change":"2024-09-17T07:31:08.097984+0000","last_active":"2024-09-17T07:31:08.097984+0000","last_peered":"2024-09-17T07:31:08.097984+0000","last_clean":"2024-09-17T07:31:08.097984+0000","last_became_active":"2024-09-17T07:31:08.096240+0000","last_became_peered":"2024-09-17T07:31:08.096240+0000","last_unstale":"2024-09-17T07:31:08.097984+0000","last_undegraded":"2024-09-17T07:31:08.097984+0000","last_fullsized":"2024-09-17T07:31:08.097984+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:07.114597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084603300000000005,"stat_sum":{"num_bytes":6851680,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":30,"num_read":2063,"num_read_kb":44416,"num_write":389,"num_write_kb":29142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":24460350,"num_keys_recovered":37,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.13","version":"150'750","reported_seq":3712,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.097768+0000","last_change":"2024-09-17T07:31:09.100458+0000","last_active":"2024-09-17T07:31:09.097768+0000","last_peered":"2024-09-17T07:31:09.097768+0000","last_clean":"2024-09-17T07:31:09.097768+0000","last_became_active":"2024-09-17T07:30:32.085258+0000","last_became_peered":"2024-09-17T07:30:32.085258+0000","last_unstale":"2024-09-17T07:31:09.097768+0000","last_undegraded":"2024-09-17T07:31:09.097768+0000","last_fullsized":"2024-09-17T07:31:09.097768+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:54:43.217997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029891060000000001,"stat_sum":{"num_bytes":9961805,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":19,"num_read":4480,"num_read_kb":28100,"num_write":1060,"num_write_kb":16610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":21539718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"b"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":131559466,"num_objects":1010,"num_object_clones":156,"num_object_copies":2020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1010,"num_whiteouts":760,"num_read":164715,"num_read_kb":1207058,"num_write":22898,"num_write_kb":792694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6836,"num_bytes_recovered":666374200,"num_keys_recovered":891,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13238272,"data_stored":135051914,"data_compressed":7385723,"data_compressed_allocated":9900032,"data_compressed_original":132337664,"omap_allocated":7902,"internal_metadata":0},"log_size":19933,"ondisk_log_size":19933,"up":64,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":101032,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62844,"kb_used_data":5280,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94308996,"statfs":{"total":96636764160,"available":96572411904,"internally_reserved":0,"allocated":5406720,"data_stored":39893620,"data_compressed":2087856,"data_compressed_allocated":2785280,"data_compressed_original":38387712,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29212,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342628,"statfs":{"total":96636764160,"available":96606851072,"internally_reserved":0,"allocated":2228224,"data_stored":5068961,"data_compressed":211122,"data_compressed_allocated":299008,"data_compressed_original":4136960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542179,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57248,"kb_used_data":3848,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314592,"statfs":{"total":96636764160,"available":96578142208,"internally_reserved":0,"allocated":3940352,"data_stored":20614275,"data_compressed":1136293,"data_compressed_allocated":1511424,"data_compressed_original":19283968,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62216,"kb_used_data":4664,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94309624,"statfs":{"total":96636764160,"available":96573054976,"internally_reserved":0,"allocated":4775936,"data_stored":29943107,"data_compressed":1580575,"data_compressed_allocated":2101248,"data_compressed_original":28364800,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056655,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45332,"kb_used_data":4288,"kb_used_omap":16,"kb_used_meta":41007,"kb_avail":94326508,"statfs":{"total":96636764160,"available":96590344192,"internally_reserved":0,"allocated":4390912,"data_stored":27283226,"data_compressed":1515609,"data_compressed_allocated":2011136,"data_compressed_original":25993216,"omap_allocated":17058,"internal_metadata":41991518},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542179,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51736,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320104,"statfs":{"total":96636764160,"available":96583786496,"internally_reserved":0,"allocated":2560000,"data_stored":7281919,"data_compressed":404211,"data_compressed_allocated":503808,"data_compressed_original":6238208,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79068,"kb_used_data":3452,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292772,"statfs":{"total":96636764160,"available":96555798528,"internally_reserved":0,"allocated":3534848,"data_stored":19424410,"data_compressed":997863,"data_compressed_allocated":1323008,"data_compressed_original":18288640,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542179,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61200,"kb_used_data":3644,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310640,"statfs":{"total":96636764160,"available":96574095360,"internally_reserved":0,"allocated":3731456,"data_stored":17576821,"data_compressed":1019924,"data_compressed_allocated":1363968,"data_compressed_original":16297984,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1671168,"data_stored":14994783,"data_compressed":922040,"data_compressed_allocated":1232896,"data_compressed_original":14647296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1523712,"data_stored":17457549,"data_compressed":916461,"data_compressed_allocated":1224704,"data_compressed_original":17240064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":6369578,"data_compressed":404211,"data_compressed_allocated":503808,"data_compressed_original":6238208,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2506752,"data_stored":26376931,"data_compressed":1515609,"data_compressed_allocated":2011136,"data_compressed_original":25993216,"omap_allocated":7902,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2297856,"data_stored":21563855,"data_compressed":1153618,"data_compressed_allocated":1548288,"data_compressed_original":20910080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1556480,"data_stored":14289240,"data_compressed":766980,"data_compressed_allocated":1044480,"data_compressed_original":13877248,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":3102119,"data_compressed":161694,"data_compressed_allocated":233472,"data_compressed_original":3088384,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":30897859,"data_compressed":1545110,"data_compressed_allocated":2101248,"data_compressed_original":30343168,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:31:10.988 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T07:31:11.073 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:31:11.071+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:31:11.133 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:11.133 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:11.142 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:31:11.185 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-17T07:31:11.184+0000 7fac0dc73640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33356) UID: 0 2024-09-17T07:31:11.252 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:11.252 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:11.261 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:31:11.265 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:11.264+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:11.366 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:11.365+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:11.405 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:11.405 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:11.408 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:11.408 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"b283e6d5-0f4a-4271-a443-15e695e35364","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":179,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":162,"num_osds":8,"num_up_osds":8,"osd_up_since":1726558198,"num_in_osds":8,"osd_in_since":1726558226,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":39},{"state_name":"active+undersized+remapped","count":1},{"state_name":"peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":1014,"data_bytes":132149853,"bytes_used":459628544,"bytes_avail":772634484736,"bytes_total":773094113280,"inactive_pgs_ratio":0.024390242993831635,"recovering_objects_per_sec":322,"recovering_bytes_per_sec":23215943,"recovering_keys_per_sec":19,"num_objects_recovered":1290,"num_bytes_recovered":92879724,"num_keys_recovered":80,"read_bytes_sec":14259310,"write_bytes_sec":10578150,"read_op_per_sec":2554,"write_op_per_sec":331},"fsmap":{"epoch":1,"btime":"2024-09-17T07:28:12:268963+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-17T07:30:58.779932+0000","services":{}},"progress_events":{}} 2024-09-17T07:31:11.413 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:31:11.422 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-17T07:31:11.422 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:31:11.497 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:11.497+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:11.504 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T07:31:11.556 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:11.556 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:11.565 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:31:11.567 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:11.566+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:11.632 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:31:11.632 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:31:11.649 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:31:11.650 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:31:11.666 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:31:11.666 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:31:11.668 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:11.668+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:11.682 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:31:11.683 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:31:11.699 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:31:11.699 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:31:11.708 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:11.708 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:11.716 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:31:11.716 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:31:11.717 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:31:11.732 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:31:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:31:11.749 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:31:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:31:11.766 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:11.766 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:31:11.769 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-09-17T07:31:11.768+0000 7fef8bf4a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 48543) UID: 0 2024-09-17T07:31:11.778 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-17T07:31:11.093550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":132149853,"num_objects":1014,"num_object_clones":156,"num_object_copies":2028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1014,"num_whiteouts":760,"num_read":164781,"num_read_kb":1207113,"num_write":23001,"num_write_kb":794324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6841,"num_bytes_recovered":667554936,"num_keys_recovered":891,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20008,"ondisk_log_size":20008,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":448856,"kb_used_data":29852,"kb_used_omap":27,"kb_used_meta":418788,"kb_avail":754525864,"statfs":{"total":773094113280,"available":772634484736,"internally_reserved":0,"allocated":30568448,"data_stored":167086339,"data_compressed":8953453,"data_compressed_allocated":11898880,"data_compressed_original":156991488,"omap_allocated":28217,"internal_metadata":428839367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-46741829,"num_objects":26,"num_object_clones":-3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":-2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":35,"num_read":10218,"num_read_kb":55710,"num_write":1327,"num_write_kb":41328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1290,"num_bytes_recovered":92879724,"num_keys_recovered":80,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.004938"},"pg_stats":[{"pgid":"3.1f","version":"159'849","reported_seq":4505,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T07:31:08.150856+0000","last_change":"2024-09-17T07:31:04.100988+0000","last_active":"2024-09-17T07:31:08.150856+0000","last_peered":"2024-09-17T07:31:08.150856+0000","last_clean":"2024-09-17T07:31:08.150856+0000","last_became_active":"2024-09-17T07:30:43.092456+0000","last_became_peered":"2024-09-17T07:30:43.092456+0000","last_unstale":"2024-09-17T07:31:08.150856+0000","last_undegraded":"2024-09-17T07:31:08.150856+0000","last_fullsized":"2024-09-17T07:31:08.150856+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":849,"log_dups_size":0,"ondisk_log_size":849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:12.174954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012895070000000001,"stat_sum":{"num_bytes":4706440,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":25,"num_read":5785,"num_read_kb":34299,"num_write":986,"num_write_kb":41977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":26159662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.1e","version":"152'516","reported_seq":3091,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T07:31:07.095063+0000","last_change":"2024-09-17T07:31:04.102043+0000","last_active":"2024-09-17T07:31:07.095063+0000","last_peered":"2024-09-17T07:31:07.095063+0000","last_clean":"2024-09-17T07:31:07.095063+0000","last_became_active":"2024-09-17T07:30:19.087213+0000","last_became_peered":"2024-09-17T07:30:19.087213+0000","last_unstale":"2024-09-17T07:31:07.095063+0000","last_undegraded":"2024-09-17T07:31:07.095063+0000","last_fullsized":"2024-09-17T07:31:07.095063+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:54.591164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046235499999999997,"stat_sum":{"num_bytes":3382661,"num_objects":49,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":40,"num_read":3553,"num_read_kb":30152,"num_write":508,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":15743718,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.1d","version":"160'536","reported_seq":2892,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.113643+0000","last_change":"2024-09-17T07:31:09.113841+0000","last_active":"2024-09-17T07:31:09.113643+0000","last_peered":"2024-09-17T07:31:09.113643+0000","last_clean":"2024-09-17T07:31:09.113643+0000","last_became_active":"2024-09-17T07:30:30.096044+0000","last_became_peered":"2024-09-17T07:30:30.096044+0000","last_unstale":"2024-09-17T07:31:09.113643+0000","last_undegraded":"2024-09-17T07:31:09.113643+0000","last_fullsized":"2024-09-17T07:31:09.113643+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:41:28.043401+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0051523380000000002,"stat_sum":{"num_bytes":2068938,"num_objects":42,"num_object_clones":6,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":34,"num_read":1948,"num_read_kb":28759,"num_write":577,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":27967854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.1c","version":"160'793","reported_seq":6089,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.136905+0000","last_change":"2024-09-17T07:31:09.115906+0000","last_active":"2024-09-17T07:31:09.136905+0000","last_peered":"2024-09-17T07:31:09.136905+0000","last_clean":"2024-09-17T07:31:09.136905+0000","last_became_active":"2024-09-17T07:30:31.098495+0000","last_became_peered":"2024-09-17T07:30:31.098495+0000","last_unstale":"2024-09-17T07:31:09.136905+0000","last_undegraded":"2024-09-17T07:31:09.136905+0000","last_fullsized":"2024-09-17T07:31:09.136905+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:26.465626+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0070809139999999998,"stat_sum":{"num_bytes":1392670,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":15,"num_read":7476,"num_read_kb":42412,"num_write":1041,"num_write_kb":38721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":26661650,"num_keys_recovered":180,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.1b","version":"154'549","reported_seq":4826,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T07:31:08.094106+0000","last_change":"2024-09-17T07:31:04.099221+0000","last_active":"2024-09-17T07:31:08.094106+0000","last_peered":"2024-09-17T07:31:08.094106+0000","last_clean":"2024-09-17T07:31:08.094106+0000","last_became_active":"2024-09-17T07:31:03.094058+0000","last_became_peered":"2024-09-17T07:31:03.094058+0000","last_unstale":"2024-09-17T07:31:08.094106+0000","last_undegraded":"2024-09-17T07:31:08.094106+0000","last_fullsized":"2024-09-17T07:31:08.094106+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:55.445792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072087799999999999,"stat_sum":{"num_bytes":2644152,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":19,"num_read":5413,"num_read_kb":52124,"num_write":561,"num_write_kb":22867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":674,"num_bytes_recovered":91784745,"num_keys_recovered":126,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.1a","version":"160'909","reported_seq":14535,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.192814+0000","last_change":"2024-09-17T07:31:09.100208+0000","last_active":"2024-09-17T07:31:09.192814+0000","last_peered":"2024-09-17T07:31:09.192814+0000","last_clean":"2024-09-17T07:31:09.192814+0000","last_became_active":"2024-09-17T07:30:31.090258+0000","last_became_peered":"2024-09-17T07:30:31.090258+0000","last_unstale":"2024-09-17T07:31:09.192814+0000","last_undegraded":"2024-09-17T07:31:09.192814+0000","last_fullsized":"2024-09-17T07:31:09.192814+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:40.042064+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0069821989999999997,"stat_sum":{"num_bytes":1673332,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":23,"num_read":16699,"num_read_kb":60966,"num_write":1187,"num_write_kb":20597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":19561990,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.19","version":"154'432","reported_seq":5598,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.094297+0000","last_change":"2024-09-17T07:31:09.094297+0000","last_active":"2024-09-17T07:31:09.094297+0000","last_peered":"2024-09-17T07:31:09.094297+0000","last_clean":"2024-09-17T07:31:09.094297+0000","last_became_active":"2024-09-17T07:30:18.083918+0000","last_became_peered":"2024-09-17T07:30:18.083918+0000","last_unstale":"2024-09-17T07:31:09.094297+0000","last_undegraded":"2024-09-17T07:31:09.094297+0000","last_fullsized":"2024-09-17T07:31:09.094297+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:30.795752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086112700000000005,"stat_sum":{"num_bytes":1916928,"num_objects":35,"num_object_clones":2,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":30,"num_read":4352,"num_read_kb":37354,"num_write":397,"num_write_kb":43089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":25536146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.18","version":"154'755","reported_seq":4574,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T07:31:07.093343+0000","last_change":"2024-09-17T07:31:04.097094+0000","last_active":"2024-09-17T07:31:07.093343+0000","last_peered":"2024-09-17T07:31:07.093343+0000","last_clean":"2024-09-17T07:31:07.093343+0000","last_became_active":"2024-09-17T07:30:43.091692+0000","last_became_peered":"2024-09-17T07:30:43.091692+0000","last_unstale":"2024-09-17T07:31:07.093343+0000","last_undegraded":"2024-09-17T07:31:07.093343+0000","last_fullsized":"2024-09-17T07:31:07.093343+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:17.851207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085812200000000001,"stat_sum":{"num_bytes":3145728,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":23,"num_read":5969,"num_read_kb":25327,"num_write":792,"num_write_kb":16603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":29456184,"num_keys_recovered":176,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.17","version":"153'427","reported_seq":4133,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T07:31:08.094494+0000","last_change":"2024-09-17T07:31:04.101208+0000","last_active":"2024-09-17T07:31:08.094494+0000","last_peered":"2024-09-17T07:31:08.094494+0000","last_clean":"2024-09-17T07:31:08.094494+0000","last_became_active":"2024-09-17T07:28:43.080428+0000","last_became_peered":"2024-09-17T07:28:43.080428+0000","last_unstale":"2024-09-17T07:31:08.094494+0000","last_undegraded":"2024-09-17T07:31:08.094494+0000","last_fullsized":"2024-09-17T07:31:08.094494+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:49.410936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001240063,"stat_sum":{"num_bytes":2056192,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":29,"num_read":4961,"num_read_kb":22717,"num_write":364,"num_write_kb":13531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.16","version":"157'484","reported_seq":4010,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.111149+0000","last_change":"2024-09-17T07:31:09.111149+0000","last_active":"2024-09-17T07:31:09.111149+0000","last_peered":"2024-09-17T07:31:09.111149+0000","last_clean":"2024-09-17T07:31:09.111149+0000","last_became_active":"2024-09-17T07:28:43.082313+0000","last_became_peered":"2024-09-17T07:28:43.082313+0000","last_unstale":"2024-09-17T07:31:09.111149+0000","last_undegraded":"2024-09-17T07:31:09.111149+0000","last_fullsized":"2024-09-17T07:31:09.111149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:23.909158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025815529999999999,"stat_sum":{"num_bytes":1703978,"num_objects":26,"num_object_clones":2,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":20,"num_read":5970,"num_read_kb":58212,"num_write":429,"num_write_kb":31849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.15","version":"152'373","reported_seq":4411,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.100516+0000","last_change":"2024-09-17T07:31:09.102318+0000","last_active":"2024-09-17T07:31:09.100516+0000","last_peered":"2024-09-17T07:31:09.100516+0000","last_clean":"2024-09-17T07:31:09.100516+0000","last_became_active":"2024-09-17T07:31:08.094940+0000","last_became_peered":"2024-09-17T07:31:08.094940+0000","last_unstale":"2024-09-17T07:31:09.100516+0000","last_undegraded":"2024-09-17T07:31:09.100516+0000","last_fullsized":"2024-09-17T07:31:09.100516+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:32.821352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048355430000000003,"stat_sum":{"num_bytes":11730944,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":21,"num_read":3586,"num_read_kb":32352,"num_write":368,"num_write_kb":22781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":431,"num_bytes_recovered":60567103,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.14","version":"160'452","reported_seq":2035,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.208351+0000","last_change":"2024-09-17T07:31:09.100477+0000","last_active":"2024-09-17T07:31:09.208351+0000","last_peered":"2024-09-17T07:31:09.208351+0000","last_clean":"2024-09-17T07:31:09.208351+0000","last_became_active":"2024-09-17T07:30:30.095364+0000","last_became_peered":"2024-09-17T07:30:30.095364+0000","last_unstale":"2024-09-17T07:31:09.208351+0000","last_undegraded":"2024-09-17T07:31:09.208351+0000","last_fullsized":"2024-09-17T07:31:09.208351+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:24.023571+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0070866380000000001,"stat_sum":{"num_bytes":1677602,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":22,"num_read":1414,"num_read_kb":17974,"num_write":481,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":16518806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"2.7","version":"0'0","reported_seq":307,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.102866+0000","last_change":"2024-09-17T07:31:05.098106+0000","last_active":"2024-09-17T07:31:09.102866+0000","last_peered":"2024-09-17T07:31:09.102866+0000","last_clean":"2024-09-17T07:31:09.102866+0000","last_became_active":"2024-09-17T07:31:05.097821+0000","last_became_peered":"2024-09-17T07:31:05.097821+0000","last_unstale":"2024-09-17T07:31:09.102866+0000","last_undegraded":"2024-09-17T07:31:09.102866+0000","last_fullsized":"2024-09-17T07:31:09.102866+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:12.297890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"159'1002","reported_seq":2651,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T07:31:08.094340+0000","last_change":"2024-09-17T07:31:04.100390+0000","last_active":"2024-09-17T07:31:08.094340+0000","last_peered":"2024-09-17T07:31:08.094340+0000","last_clean":"2024-09-17T07:31:08.094340+0000","last_became_active":"2024-09-17T07:28:43.079229+0000","last_became_peered":"2024-09-17T07:28:43.079229+0000","last_unstale":"2024-09-17T07:31:08.094340+0000","last_undegraded":"2024-09-17T07:31:08.094340+0000","last_fullsized":"2024-09-17T07:31:08.094340+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1002,"log_dups_size":0,"ondisk_log_size":1002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:32.510405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002047138,"stat_sum":{"num_bytes":6126684,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":28,"num_read":4625,"num_read_kb":48744,"num_write":1477,"num_write_kb":29297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.093115+0000","last_change":"2024-09-17T07:31:01.268641+0000","last_active":"2024-09-17T07:31:09.093115+0000","last_peered":"2024-09-17T07:31:09.093115+0000","last_clean":"2024-09-17T07:31:09.093115+0000","last_became_active":"2024-09-17T07:31:01.268257+0000","last_became_peered":"2024-09-17T07:31:01.268257+0000","last_unstale":"2024-09-17T07:31:09.093115+0000","last_undegraded":"2024-09-17T07:31:09.093115+0000","last_fullsized":"2024-09-17T07:31:09.093115+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:04.188817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"149'562","reported_seq":3599,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T07:31:07.095063+0000","last_change":"2024-09-17T07:31:04.098106+0000","last_active":"2024-09-17T07:31:07.095063+0000","last_peered":"2024-09-17T07:31:07.095063+0000","last_clean":"2024-09-17T07:31:07.095063+0000","last_became_active":"2024-09-17T07:30:19.087787+0000","last_became_peered":"2024-09-17T07:30:19.087787+0000","last_unstale":"2024-09-17T07:31:07.095063+0000","last_undegraded":"2024-09-17T07:31:07.095063+0000","last_fullsized":"2024-09-17T07:31:07.095063+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:32.768656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00127473,"stat_sum":{"num_bytes":4497670,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":26,"num_read":3993,"num_read_kb":36994,"num_write":611,"num_write_kb":22622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":15169480,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":296,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.097659+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:31:09.097659+0000","last_peered":"2024-09-17T07:31:09.097659+0000","last_clean":"2024-09-17T07:31:09.097659+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:31:09.097659+0000","last_undegraded":"2024-09-17T07:31:09.097659+0000","last_fullsized":"2024-09-17T07:31:09.097659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"149'1050","reported_seq":3512,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.093639+0000","last_change":"2024-09-17T07:31:09.095226+0000","last_active":"2024-09-17T07:31:09.093639+0000","last_peered":"2024-09-17T07:31:09.093639+0000","last_clean":"2024-09-17T07:31:09.093639+0000","last_became_active":"2024-09-17T07:28:43.239387+0000","last_became_peered":"2024-09-17T07:28:43.239387+0000","last_unstale":"2024-09-17T07:31:09.093639+0000","last_undegraded":"2024-09-17T07:31:09.093639+0000","last_fullsized":"2024-09-17T07:31:09.093639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1050,"log_dups_size":0,"ondisk_log_size":1050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:55.931489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020015760000000001,"stat_sum":{"num_bytes":2228224,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":23,"num_read":6807,"num_read_kb":16873,"num_write":1443,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"2.4","version":"0'0","reported_seq":119,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.108398+0000","last_change":"2024-09-17T07:30:15.200774+0000","last_active":"2024-09-17T07:31:09.108398+0000","last_peered":"2024-09-17T07:31:09.108398+0000","last_clean":"2024-09-17T07:31:09.108398+0000","last_became_active":"2024-09-17T07:30:15.200470+0000","last_became_peered":"2024-09-17T07:30:15.200470+0000","last_unstale":"2024-09-17T07:31:09.108398+0000","last_undegraded":"2024-09-17T07:31:09.108398+0000","last_fullsized":"2024-09-17T07:31:09.108398+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:58.984439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"160'550","reported_seq":3199,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.205591+0000","last_change":"2024-09-17T07:31:09.116306+0000","last_active":"2024-09-17T07:31:09.205591+0000","last_peered":"2024-09-17T07:31:09.205591+0000","last_clean":"2024-09-17T07:31:09.205591+0000","last_became_active":"2024-09-17T07:30:19.085184+0000","last_became_peered":"2024-09-17T07:30:19.085184+0000","last_unstale":"2024-09-17T07:31:09.205591+0000","last_undegraded":"2024-09-17T07:31:09.205591+0000","last_fullsized":"2024-09-17T07:31:09.205591+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:25:42.165878+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0073725539999999999,"stat_sum":{"num_bytes":1633411,"num_objects":34,"num_object_clones":3,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":25,"num_read":2671,"num_read_kb":46286,"num_write":619,"num_write_kb":27531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":35691417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.3","version":"160'633","reported_seq":2778,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.185780+0000","last_change":"2024-09-17T07:31:09.094020+0000","last_active":"2024-09-17T07:31:09.185780+0000","last_peered":"2024-09-17T07:31:09.185780+0000","last_clean":"2024-09-17T07:31:09.185780+0000","last_became_active":"2024-09-17T07:30:31.091522+0000","last_became_peered":"2024-09-17T07:30:31.091522+0000","last_unstale":"2024-09-17T07:31:09.185780+0000","last_undegraded":"2024-09-17T07:31:09.185780+0000","last_fullsized":"2024-09-17T07:31:09.185780+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:44:34.682337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072082400000000001,"stat_sum":{"num_bytes":3824433,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":29,"num_read":2208,"num_read_kb":35347,"num_write":607,"num_write_kb":16669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":26727758,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"2.2","version":"22'2","reported_seq":299,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.108350+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:31:09.108350+0000","last_peered":"2024-09-17T07:31:09.108350+0000","last_clean":"2024-09-17T07:31:09.108350+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:31:09.108350+0000","last_undegraded":"2024-09-17T07:31:09.108350+0000","last_fullsized":"2024-09-17T07:31:09.108350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"160'811","reported_seq":11250,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.099147+0000","last_change":"2024-09-17T07:31:09.099621+0000","last_active":"2024-09-17T07:31:09.099147+0000","last_peered":"2024-09-17T07:31:09.099147+0000","last_clean":"2024-09-17T07:31:09.099147+0000","last_became_active":"2024-09-17T07:28:43.239409+0000","last_became_peered":"2024-09-17T07:28:43.239409+0000","last_unstale":"2024-09-17T07:31:09.099147+0000","last_undegraded":"2024-09-17T07:31:09.099147+0000","last_fullsized":"2024-09-17T07:31:09.099147+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:41.662067+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0058246280000000001,"stat_sum":{"num_bytes":3405424,"num_objects":44,"num_object_clones":4,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":34,"num_read":12614,"num_read_kb":91705,"num_write":1234,"num_write_kb":58366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"2.1","version":"0'0","reported_seq":306,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T07:31:07.090648+0000","last_change":"2024-09-17T07:30:15.201999+0000","last_active":"2024-09-17T07:31:07.090648+0000","last_peered":"2024-09-17T07:31:07.090648+0000","last_clean":"2024-09-17T07:31:07.090648+0000","last_became_active":"2024-09-17T07:30:15.201642+0000","last_became_peered":"2024-09-17T07:30:15.201642+0000","last_unstale":"2024-09-17T07:31:07.090648+0000","last_undegraded":"2024-09-17T07:31:07.090648+0000","last_fullsized":"2024-09-17T07:31:07.090648+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:47.871565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"159'415","reported_seq":4808,"reported_epoch":159,"state":"active+undersized+remapped","last_fresh":"2024-09-17T07:31:08.323101+0000","last_change":"2024-09-17T07:31:08.323260+0000","last_active":"2024-09-17T07:31:08.323101+0000","last_peered":"2024-09-17T07:31:08.323101+0000","last_clean":"2024-09-17T07:31:05.095225+0000","last_became_active":"2024-09-17T07:31:08.105821+0000","last_became_peered":"2024-09-17T07:31:08.105821+0000","last_unstale":"2024-09-17T07:31:08.323101+0000","last_undegraded":"2024-09-17T07:31:08.323101+0000","last_fullsized":"2024-09-17T07:31:08.093971+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010734379999999999,"stat_sum":{"num_bytes":3406614,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":3788,"num_read_kb":51682,"num_write":422,"num_write_kb":28012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":37615005,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0],"avail_no_missing":["0","3","4"],"object_location_counts":[{"shards":"0","objects":27}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.0","version":"0'0","reported_seq":308,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.097293+0000","last_change":"2024-09-17T07:31:01.231085+0000","last_active":"2024-09-17T07:31:09.097293+0000","last_peered":"2024-09-17T07:31:09.097293+0000","last_clean":"2024-09-17T07:31:09.097293+0000","last_became_active":"2024-09-17T07:31:01.230794+0000","last_became_peered":"2024-09-17T07:31:01.230794+0000","last_unstale":"2024-09-17T07:31:09.097293+0000","last_undegraded":"2024-09-17T07:31:09.097293+0000","last_fullsized":"2024-09-17T07:31:09.097293+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:25.321726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"151'392","reported_seq":2394,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T07:31:07.094614+0000","last_change":"2024-09-17T07:31:04.101308+0000","last_active":"2024-09-17T07:31:07.094614+0000","last_peered":"2024-09-17T07:31:07.094614+0000","last_clean":"2024-09-17T07:31:07.094614+0000","last_became_active":"2024-09-17T07:30:18.084247+0000","last_became_peered":"2024-09-17T07:30:18.084247+0000","last_unstale":"2024-09-17T07:31:07.094614+0000","last_undegraded":"2024-09-17T07:31:07.094614+0000","last_fullsized":"2024-09-17T07:31:07.094614+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:19:10.521316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038868459999999998,"stat_sum":{"num_bytes":7770197,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":18,"num_read":2955,"num_read_kb":9260,"num_write":357,"num_write_kb":6317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":6805602,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"2.3","version":"20'1","reported_seq":83,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T07:31:08.094066+0000","last_change":"2024-09-17T07:30:34.116563+0000","last_active":"2024-09-17T07:31:08.094066+0000","last_peered":"2024-09-17T07:31:08.094066+0000","last_clean":"2024-09-17T07:31:08.094066+0000","last_became_active":"2024-09-17T07:30:34.113598+0000","last_became_peered":"2024-09-17T07:30:34.113598+0000","last_unstale":"2024-09-17T07:31:08.094066+0000","last_undegraded":"2024-09-17T07:31:08.094066+0000","last_fullsized":"2024-09-17T07:31:08.094066+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:52.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":25,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.097284+0000","last_change":"2024-09-17T07:31:05.102195+0000","last_active":"2024-09-17T07:31:09.097284+0000","last_peered":"2024-09-17T07:31:09.097284+0000","last_clean":"2024-09-17T07:31:09.097284+0000","last_became_active":"2024-09-17T07:31:05.096988+0000","last_became_peered":"2024-09-17T07:31:05.096988+0000","last_unstale":"2024-09-17T07:31:09.097284+0000","last_undegraded":"2024-09-17T07:31:09.097284+0000","last_fullsized":"2024-09-17T07:31:09.097284+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:01.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"155'509","reported_seq":2334,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T07:31:07.093459+0000","last_change":"2024-09-17T07:31:04.104378+0000","last_active":"2024-09-17T07:31:07.093459+0000","last_peered":"2024-09-17T07:31:07.093459+0000","last_clean":"2024-09-17T07:31:07.093459+0000","last_became_active":"2024-09-17T07:30:19.086777+0000","last_became_peered":"2024-09-17T07:30:19.086777+0000","last_unstale":"2024-09-17T07:31:07.093459+0000","last_undegraded":"2024-09-17T07:31:07.093459+0000","last_fullsized":"2024-09-17T07:31:07.093459+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:01:45.454678+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0078155919999999997,"stat_sum":{"num_bytes":1533740,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":1637,"num_read_kb":24381,"num_write":524,"num_write_kb":33950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":24920122,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.9","version":"160'510","reported_seq":3479,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.105690+0000","last_change":"2024-09-17T07:31:09.106055+0000","last_active":"2024-09-17T07:31:09.105690+0000","last_peered":"2024-09-17T07:31:09.105690+0000","last_clean":"2024-09-17T07:31:09.105690+0000","last_became_active":"2024-09-17T07:30:30.095518+0000","last_became_peered":"2024-09-17T07:30:30.095518+0000","last_unstale":"2024-09-17T07:31:09.105690+0000","last_undegraded":"2024-09-17T07:31:09.105690+0000","last_fullsized":"2024-09-17T07:31:09.105690+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:01.990905+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012542207999999999,"stat_sum":{"num_bytes":11287336,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":21,"num_read":4313,"num_read_kb":20899,"num_write":510,"num_write_kb":12190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":11434290,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.a","version":"160'728","reported_seq":8897,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.108386+0000","last_change":"2024-09-17T07:31:09.108386+0000","last_active":"2024-09-17T07:31:09.108386+0000","last_peered":"2024-09-17T07:31:09.108386+0000","last_clean":"2024-09-17T07:31:09.108386+0000","last_became_active":"2024-09-17T07:30:31.090321+0000","last_became_peered":"2024-09-17T07:30:31.090321+0000","last_unstale":"2024-09-17T07:31:09.108386+0000","last_undegraded":"2024-09-17T07:31:09.108386+0000","last_fullsized":"2024-09-17T07:31:09.108386+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:10.319521+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.005464076,"stat_sum":{"num_bytes":2260992,"num_objects":34,"num_object_clones":3,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":29,"num_read":9364,"num_read_kb":52919,"num_write":683,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":24254406,"num_keys_recovered":93,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.b","version":"153'547","reported_seq":2812,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-17T07:31:07.094746+0000","last_change":"2024-09-17T07:31:04.097086+0000","last_active":"2024-09-17T07:31:07.094746+0000","last_peered":"2024-09-17T07:31:07.094746+0000","last_clean":"2024-09-17T07:31:07.094746+0000","last_became_active":"2024-09-17T07:30:19.087515+0000","last_became_peered":"2024-09-17T07:30:19.087515+0000","last_unstale":"2024-09-17T07:31:07.094746+0000","last_undegraded":"2024-09-17T07:31:07.094746+0000","last_fullsized":"2024-09-17T07:31:07.094746+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:27:30.824856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069721599999999996,"stat_sum":{"num_bytes":2266232,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":20,"num_read":2408,"num_read_kb":29904,"num_write":665,"num_write_kb":15633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":11178824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.c","version":"149'854","reported_seq":5329,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.108887+0000","last_change":"2024-09-17T07:31:09.111198+0000","last_active":"2024-09-17T07:31:09.108887+0000","last_peered":"2024-09-17T07:31:09.108887+0000","last_clean":"2024-09-17T07:31:09.108887+0000","last_became_active":"2024-09-17T07:30:19.085668+0000","last_became_peered":"2024-09-17T07:30:19.085668+0000","last_unstale":"2024-09-17T07:31:09.108887+0000","last_undegraded":"2024-09-17T07:31:09.108887+0000","last_fullsized":"2024-09-17T07:31:09.108887+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:38.251102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026294579999999999,"stat_sum":{"num_bytes":461193,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":20,"num_read":6039,"num_read_kb":27697,"num_write":1098,"num_write_kb":14822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":15619889,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.d","version":"160'717","reported_seq":4526,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.111950+0000","last_change":"2024-09-17T07:31:09.112181+0000","last_active":"2024-09-17T07:31:09.111950+0000","last_peered":"2024-09-17T07:31:09.111950+0000","last_clean":"2024-09-17T07:31:09.111950+0000","last_became_active":"2024-09-17T07:28:43.080816+0000","last_became_peered":"2024-09-17T07:28:43.080816+0000","last_unstale":"2024-09-17T07:31:09.111950+0000","last_undegraded":"2024-09-17T07:31:09.111950+0000","last_fullsized":"2024-09-17T07:31:09.111950+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:02.677029+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014645498999999999,"stat_sum":{"num_bytes":11808030,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":15,"num_read":6005,"num_read_kb":48957,"num_write":830,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.e","version":"159'447","reported_seq":7811,"reported_epoch":159,"state":"peering","last_fresh":"2024-09-17T07:31:08.589644+0000","last_change":"2024-09-17T07:31:09.097834+0000","last_active":"2024-09-17T07:31:09.097834+0000","last_peered":"2024-09-17T07:31:08.589644+0000","last_clean":"2024-09-17T07:31:05.095539+0000","last_became_active":"2024-09-17T07:31:08.104933+0000","last_became_peered":"2024-09-17T07:31:08.104933+0000","last_unstale":"2024-09-17T07:31:08.589644+0000","last_undegraded":"2024-09-17T07:31:08.589644+0000","last_fullsized":"2024-09-17T07:31:08.093302+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0095779550000000008,"stat_sum":{"num_bytes":7907890,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":7081,"num_read_kb":55687,"num_write":389,"num_write_kb":41376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":13797349,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.f","version":"160'824","reported_seq":5736,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.112670+0000","last_change":"2024-09-17T07:31:09.112900+0000","last_active":"2024-09-17T07:31:09.112670+0000","last_peered":"2024-09-17T07:31:09.112670+0000","last_clean":"2024-09-17T07:31:09.112670+0000","last_became_active":"2024-09-17T07:31:08.095644+0000","last_became_peered":"2024-09-17T07:31:08.095644+0000","last_unstale":"2024-09-17T07:31:09.112670+0000","last_undegraded":"2024-09-17T07:31:09.112670+0000","last_fullsized":"2024-09-17T07:31:09.112670+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:15.091885+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015345062,"stat_sum":{"num_bytes":1409501,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":27,"num_read":6418,"num_read_kb":23171,"num_write":921,"num_write_kb":18541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":25525458,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.10","version":"155'394","reported_seq":2681,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.103156+0000","last_change":"2024-09-17T07:31:09.103717+0000","last_active":"2024-09-17T07:31:09.103156+0000","last_peered":"2024-09-17T07:31:09.103156+0000","last_clean":"2024-09-17T07:31:09.103156+0000","last_became_active":"2024-09-17T07:28:43.077360+0000","last_became_peered":"2024-09-17T07:28:43.077360+0000","last_unstale":"2024-09-17T07:31:09.103156+0000","last_undegraded":"2024-09-17T07:31:09.103156+0000","last_fullsized":"2024-09-17T07:31:09.103156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:49:16.285601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079397599999999995,"stat_sum":{"num_bytes":3301376,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":19,"num_read":2841,"num_read_kb":26198,"num_write":447,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.11","version":"160'782","reported_seq":3747,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.106076+0000","last_change":"2024-09-17T07:31:09.108759+0000","last_active":"2024-09-17T07:31:09.106076+0000","last_peered":"2024-09-17T07:31:09.106076+0000","last_clean":"2024-09-17T07:31:09.106076+0000","last_became_active":"2024-09-17T07:30:31.091943+0000","last_became_peered":"2024-09-17T07:30:31.091943+0000","last_unstale":"2024-09-17T07:31:09.106076+0000","last_undegraded":"2024-09-17T07:31:09.106076+0000","last_fullsized":"2024-09-17T07:31:09.106076+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:37:28.427138+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.011363589,"stat_sum":{"num_bytes":1517469,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":16,"num_read":5279,"num_read_kb":45190,"num_write":921,"num_write_kb":26416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":31676674,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"b"}]},{"pgid":"3.12","version":"153'381","reported_seq":2541,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-09-17T07:31:08.097984+0000","last_change":"2024-09-17T07:31:08.097984+0000","last_active":"2024-09-17T07:31:08.097984+0000","last_peered":"2024-09-17T07:31:08.097984+0000","last_clean":"2024-09-17T07:31:08.097984+0000","last_became_active":"2024-09-17T07:31:08.096240+0000","last_became_peered":"2024-09-17T07:31:08.096240+0000","last_unstale":"2024-09-17T07:31:08.097984+0000","last_undegraded":"2024-09-17T07:31:08.097984+0000","last_fullsized":"2024-09-17T07:31:08.097984+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:07.114597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084603300000000005,"stat_sum":{"num_bytes":6851680,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":30,"num_read":2063,"num_read_kb":44416,"num_write":389,"num_write_kb":29142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":24460350,"num_keys_recovered":37,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"a"}]},{"pgid":"3.13","version":"150'750","reported_seq":3712,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-17T07:31:09.097768+0000","last_change":"2024-09-17T07:31:09.100458+0000","last_active":"2024-09-17T07:31:09.097768+0000","last_peered":"2024-09-17T07:31:09.097768+0000","last_clean":"2024-09-17T07:31:09.097768+0000","last_became_active":"2024-09-17T07:30:32.085258+0000","last_became_peered":"2024-09-17T07:30:32.085258+0000","last_unstale":"2024-09-17T07:31:09.097768+0000","last_undegraded":"2024-09-17T07:31:09.097768+0000","last_fullsized":"2024-09-17T07:31:09.097768+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:54:43.217997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029891060000000001,"stat_sum":{"num_bytes":9961805,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":19,"num_read":4480,"num_read_kb":28100,"num_write":1060,"num_write_kb":16610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":21539718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"b"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":131559466,"num_objects":1010,"num_object_clones":156,"num_object_copies":2020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1010,"num_whiteouts":760,"num_read":164715,"num_read_kb":1207058,"num_write":22898,"num_write_kb":792694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6836,"num_bytes_recovered":666374200,"num_keys_recovered":891,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13238272,"data_stored":135051914,"data_compressed":7385723,"data_compressed_allocated":9900032,"data_compressed_original":132337664,"omap_allocated":7902,"internal_metadata":0},"log_size":19933,"ondisk_log_size":19933,"up":64,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":101032,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62844,"kb_used_data":5280,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94308996,"statfs":{"total":96636764160,"available":96572411904,"internally_reserved":0,"allocated":5406720,"data_stored":39893620,"data_compressed":2087856,"data_compressed_allocated":2785280,"data_compressed_original":38387712,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29212,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342628,"statfs":{"total":96636764160,"available":96606851072,"internally_reserved":0,"allocated":2228224,"data_stored":5068961,"data_compressed":211122,"data_compressed_allocated":299008,"data_compressed_original":4136960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542179,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57248,"kb_used_data":3848,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314592,"statfs":{"total":96636764160,"available":96578142208,"internally_reserved":0,"allocated":3940352,"data_stored":20614275,"data_compressed":1136293,"data_compressed_allocated":1511424,"data_compressed_original":19283968,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62216,"kb_used_data":4664,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94309624,"statfs":{"total":96636764160,"available":96573054976,"internally_reserved":0,"allocated":4775936,"data_stored":29943107,"data_compressed":1580575,"data_compressed_allocated":2101248,"data_compressed_original":28364800,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056655,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45332,"kb_used_data":4288,"kb_used_omap":16,"kb_used_meta":41007,"kb_avail":94326508,"statfs":{"total":96636764160,"available":96590344192,"internally_reserved":0,"allocated":4390912,"data_stored":27283226,"data_compressed":1515609,"data_compressed_allocated":2011136,"data_compressed_original":25993216,"omap_allocated":17058,"internal_metadata":41991518},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542179,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51736,"kb_used_data":2500,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320104,"statfs":{"total":96636764160,"available":96583786496,"internally_reserved":0,"allocated":2560000,"data_stored":7281919,"data_compressed":404211,"data_compressed_allocated":503808,"data_compressed_original":6238208,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79068,"kb_used_data":3452,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94292772,"statfs":{"total":96636764160,"available":96555798528,"internally_reserved":0,"allocated":3534848,"data_stored":19424410,"data_compressed":997863,"data_compressed_allocated":1323008,"data_compressed_original":18288640,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542179,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61200,"kb_used_data":3644,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310640,"statfs":{"total":96636764160,"available":96574095360,"internally_reserved":0,"allocated":3731456,"data_stored":17576821,"data_compressed":1019924,"data_compressed_allocated":1363968,"data_compressed_original":16297984,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1671168,"data_stored":14994783,"data_compressed":922040,"data_compressed_allocated":1232896,"data_compressed_original":14647296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1523712,"data_stored":17457549,"data_compressed":916461,"data_compressed_allocated":1224704,"data_compressed_original":17240064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":6369578,"data_compressed":404211,"data_compressed_allocated":503808,"data_compressed_original":6238208,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2506752,"data_stored":26376931,"data_compressed":1515609,"data_compressed_allocated":2011136,"data_compressed_original":25993216,"omap_allocated":7902,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2297856,"data_stored":21563855,"data_compressed":1153618,"data_compressed_allocated":1548288,"data_compressed_original":20910080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1556480,"data_stored":14289240,"data_compressed":766980,"data_compressed_allocated":1044480,"data_compressed_original":13877248,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":3102119,"data_compressed":161694,"data_compressed_allocated":233472,"data_compressed_original":3088384,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":30897859,"data_compressed":1545110,"data_compressed_allocated":2101248,"data_compressed_original":30343168,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:31:11.862 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:11.862 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:31:11.862 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:31:11.862 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:31:11.863 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:11.870 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:11.869+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:11.871 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:31:11.971 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:11.970+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:12.014 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:12.015 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:31:12.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:31:12.015 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:31:12.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.015 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19142 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 147456~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:12.015 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:31:12.015 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.10770067499999, 2024-09-17T07:31:12.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047860145, 2024-09-17T07:31:12.015 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.016 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.016 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.016 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.016 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.016 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.016 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19142 2024-09-17T07:31:12.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.016 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:31:12.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:31:12.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905568+0000", 2024-09-17T07:31:12.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9700000000000002e-07 2024-09-17T07:31:12.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905570+0000", 2024-09-17T07:31:12.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0200000000000001e-06 2024-09-17T07:31:12.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905576+0000", 2024-09-17T07:31:12.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7089999999999999e-06 2024-09-17T07:31:12.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905580+0000", 2024-09-17T07:31:12.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2570000000000001e-06 2024-09-17T07:31:12.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953223+0000", 2024-09-17T07:31:12.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047642689000000002 2024-09-17T07:31:12.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953265+0000", 2024-09-17T07:31:12.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2304e-05 2024-09-17T07:31:12.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953428+0000", 2024-09-17T07:31:12.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162769 2024-09-17T07:31:12.020 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.020 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.020 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.021 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19143 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 376832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:12.021 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:31:12.021 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.107681998, 2024-09-17T07:31:12.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048069677999999998, 2024-09-17T07:31:12.021 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.021 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.021 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.021 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.021 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.021 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.022 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19143 2024-09-17T07:31:12.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.022 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:31:12.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905586+0000", 2024-09-17T07:31:12.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905587+0000", 2024-09-17T07:31:12.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:31:12.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905588+0000", 2024-09-17T07:31:12.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.829e-06 2024-09-17T07:31:12.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905594+0000", 2024-09-17T07:31:12.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.947e-06 2024-09-17T07:31:12.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905599+0000", 2024-09-17T07:31:12.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2910000000000001e-06 2024-09-17T07:31:12.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953451+0000", 2024-09-17T07:31:12.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047852230000000003 2024-09-17T07:31:12.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953493+0000", 2024-09-17T07:31:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2259999999999999e-05 2024-09-17T07:31:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953656+0000", 2024-09-17T07:31:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000162721 2024-09-17T07:31:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19144 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:12.025 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:31:12.026 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.107663645, 2024-09-17T07:31:12.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048293464000000001, 2024-09-17T07:31:12.026 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.026 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.026 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.026 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19144 2024-09-17T07:31:12.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.026 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:31:12.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:31:12.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905605+0000", 2024-09-17T07:31:12.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0499999999999999e-07 2024-09-17T07:31:12.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905607+0000", 2024-09-17T07:31:12.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.838e-06 2024-09-17T07:31:12.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905614+0000", 2024-09-17T07:31:12.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7220000000000002e-06 2024-09-17T07:31:12.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905618+0000", 2024-09-17T07:31:12.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5240000000000002e-06 2024-09-17T07:31:12.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953681+0000", 2024-09-17T07:31:12.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048063297999999997 2024-09-17T07:31:12.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953727+0000", 2024-09-17T07:31:12.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5306000000000003e-05 2024-09-17T07:31:12.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953898+0000", 2024-09-17T07:31:12.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000171371 2024-09-17T07:31:12.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.030 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.030 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19145 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:12.030 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:31:12.030 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.10764404700001, 2024-09-17T07:31:12.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048558729000000002, 2024-09-17T07:31:12.031 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.031 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.031 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.031 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.031 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19145 2024-09-17T07:31:12.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.032 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:31:12.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905624+0000", 2024-09-17T07:31:12.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905625+0000", 2024-09-17T07:31:12.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5499999999999999e-07 2024-09-17T07:31:12.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905626+0000", 2024-09-17T07:31:12.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.793e-06 2024-09-17T07:31:12.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905633+0000", 2024-09-17T07:31:12.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6810000000000001e-06 2024-09-17T07:31:12.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905638+0000", 2024-09-17T07:31:12.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1320000000000002e-06 2024-09-17T07:31:12.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953920+0000", 2024-09-17T07:31:12.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048281790999999998 2024-09-17T07:31:12.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.953966+0000", 2024-09-17T07:31:12.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5904e-05 2024-09-17T07:31:12.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954183+0000", 2024-09-17T07:31:12.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000217073 2024-09-17T07:31:12.035 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.035 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.035 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.035 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19146 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1064960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:12.036 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:31:12.036 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.10762409200001, 2024-09-17T07:31:12.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048820860000000001, 2024-09-17T07:31:12.036 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.036 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.036 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.036 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.036 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.036 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.036 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19146 2024-09-17T07:31:12.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.037 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:31:12.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:31:12.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905644+0000", 2024-09-17T07:31:12.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5900000000000003e-07 2024-09-17T07:31:12.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905646+0000", 2024-09-17T07:31:12.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8199999999999999e-06 2024-09-17T07:31:12.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905652+0000", 2024-09-17T07:31:12.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8069999999999996e-06 2024-09-17T07:31:12.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905657+0000", 2024-09-17T07:31:12.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0200000000000002e-06 2024-09-17T07:31:12.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954206+0000", 2024-09-17T07:31:12.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048548767 2024-09-17T07:31:12.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954249+0000", 2024-09-17T07:31:12.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.295e-05 2024-09-17T07:31:12.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954465+0000", 2024-09-17T07:31:12.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021613700000000001 2024-09-17T07:31:12.040 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.041 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.041 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.041 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19147 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:12.042 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:31:12.042 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.10760543699999, 2024-09-17T07:31:12.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049078200000000002, 2024-09-17T07:31:12.042 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.042 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.042 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.042 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.042 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19147 2024-09-17T07:31:12.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.042 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:31:12.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:31:12.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905663+0000", 2024-09-17T07:31:12.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2599999999999998e-07 2024-09-17T07:31:12.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905665+0000", 2024-09-17T07:31:12.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6029999999999999e-06 2024-09-17T07:31:12.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905670+0000", 2024-09-17T07:31:12.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7590000000000003e-06 2024-09-17T07:31:12.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905675+0000", 2024-09-17T07:31:12.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8570000000000003e-06 2024-09-17T07:31:12.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954487+0000", 2024-09-17T07:31:12.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048811347999999997 2024-09-17T07:31:12.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954531+0000", 2024-09-17T07:31:12.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4008999999999998e-05 2024-09-17T07:31:12.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954741+0000", 2024-09-17T07:31:12.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021029800000000001 2024-09-17T07:31:12.046 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.046 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.046 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.047 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19148 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:12.047 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:31:12.047 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.10758696100001, 2024-09-17T07:31:12.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049336390000000001, 2024-09-17T07:31:12.047 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.047 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.047 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.047 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.047 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.047 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.047 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19148 2024-09-17T07:31:12.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.047 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:31:12.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905681+0000", 2024-09-17T07:31:12.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905682+0000", 2024-09-17T07:31:12.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3099999999999998e-07 2024-09-17T07:31:12.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905684+0000", 2024-09-17T07:31:12.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7430000000000001e-06 2024-09-17T07:31:12.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905689+0000", 2024-09-17T07:31:12.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6250000000000004e-06 2024-09-17T07:31:12.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905694+0000", 2024-09-17T07:31:12.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7679999999999999e-06 2024-09-17T07:31:12.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954764+0000", 2024-09-17T07:31:12.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049069660000000001 2024-09-17T07:31:12.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.954807+0000", 2024-09-17T07:31:12.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3025999999999998e-05 2024-09-17T07:31:12.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955018+0000", 2024-09-17T07:31:12.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021103699999999999 2024-09-17T07:31:12.051 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.051 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.051 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.051 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19149 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:12.052 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:31:12.052 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.107567362, 2024-09-17T07:31:12.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049543718, 2024-09-17T07:31:12.052 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.052 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.052 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.052 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19149 2024-09-17T07:31:12.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.052 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:31:12.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:31:12.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905701+0000", 2024-09-17T07:31:12.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4000000000000002e-07 2024-09-17T07:31:12.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905703+0000", 2024-09-17T07:31:12.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.801e-06 2024-09-17T07:31:12.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905709+0000", 2024-09-17T07:31:12.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-17T07:31:12.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905714+0000", 2024-09-17T07:31:12.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8239999999999999e-06 2024-09-17T07:31:12.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955040+0000", 2024-09-17T07:31:12.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049326644000000003 2024-09-17T07:31:12.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955084+0000", 2024-09-17T07:31:12.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3723999999999997e-05 2024-09-17T07:31:12.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955245+0000", 2024-09-17T07:31:12.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160385 2024-09-17T07:31:12.056 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.056 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.056 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.056 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19150 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:12.056 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:31:12.056 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.10754878700001, 2024-09-17T07:31:12.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049798856000000002, 2024-09-17T07:31:12.056 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.056 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.057 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.057 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.057 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19150 2024-09-17T07:31:12.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.057 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:31:12.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905719+0000", 2024-09-17T07:31:12.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905720+0000", 2024-09-17T07:31:12.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:31:12.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905721+0000", 2024-09-17T07:31:12.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.72e-06 2024-09-17T07:31:12.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905727+0000", 2024-09-17T07:31:12.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6949999999999997e-06 2024-09-17T07:31:12.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905735+0000", 2024-09-17T07:31:12.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2449999999999998e-06 2024-09-17T07:31:12.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955268+0000", 2024-09-17T07:31:12.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049532303999999999 2024-09-17T07:31:12.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955311+0000", 2024-09-17T07:31:12.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3544000000000002e-05 2024-09-17T07:31:12.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955518+0000", 2024-09-17T07:31:12.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020701399999999999 2024-09-17T07:31:12.060 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.060 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.060 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.061 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19151 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2211840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:12.061 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:31:12.061 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.10752636000001, 2024-09-17T07:31:12.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050009946, 2024-09-17T07:31:12.061 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.061 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.061 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.061 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19151 2024-09-17T07:31:12.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.061 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:31:12.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:31:12.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905742+0000", 2024-09-17T07:31:12.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5600000000000001e-07 2024-09-17T07:31:12.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905751+0000", 2024-09-17T07:31:12.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5599999999999994e-06 2024-09-17T07:31:12.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905757+0000", 2024-09-17T07:31:12.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.144e-06 2024-09-17T07:31:12.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905763+0000", 2024-09-17T07:31:12.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9510000000000003e-06 2024-09-17T07:31:12.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955541+0000", 2024-09-17T07:31:12.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049778478000000001 2024-09-17T07:31:12.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955584+0000", 2024-09-17T07:31:12.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2410999999999998e-05 2024-09-17T07:31:12.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955752+0000", 2024-09-17T07:31:12.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016804600000000001 2024-09-17T07:31:12.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.065 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.065 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19152 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2441216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:12.065 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:31:12.065 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.10749854299999, 2024-09-17T07:31:12.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050267272000000002, 2024-09-17T07:31:12.065 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.066 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.066 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.066 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.066 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19152 2024-09-17T07:31:12.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.066 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:31:12.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:31:12.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905770+0000", 2024-09-17T07:31:12.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1100000000000001e-07 2024-09-17T07:31:12.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905772+0000", 2024-09-17T07:31:12.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.792e-06 2024-09-17T07:31:12.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905779+0000", 2024-09-17T07:31:12.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3170000000000004e-06 2024-09-17T07:31:12.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905784+0000", 2024-09-17T07:31:12.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1209999999999998e-06 2024-09-17T07:31:12.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955777+0000", 2024-09-17T07:31:12.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.049992700000000001 2024-09-17T07:31:12.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.955821+0000", 2024-09-17T07:31:12.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3599e-05 2024-09-17T07:31:12.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956037+0000", 2024-09-17T07:31:12.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021633199999999999 2024-09-17T07:31:12.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.070 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.070 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.070 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19153 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:12.070 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:31:12.070 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.10747659899999, 2024-09-17T07:31:12.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050544471000000001, 2024-09-17T07:31:12.070 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.070 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.070 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.070 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.071 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19153 2024-09-17T07:31:12.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.071 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:31:12.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:31:12.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905792+0000", 2024-09-17T07:31:12.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6899999999999998e-07 2024-09-17T07:31:12.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905794+0000", 2024-09-17T07:31:12.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.95e-06 2024-09-17T07:31:12.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905801+0000", 2024-09-17T07:31:12.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.9659999999999999e-06 2024-09-17T07:31:12.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905806+0000", 2024-09-17T07:31:12.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7380000000000002e-06 2024-09-17T07:31:12.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956060+0000", 2024-09-17T07:31:12.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050254437999999998 2024-09-17T07:31:12.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956103+0000", 2024-09-17T07:31:12.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3399999999999998e-05 2024-09-17T07:31:12.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956336+0000", 2024-09-17T07:31:12.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023261000000000001 2024-09-17T07:31:12.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.074 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.074 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.074 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19154 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 2899968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:12.075 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:31:12.075 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.10745549500001, 2024-09-17T07:31:12.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050755928999999998, 2024-09-17T07:31:12.075 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.075 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.075 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.075 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.075 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19154 2024-09-17T07:31:12.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.075 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:31:12.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:31:12.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905813+0000", 2024-09-17T07:31:12.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6300000000000001e-07 2024-09-17T07:31:12.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905815+0000", 2024-09-17T07:31:12.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7570000000000001e-06 2024-09-17T07:31:12.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905822+0000", 2024-09-17T07:31:12.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8499999999999996e-06 2024-09-17T07:31:12.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905826+0000", 2024-09-17T07:31:12.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4859999999999999e-06 2024-09-17T07:31:12.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956359+0000", 2024-09-17T07:31:12.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050533061999999997 2024-09-17T07:31:12.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956403+0000", 2024-09-17T07:31:12.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3829999999999999e-05 2024-09-17T07:31:12.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956569+0000", 2024-09-17T07:31:12.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016558100000000001 2024-09-17T07:31:12.078 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.079 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.079 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.079 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:19155 3.a 3:557b10b0:::rbd_data.11838dd88fc3.0000000000000004:d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-17T07:31:12.079 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:31:12.079 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 128.10743589200001, 2024-09-17T07:31:12.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.05103013, 2024-09-17T07:31:12.079 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.079 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.079 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.079 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 19155 2024-09-17T07:31:12.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.080 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:31:12.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905832+0000", 2024-09-17T07:31:12.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905833+0000", 2024-09-17T07:31:12.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.41e-07 2024-09-17T07:31:12.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905835+0000", 2024-09-17T07:31:12.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9420000000000002e-06 2024-09-17T07:31:12.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905841+0000", 2024-09-17T07:31:12.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6229999999999999e-06 2024-09-17T07:31:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.905847+0000", 2024-09-17T07:31:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5929999999999996e-06 2024-09-17T07:31:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956586+0000", 2024-09-17T07:31:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.050739536000000002 2024-09-17T07:31:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956629+0000", 2024-09-17T07:31:12.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.231e-05 2024-09-17T07:31:12.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:03.956862+0000", 2024-09-17T07:31:12.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000233785 2024-09-17T07:31:12.083 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.083 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.083 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.083 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46525 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5656576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:12.083 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:31:12.084 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 110.154494779, 2024-09-17T07:31:12.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047886419999999999, 2024-09-17T07:31:12.084 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.084 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.084 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.084 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.084 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.084 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.084 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46525 2024-09-17T07:31:12.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.084 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:31:12.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858773+0000", 2024-09-17T07:31:12.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858774+0000", 2024-09-17T07:31:12.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.23e-07 2024-09-17T07:31:12.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858778+0000", 2024-09-17T07:31:12.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3950000000000003e-06 2024-09-17T07:31:12.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858781+0000", 2024-09-17T07:31:12.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1949999999999999e-06 2024-09-17T07:31:12.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858784+0000", 2024-09-17T07:31:12.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5179999999999999e-06 2024-09-17T07:31:12.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906499+0000", 2024-09-17T07:31:12.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047715247000000002 2024-09-17T07:31:12.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906576+0000", 2024-09-17T07:31:12.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6651999999999997e-05 2024-09-17T07:31:12.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906660+0000", 2024-09-17T07:31:12.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4190000000000005e-05 2024-09-17T07:31:12.088 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.088 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.088 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.088 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46526 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5672960~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:12.088 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:31:12.088 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 110.154481119, 2024-09-17T07:31:12.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047996023999999998, 2024-09-17T07:31:12.088 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.088 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.088 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.088 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.089 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46526 2024-09-17T07:31:12.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.089 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:31:12.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:31:12.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858787+0000", 2024-09-17T07:31:12.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0100000000000001e-07 2024-09-17T07:31:12.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858788+0000", 2024-09-17T07:31:12.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.9199999999999999e-07 2024-09-17T07:31:12.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858795+0000", 2024-09-17T07:31:12.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.392e-06 2024-09-17T07:31:12.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858799+0000", 2024-09-17T07:31:12.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7670000000000001e-06 2024-09-17T07:31:12.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906684+0000", 2024-09-17T07:31:12.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.047885775999999998 2024-09-17T07:31:12.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906726+0000", 2024-09-17T07:31:12.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2070999999999998e-05 2024-09-17T07:31:12.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906783+0000", 2024-09-17T07:31:12.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6725000000000003e-05 2024-09-17T07:31:12.092 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.092 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.092 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.092 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46527 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5689344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:12.092 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:31:12.093 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 110.154466337, 2024-09-17T07:31:12.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048162231, 2024-09-17T07:31:12.093 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.093 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.093 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46527 2024-09-17T07:31:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:31:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:31:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858802+0000", 2024-09-17T07:31:12.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2600000000000001e-07 2024-09-17T07:31:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858803+0000", 2024-09-17T07:31:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1570000000000001e-06 2024-09-17T07:31:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858807+0000", 2024-09-17T07:31:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8160000000000004e-06 2024-09-17T07:31:12.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858810+0000", 2024-09-17T07:31:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6869999999999999e-06 2024-09-17T07:31:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906813+0000", 2024-09-17T07:31:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048003177000000001 2024-09-17T07:31:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906886+0000", 2024-09-17T07:31:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2754000000000002e-05 2024-09-17T07:31:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906964+0000", 2024-09-17T07:31:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8413999999999998e-05 2024-09-17T07:31:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46528 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5705728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:12.100 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:31:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 110.154455446, 2024-09-17T07:31:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048258747999999997, 2024-09-17T07:31:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46528 2024-09-17T07:31:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:31:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:31:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858813+0000", 2024-09-17T07:31:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:31:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858814+0000", 2024-09-17T07:31:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0410000000000001e-06 2024-09-17T07:31:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858817+0000", 2024-09-17T07:31:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9919999999999999e-06 2024-09-17T07:31:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858820+0000", 2024-09-17T07:31:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.491e-06 2024-09-17T07:31:12.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.906988+0000", 2024-09-17T07:31:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048168420000000003 2024-09-17T07:31:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907029+0000", 2024-09-17T07:31:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0914999999999997e-05 2024-09-17T07:31:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907072+0000", 2024-09-17T07:31:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2675000000000001e-05 2024-09-17T07:31:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46529 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5722112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:31:12.105 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 110.154445886, 2024-09-17T07:31:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048436212999999999, 2024-09-17T07:31:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46529 2024-09-17T07:31:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:31:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858822+0000", 2024-09-17T07:31:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858823+0000", 2024-09-17T07:31:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.11e-07 2024-09-17T07:31:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858824+0000", 2024-09-17T07:31:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2449999999999999e-06 2024-09-17T07:31:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858827+0000", 2024-09-17T07:31:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.653e-06 2024-09-17T07:31:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858830+0000", 2024-09-17T07:31:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5579999999999999e-06 2024-09-17T07:31:12.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907106+0000", 2024-09-17T07:31:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048276449999999999 2024-09-17T07:31:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907179+0000", 2024-09-17T07:31:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2249000000000006e-05 2024-09-17T07:31:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907259+0000", 2024-09-17T07:31:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9846999999999996e-05 2024-09-17T07:31:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:46530 3.a 3:56b5b92c:::rbd_data.11835e2dca9d.0000000000000003:15 [read 5738496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e58)", 2024-09-17T07:31:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:31:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 110.154435326, 2024-09-17T07:31:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.048531352, 2024-09-17T07:31:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.110 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46530 2024-09-17T07:31:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:31:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.858833+0000", 2024-09-17T07:31:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864909+0000", 2024-09-17T07:31:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0060763789999999998 2024-09-17T07:31:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864914+0000", 2024-09-17T07:31:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3080000000000001e-06 2024-09-17T07:31:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864925+0000", 2024-09-17T07:31:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1069e-05 2024-09-17T07:31:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.864930+0000", 2024-09-17T07:31:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3340000000000001e-06 2024-09-17T07:31:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907272+0000", 2024-09-17T07:31:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.042341880999999998 2024-09-17T07:31:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907314+0000", 2024-09-17T07:31:12.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1643000000000001e-05 2024-09-17T07:31:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:29:21.907364+0000", 2024-09-17T07:31:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0738000000000001e-05 2024-09-17T07:31:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.114 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.114 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:12.116 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:31:12.116 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:12.114+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:12.118 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T07:31:12.217 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-17T07:31:12.216+0000 7fb4f5252640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33357) UID: 0 2024-09-17T07:31:12.232 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:31:12.232 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-17T07:31:12.232 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:12.241 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:31:12.318 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:12.317+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:12.385 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:12.385 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-17T07:31:12.385 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-17T07:31:12.385 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-17T07:31:12.385 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:12.393 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:31:12.437 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:12.437+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:12.498 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-17T07:31:12.499 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-17T07:31:12.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-17T07:31:12.499 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-17T07:31:12.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.499 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4212.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T07:31:12.499 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:31:12.499 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.16269213800001, 2024-09-17T07:31:12.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21334391999999999, 2024-09-17T07:31:12.500 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.500 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.500 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:12.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4212", 2024-09-17T07:31:12.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2705810036", 2024-09-17T07:31:12.500 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-17T07:31:12.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.500 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:31:12.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334670+0000", 2024-09-17T07:31:12.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334671+0000", 2024-09-17T07:31:12.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4589999999999999e-06 2024-09-17T07:31:12.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334679+0000", 2024-09-17T07:31:12.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.841e-06 2024-09-17T07:31:12.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334689+0000", 2024-09-17T07:31:12.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.5310000000000007e-06 2024-09-17T07:31:12.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334697+0000", 2024-09-17T07:31:12.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3410000000000003e-06 2024-09-17T07:31:12.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334759+0000", 2024-09-17T07:31:12.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2310999999999993e-05 2024-09-17T07:31:12.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:31:12.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.334767+0000", 2024-09-17T07:31:12.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3220000000000004e-06 2024-09-17T07:31:12.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546513+0000", 2024-09-17T07:31:12.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.21174608 2024-09-17T07:31:12.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546765+0000", 2024-09-17T07:31:12.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025228099999999998 2024-09-17T07:31:12.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-17T07:31:12.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.546983+0000", 2024-09-17T07:31:12.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021788800000000001 2024-09-17T07:31:12.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:31:12.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547752+0000", 2024-09-17T07:31:12.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00076898999999999999 2024-09-17T07:31:12.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:12.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547943+0000", 2024-09-17T07:31:12.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000191335 2024-09-17T07:31:12.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:31:12.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.547982+0000", 2024-09-17T07:31:12.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8726000000000002e-05 2024-09-17T07:31:12.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:26.548014+0000", 2024-09-17T07:31:12.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1814999999999999e-05 2024-09-17T07:31:12.506 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.506 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.506 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.506 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7379 3.d 3:b7eacecb:::rbd_data.118312a5715.0000000000000001:6 [read 7241728~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.506 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:31:12.506 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.573639988, 2024-09-17T07:31:12.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073435577000000002, 2024-09-17T07:31:12.506 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.507 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.507 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.507 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.507 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.507 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.507 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7379 2024-09-17T07:31:12.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.507 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:31:12.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.923722+0000", 2024-09-17T07:31:12.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928741+0000", 2024-09-17T07:31:12.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0050187399999999998 2024-09-17T07:31:12.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928747+0000", 2024-09-17T07:31:12.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0070000000000003e-06 2024-09-17T07:31:12.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928757+0000", 2024-09-17T07:31:12.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8109999999999997e-06 2024-09-17T07:31:12.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.928764+0000", 2024-09-17T07:31:12.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:31:12.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996766+0000", 2024-09-17T07:31:12.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068002148999999998 2024-09-17T07:31:12.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.996853+0000", 2024-09-17T07:31:12.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6448999999999999e-05 2024-09-17T07:31:12.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.997158+0000", 2024-09-17T07:31:12.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00030484000000000002 2024-09-17T07:31:12.511 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.511 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.511 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.511 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7399 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6455296~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.511 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:31:12.511 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.56835687200001, 2024-09-17T07:31:12.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073226299999999994, 2024-09-17T07:31:12.511 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.511 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.511 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.512 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7399 2024-09-17T07:31:12.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.512 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:31:12.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:31:12.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929005+0000", 2024-09-17T07:31:12.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3999999999999998e-07 2024-09-17T07:31:12.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929006+0000", 2024-09-17T07:31:12.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0100000000000001e-06 2024-09-17T07:31:12.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929015+0000", 2024-09-17T07:31:12.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1960000000000008e-06 2024-09-17T07:31:12.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929018+0000", 2024-09-17T07:31:12.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0149999999999999e-06 2024-09-17T07:31:12.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.001918+0000", 2024-09-17T07:31:12.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072900521999999995 2024-09-17T07:31:12.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002000+0000", 2024-09-17T07:31:12.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2276999999999998e-05 2024-09-17T07:31:12.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002231+0000", 2024-09-17T07:31:12.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023104000000000001 2024-09-17T07:31:12.515 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.515 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.515 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.516 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7400 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6520832~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.516 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:31:12.516 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.56832589199999, 2024-09-17T07:31:12.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073478582000000001, 2024-09-17T07:31:12.516 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.516 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.516 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.516 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7400 2024-09-17T07:31:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:31:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929036+0000", 2024-09-17T07:31:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929037+0000", 2024-09-17T07:31:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7599999999999997e-07 2024-09-17T07:31:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929039+0000", 2024-09-17T07:31:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1160000000000002e-06 2024-09-17T07:31:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929045+0000", 2024-09-17T07:31:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7679999999999997e-06 2024-09-17T07:31:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929059+0000", 2024-09-17T07:31:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4195e-05 2024-09-17T07:31:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002260+0000", 2024-09-17T07:31:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073201117999999996 2024-09-17T07:31:12.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002300+0000", 2024-09-17T07:31:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0222000000000002e-05 2024-09-17T07:31:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002515+0000", 2024-09-17T07:31:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000214587 2024-09-17T07:31:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.520 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7401 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6586368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:31:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.56829584299999, 2024-09-17T07:31:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073655056999999996, 2024-09-17T07:31:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7401 2024-09-17T07:31:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:31:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929066+0000", 2024-09-17T07:31:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929067+0000", 2024-09-17T07:31:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.03e-07 2024-09-17T07:31:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929069+0000", 2024-09-17T07:31:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2280000000000001e-06 2024-09-17T07:31:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929079+0000", 2024-09-17T07:31:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7089999999999997e-06 2024-09-17T07:31:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929088+0000", 2024-09-17T07:31:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9819999999999997e-06 2024-09-17T07:31:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002536+0000", 2024-09-17T07:31:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073448357000000006 2024-09-17T07:31:12.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002581+0000", 2024-09-17T07:31:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4638999999999997e-05 2024-09-17T07:31:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002721+0000", 2024-09-17T07:31:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000140739 2024-09-17T07:31:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7402 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6651904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:31:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.56826743600001, 2024-09-17T07:31:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881457999999997, 2024-09-17T07:31:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7402 2024-09-17T07:31:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.526 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:31:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:31:12.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929095+0000", 2024-09-17T07:31:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7899999999999999e-07 2024-09-17T07:31:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929097+0000", 2024-09-17T07:31:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1900000000000002e-06 2024-09-17T07:31:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929103+0000", 2024-09-17T07:31:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0959999999999997e-06 2024-09-17T07:31:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929108+0000", 2024-09-17T07:31:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.118e-06 2024-09-17T07:31:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002749+0000", 2024-09-17T07:31:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073640705000000001 2024-09-17T07:31:12.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002804+0000", 2024-09-17T07:31:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4889000000000003e-05 2024-09-17T07:31:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.002976+0000", 2024-09-17T07:31:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017208100000000001 2024-09-17T07:31:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.530 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7403 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6717440~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:31:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.56824761499999, 2024-09-17T07:31:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074128469000000002, 2024-09-17T07:31:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.531 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7403 2024-09-17T07:31:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:31:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929114+0000", 2024-09-17T07:31:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929115+0000", 2024-09-17T07:31:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.72e-07 2024-09-17T07:31:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929117+0000", 2024-09-17T07:31:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8479999999999999e-06 2024-09-17T07:31:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929124+0000", 2024-09-17T07:31:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4900000000000003e-06 2024-09-17T07:31:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929129+0000", 2024-09-17T07:31:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3499999999999996e-06 2024-09-17T07:31:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003011+0000", 2024-09-17T07:31:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073881464999999993 2024-09-17T07:31:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003088+0000", 2024-09-17T07:31:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7391000000000005e-05 2024-09-17T07:31:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003243+0000", 2024-09-17T07:31:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015455300000000001 2024-09-17T07:31:12.535 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7404 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 6782976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:31:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.56822336299999, 2024-09-17T07:31:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074397772000000001, 2024-09-17T07:31:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.536 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7404 2024-09-17T07:31:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:31:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.929139+0000", 2024-09-17T07:31:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934287+0000", 2024-09-17T07:31:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051480229999999998 2024-09-17T07:31:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934291+0000", 2024-09-17T07:31:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.583e-06 2024-09-17T07:31:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934304+0000", 2024-09-17T07:31:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2534999999999999e-05 2024-09-17T07:31:12.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934310+0000", 2024-09-17T07:31:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2600000000000002e-06 2024-09-17T07:31:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003276+0000", 2024-09-17T07:31:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.068966229000000004 2024-09-17T07:31:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003332+0000", 2024-09-17T07:31:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.5486999999999999e-05 2024-09-17T07:31:12.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.003536+0000", 2024-09-17T07:31:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020465500000000001 2024-09-17T07:31:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7419 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7766016~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:31:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.562800775, 2024-09-17T07:31:12.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073006585999999998, 2024-09-17T07:31:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7419 2024-09-17T07:31:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:31:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934561+0000", 2024-09-17T07:31:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934562+0000", 2024-09-17T07:31:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4400000000000001e-07 2024-09-17T07:31:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934563+0000", 2024-09-17T07:31:12.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4389999999999999e-06 2024-09-17T07:31:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934578+0000", 2024-09-17T07:31:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5446000000000001e-05 2024-09-17T07:31:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934583+0000", 2024-09-17T07:31:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3089999999999997e-06 2024-09-17T07:31:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007325+0000", 2024-09-17T07:31:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072741907999999994 2024-09-17T07:31:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007368+0000", 2024-09-17T07:31:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3730999999999999e-05 2024-09-17T07:31:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007568+0000", 2024-09-17T07:31:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000199509 2024-09-17T07:31:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.544 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7420 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7831552~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:31:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.56275389999999, 2024-09-17T07:31:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073177543999999997, 2024-09-17T07:31:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7420 2024-09-17T07:31:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:31:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:31:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934608+0000", 2024-09-17T07:31:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.34e-07 2024-09-17T07:31:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934610+0000", 2024-09-17T07:31:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6929999999999999e-06 2024-09-17T07:31:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934614+0000", 2024-09-17T07:31:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1389999999999997e-06 2024-09-17T07:31:12.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934617+0000", 2024-09-17T07:31:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9730000000000002e-06 2024-09-17T07:31:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007585+0000", 2024-09-17T07:31:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.072967515999999996 2024-09-17T07:31:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007630+0000", 2024-09-17T07:31:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5330000000000001e-05 2024-09-17T07:31:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007786+0000", 2024-09-17T07:31:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015555899999999999 2024-09-17T07:31:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7421 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7897088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:31:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.562739196, 2024-09-17T07:31:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073411882999999997, 2024-09-17T07:31:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7421 2024-09-17T07:31:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.550 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:31:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:31:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934623+0000", 2024-09-17T07:31:12.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1400000000000001e-07 2024-09-17T07:31:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934624+0000", 2024-09-17T07:31:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1370000000000001e-06 2024-09-17T07:31:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934628+0000", 2024-09-17T07:31:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8519999999999997e-06 2024-09-17T07:31:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934639+0000", 2024-09-17T07:31:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1267000000000001e-05 2024-09-17T07:31:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007825+0000", 2024-09-17T07:31:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073185306000000006 2024-09-17T07:31:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.007881+0000", 2024-09-17T07:31:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6029999999999997e-05 2024-09-17T07:31:12.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008035+0000", 2024-09-17T07:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000154077 2024-09-17T07:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7422 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 7962624~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.56271610100001, 2024-09-17T07:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073597830000000003, 2024-09-17T07:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.554 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7422 2024-09-17T07:31:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:31:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:31:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934646+0000", 2024-09-17T07:31:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-17T07:31:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934651+0000", 2024-09-17T07:31:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9250000000000003e-06 2024-09-17T07:31:12.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934656+0000", 2024-09-17T07:31:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9370000000000003e-06 2024-09-17T07:31:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934661+0000", 2024-09-17T07:31:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3029999999999999e-06 2024-09-17T07:31:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008047+0000", 2024-09-17T07:31:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073385985000000001 2024-09-17T07:31:12.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008088+0000", 2024-09-17T07:31:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0763999999999998e-05 2024-09-17T07:31:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008244+0000", 2024-09-17T07:31:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015565800000000001 2024-09-17T07:31:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7423 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8028160~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:31:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.56268472599999, 2024-09-17T07:31:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073860005000000006, 2024-09-17T07:31:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7423 2024-09-17T07:31:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.559 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:31:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934677+0000", 2024-09-17T07:31:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934678+0000", 2024-09-17T07:31:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8000000000000002e-07 2024-09-17T07:31:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934679+0000", 2024-09-17T07:31:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.592e-06 2024-09-17T07:31:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934684+0000", 2024-09-17T07:31:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9400000000000001e-06 2024-09-17T07:31:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934688+0000", 2024-09-17T07:31:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5599999999999998e-06 2024-09-17T07:31:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008278+0000", 2024-09-17T07:31:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073590820000000001 2024-09-17T07:31:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008368+0000", 2024-09-17T07:31:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.9197000000000007e-05 2024-09-17T07:31:12.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008537+0000", 2024-09-17T07:31:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00016961600000000001 2024-09-17T07:31:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7424 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8093696~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:31:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.56266946599999, 2024-09-17T07:31:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074111708999999998, 2024-09-17T07:31:12.563 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7424 2024-09-17T07:31:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:31:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:31:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934693+0000", 2024-09-17T07:31:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1299999999999999e-07 2024-09-17T07:31:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934694+0000", 2024-09-17T07:31:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1969999999999999e-06 2024-09-17T07:31:12.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934699+0000", 2024-09-17T07:31:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6600000000000003e-06 2024-09-17T07:31:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934706+0000", 2024-09-17T07:31:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6499999999999996e-06 2024-09-17T07:31:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008562+0000", 2024-09-17T07:31:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.073855546999999994 2024-09-17T07:31:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008607+0000", 2024-09-17T07:31:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4688999999999999e-05 2024-09-17T07:31:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008804+0000", 2024-09-17T07:31:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000197753 2024-09-17T07:31:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.567 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7425 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8159232~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:31:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.56264939600001, 2024-09-17T07:31:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074319461000000003, 2024-09-17T07:31:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7425 2024-09-17T07:31:12.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:31:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:31:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934713+0000", 2024-09-17T07:31:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3300000000000001e-07 2024-09-17T07:31:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934714+0000", 2024-09-17T07:31:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3969999999999999e-06 2024-09-17T07:31:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934718+0000", 2024-09-17T07:31:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7270000000000001e-06 2024-09-17T07:31:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934721+0000", 2024-09-17T07:31:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.005e-06 2024-09-17T07:31:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008837+0000", 2024-09-17T07:31:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074115586999999997 2024-09-17T07:31:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.008877+0000", 2024-09-17T07:31:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0280000000000001e-05 2024-09-17T07:31:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009032+0000", 2024-09-17T07:31:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015523199999999999 2024-09-17T07:31:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7426 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8224768~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:31:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.562637456, 2024-09-17T07:31:12.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074524551999999994, 2024-09-17T07:31:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7426 2024-09-17T07:31:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:31:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:31:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934725+0000", 2024-09-17T07:31:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1199999999999999e-07 2024-09-17T07:31:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934726+0000", 2024-09-17T07:31:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2589999999999999e-06 2024-09-17T07:31:12.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934735+0000", 2024-09-17T07:31:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.7979999999999995e-06 2024-09-17T07:31:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934738+0000", 2024-09-17T07:31:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3129999999999999e-06 2024-09-17T07:31:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009059+0000", 2024-09-17T07:31:12.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074320905000000007 2024-09-17T07:31:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009100+0000", 2024-09-17T07:31:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1362999999999997e-05 2024-09-17T07:31:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009249+0000", 2024-09-17T07:31:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014870199999999999 2024-09-17T07:31:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:7427 3.15 3:ad869e43:::rbd_data.118312a5715.0000000000000002:6 [read 8290304~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-17T07:31:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:31:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.562614298, 2024-09-17T07:31:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.074782629000000003, 2024-09-17T07:31:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7427 2024-09-17T07:31:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.577 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:31:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.934748+0000", 2024-09-17T07:31:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938218+0000", 2024-09-17T07:31:12.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034701559999999998 2024-09-17T07:31:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938223+0000", 2024-09-17T07:31:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9219999999999996e-06 2024-09-17T07:31:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938235+0000", 2024-09-17T07:31:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2591e-05 2024-09-17T07:31:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:54.938245+0000", 2024-09-17T07:31:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.8730000000000002e-06 2024-09-17T07:31:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009280+0000", 2024-09-17T07:31:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.071034923 2024-09-17T07:31:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009321+0000", 2024-09-17T07:31:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1221999999999999e-05 2024-09-17T07:31:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:28:55.009530+0000", 2024-09-17T07:31:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020894199999999999 2024-09-17T07:31:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:115644 3.15 3:af37e0f6:::rbd_header.118311e0d6f8:head [watch reconnect cookie 140227662450992 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T07:31:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:31:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.419457403999999, 2024-09-17T07:31:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.013658722, 2024-09-17T07:31:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.581 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-17T07:31:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115644 2024-09-17T07:31:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:31:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077905+0000", 2024-09-17T07:31:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077907+0000", 2024-09-17T07:31:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.08e-06 2024-09-17T07:31:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077916+0000", 2024-09-17T07:31:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1579999999999994e-06 2024-09-17T07:31:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077938+0000", 2024-09-17T07:31:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1883999999999999e-05 2024-09-17T07:31:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.077950+0000", 2024-09-17T07:31:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2298e-05 2024-09-17T07:31:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.078037+0000", 2024-09-17T07:31:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6726999999999998e-05 2024-09-17T07:31:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-17T07:31:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.078046+0000", 2024-09-17T07:31:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.6010000000000001e-06 2024-09-17T07:31:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085705+0000", 2024-09-17T07:31:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007659084 2024-09-17T07:31:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:31:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085718+0000", 2024-09-17T07:31:12.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2081e-05 2024-09-17T07:31:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.089697+0000", 2024-09-17T07:31:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.003979373 2024-09-17T07:31:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.089875+0000", 2024-09-17T07:31:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017760000000000001 2024-09-17T07:31:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.091563+0000", 2024-09-17T07:31:12.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001688836 2024-09-17T07:31:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:115645 3.15 3:af37e0f6:::rbd_header.118311e0d6f8:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-17T07:31:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:31:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.416639883999999, 2024-09-17T07:31:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.011649171, 2024-09-17T07:31:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 115645 2024-09-17T07:31:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:31:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080722+0000", 2024-09-17T07:31:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080724+0000", 2024-09-17T07:31:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5719999999999999e-06 2024-09-17T07:31:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080735+0000", 2024-09-17T07:31:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1544e-05 2024-09-17T07:31:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080751+0000", 2024-09-17T07:31:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5244e-05 2024-09-17T07:31:12.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080761+0000", 2024-09-17T07:31:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0372e-05 2024-09-17T07:31:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080834+0000", 2024-09-17T07:31:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3263999999999995e-05 2024-09-17T07:31:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting_for_map not empty", 2024-09-17T07:31:12.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.080842+0000", 2024-09-17T07:31:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2209999999999997e-06 2024-09-17T07:31:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085755+0000", 2024-09-17T07:31:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0049130729999999996 2024-09-17T07:31:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:31:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:17.085766+0000", 2024-09-17T07:31:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0288e-05 2024-09-17T07:31:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.090746+0000", 2024-09-17T07:31:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0049801190000001 2024-09-17T07:31:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.090798+0000", 2024-09-17T07:31:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2502e-05 2024-09-17T07:31:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 3", 2024-09-17T07:31:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.091037+0000", 2024-09-17T07:31:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0002385 2024-09-17T07:31:12.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092119+0000", 2024-09-17T07:31:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010820249999999999 2024-09-17T07:31:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:31:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092276+0000", 2024-09-17T07:31:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000156737 2024-09-17T07:31:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:31:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092330+0000", 2024-09-17T07:31:12.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4666999999999997e-05 2024-09-17T07:31:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:18.092371+0000", 2024-09-17T07:31:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1043e-05 2024-09-17T07:31:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4483.0:116418 3.13 3:cdf2692d:::rbd_data.1183cb746bb8.0000000000000024:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-17T07:31:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:31:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 41.327223486999998, 2024-09-17T07:31:12.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.92140365899999999, 2024-09-17T07:31:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-17T07:31:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-17T07:31:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-17T07:31:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4483", 2024-09-17T07:31:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 116418 2024-09-17T07:31:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-17T07:31:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-17T07:31:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:31:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-17T07:31:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170139+0000", 2024-09-17T07:31:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-17T07:31:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-17T07:31:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170140+0000", 2024-09-17T07:31:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.398e-06 2024-09-17T07:31:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-17T07:31:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170144+0000", 2024-09-17T07:31:12.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4399999999999998e-06 2024-09-17T07:31:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-17T07:31:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170157+0000", 2024-09-17T07:31:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2133999999999999e-05 2024-09-17T07:31:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170164+0000", 2024-09-17T07:31:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7200000000000006e-06 2024-09-17T07:31:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170227+0000", 2024-09-17T07:31:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3035999999999998e-05 2024-09-17T07:31:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-17T07:31:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:31.170238+0000", 2024-09-17T07:31:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0397000000000001e-05 2024-09-17T07:31:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-17T07:31:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086367+0000", 2024-09-17T07:31:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.916128889 2024-09-17T07:31:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-17T07:31:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086523+0000", 2024-09-17T07:31:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015638800000000001 2024-09-17T07:31:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 4", 2024-09-17T07:31:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.086652+0000", 2024-09-17T07:31:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000128806 2024-09-17T07:31:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-17T07:31:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091261+0000", 2024-09-17T07:31:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0046091020000000003 2024-09-17T07:31:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091449+0000", 2024-09-17T07:31:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000188486 2024-09-17T07:31:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-17T07:31:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091503+0000", 2024-09-17T07:31:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3254999999999997e-05 2024-09-17T07:31:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-17T07:31:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-17T07:31:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-17T07:31:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-17T07:30:32.091542+0000", 2024-09-17T07:31:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9607999999999998e-05 2024-09-17T07:31:12.600 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-17T07:31:12.601 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-17T07:31:12.601 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-17T07:31:12.602 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:31:12.603 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:12.601+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:12.703 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:12.702+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:12.711 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:12.711 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:31:12.711 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:31:12.711 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:12.720 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:31:12.804 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:12.804+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:12.827 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:31:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:31:12.827 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:31:12.827 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:12.836 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:31:12.906 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-17T07:31:12.905+0000 7f147d0f8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33358) UID: 0 2024-09-17T07:31:12.945 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:31:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:31:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:31:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115899 3.b 3:d406e0ff:::rbd_data.118340e4fbf.0000000000000001:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:31:12.945 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:31:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.752364403999998, 2024-09-17T07:31:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0040641219999999999, 2024-09-17T07:31:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115899 2024-09-17T07:31:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.946 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:31:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190935+0000", 2024-09-17T07:31:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.947 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190936+0000", 2024-09-17T07:31:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.26e-06 2024-09-17T07:31:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190941+0000", 2024-09-17T07:31:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0499999999999999e-06 2024-09-17T07:31:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.948 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190958+0000", 2024-09-17T07:31:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6548000000000001e-05 2024-09-17T07:31:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.190966+0000", 2024-09-17T07:31:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6799999999999999e-06 2024-09-17T07:31:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191028+0000", 2024-09-17T07:31:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2082999999999995e-05 2024-09-17T07:31:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191161+0000", 2024-09-17T07:31:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000132461 2024-09-17T07:31:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:31:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191284+0000", 2024-09-17T07:31:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012301200000000001 2024-09-17T07:31:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.951 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192371+0000", 2024-09-17T07:31:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00108709 2024-09-17T07:31:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194898+0000", 2024-09-17T07:31:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0025272710000000002 2024-09-17T07:31:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.952 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194957+0000", 2024-09-17T07:31:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8792000000000002e-05 2024-09-17T07:31:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.194999+0000", 2024-09-17T07:31:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1875000000000001e-05 2024-09-17T07:31:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.953 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115903 3.18 3:1fc9b9ef:::rbd_data.118340e4fbf.0000000000000005:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.751566281000002, 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00477466, 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115903 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:31:12.954 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:31:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191733+0000", 2024-09-17T07:31:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9800000000000004e-07 2024-09-17T07:31:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191736+0000", 2024-09-17T07:31:12.955 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6620000000000001e-06 2024-09-17T07:31:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191743+0000", 2024-09-17T07:31:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6939999999999998e-06 2024-09-17T07:31:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191748+0000", 2024-09-17T07:31:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9749999999999998e-06 2024-09-17T07:31:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191823+0000", 2024-09-17T07:31:12.956 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4954000000000001e-05 2024-09-17T07:31:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191933+0000", 2024-09-17T07:31:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000110708 2024-09-17T07:31:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:31:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192031+0000", 2024-09-17T07:31:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7974999999999996e-05 2024-09-17T07:31:12.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193329+0000", 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0012976419999999999 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196404+0000", 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003075174 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196461+0000", 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6663999999999998e-05 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196507+0000", 2024-09-17T07:31:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6714e-05 2024-09-17T07:31:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:115905 3.18 3:19f08345:::rbd_data.118340e4fbf.0000000000000009:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-17T07:31:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:31:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 50.751535410999999, 2024-09-17T07:31:12.959 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0049339900000000001, 2024-09-17T07:31:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 115905 2024-09-17T07:31:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:31:12.960 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:31:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191764+0000", 2024-09-17T07:31:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.0699999999999998e-07 2024-09-17T07:31:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191766+0000", 2024-09-17T07:31:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.928e-06 2024-09-17T07:31:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191773+0000", 2024-09-17T07:31:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2749999999999998e-06 2024-09-17T07:31:12.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.191778+0000", 2024-09-17T07:31:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7849999999999999e-06 2024-09-17T07:31:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192756+0000", 2024-09-17T07:31:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00097834399999999992 2024-09-17T07:31:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.963 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192868+0000", 2024-09-17T07:31:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000111575 2024-09-17T07:31:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:31:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.192970+0000", 2024-09-17T07:31:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00010192300000000001 2024-09-17T07:31:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.193865+0000", 2024-09-17T07:31:12.964 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00089556100000000001 2024-09-17T07:31:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196583+0000", 2024-09-17T07:31:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027174180000000001 2024-09-17T07:31:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196648+0000", 2024-09-17T07:31:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4743e-05 2024-09-17T07:31:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:22.196698+0000", 2024-09-17T07:31:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0031000000000003e-05 2024-09-17T07:31:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117007 3.8 3:11c9baa3:::rbd_object_map.118394b259f9:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-17T07:31:12.966 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:36.120261+0000", 2024-09-17T07:31:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 36.823038353000001, 2024-09-17T07:31:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045697289999999998, 2024-09-17T07:31:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117007 2024-09-17T07:31:12.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120261+0000", 2024-09-17T07:31:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120261+0000", 2024-09-17T07:31:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:12.968 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120262+0000", 2024-09-17T07:31:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.77e-06 2024-09-17T07:31:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120270+0000", 2024-09-17T07:31:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0080000000000002e-06 2024-09-17T07:31:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120287+0000", 2024-09-17T07:31:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6847e-05 2024-09-17T07:31:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.969 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120297+0000", 2024-09-17T07:31:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8830000000000001e-06 2024-09-17T07:31:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120368+0000", 2024-09-17T07:31:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0689e-05 2024-09-17T07:31:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:31:12.970 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.120375+0000", 2024-09-17T07:31:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.187e-06 2024-09-17T07:31:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.122574+0000", 2024-09-17T07:31:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0021992980000000001 2024-09-17T07:31:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.122670+0000", 2024-09-17T07:31:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5830999999999998e-05 2024-09-17T07:31:12.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 1", 2024-09-17T07:31:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.122866+0000", 2024-09-17T07:31:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000195303 2024-09-17T07:31:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124084+0000", 2024-09-17T07:31:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001218076 2024-09-17T07:31:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:12.972 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124153+0000", 2024-09-17T07:31:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9839999999999995e-05 2024-09-17T07:31:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124784+0000", 2024-09-17T07:31:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00063082299999999997 2024-09-17T07:31:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:12.973 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:36.124830+0000", 2024-09-17T07:31:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6174000000000002e-05 2024-09-17T07:31:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117151 3.7 3:e1d03a89:::rbd_header.11832ac10dcc:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T07:31:12.974 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.097104+0000", 2024-09-17T07:31:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.846194697000001, 2024-09-17T07:31:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0062858530000000001, 2024-09-17T07:31:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.975 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117151 2024-09-17T07:31:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097104+0000", 2024-09-17T07:31:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.976 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097104+0000", 2024-09-17T07:31:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097107+0000", 2024-09-17T07:31:12.977 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7429999999999998e-06 2024-09-17T07:31:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097121+0000", 2024-09-17T07:31:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3774000000000001e-05 2024-09-17T07:31:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.978 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097150+0000", 2024-09-17T07:31:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.8762e-05 2024-09-17T07:31:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097163+0000", 2024-09-17T07:31:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3631e-05 2024-09-17T07:31:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097245+0000", 2024-09-17T07:31:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.1874999999999998e-05 2024-09-17T07:31:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:12.980 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097337+0000", 2024-09-17T07:31:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1465000000000006e-05 2024-09-17T07:31:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:31:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.097553+0000", 2024-09-17T07:31:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00021630999999999999 2024-09-17T07:31:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.098320+0000", 2024-09-17T07:31:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00076674000000000002 2024-09-17T07:31:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.103265+0000", 2024-09-17T07:31:12.982 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0049452250000000001 2024-09-17T07:31:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.103335+0000", 2024-09-17T07:31:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0463999999999995e-05 2024-09-17T07:31:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:12.983 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.103390+0000", 2024-09-17T07:31:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4864000000000002e-05 2024-09-17T07:31:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.984 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:117237 3.2 3:40c8b221:::rbd_data.11832ac10dcc.0000000000000000:head [delete] snapc 1d=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-17T07:31:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.169608+0000", 2024-09-17T07:31:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.773690700000003, 2024-09-17T07:31:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003512509, 2024-09-17T07:31:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:12.985 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 117237 2024-09-17T07:31:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169608+0000", 2024-09-17T07:31:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169608+0000", 2024-09-17T07:31:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:12.987 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169610+0000", 2024-09-17T07:31:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8020000000000001e-06 2024-09-17T07:31:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169618+0000", 2024-09-17T07:31:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0080000000000002e-06 2024-09-17T07:31:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:12.988 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169635+0000", 2024-09-17T07:31:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6611000000000001e-05 2024-09-17T07:31:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169643+0000", 2024-09-17T07:31:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6549999999999993e-06 2024-09-17T07:31:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169723+0000", 2024-09-17T07:31:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0034e-05 2024-09-17T07:31:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.989 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169860+0000", 2024-09-17T07:31:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00013645899999999999 2024-09-17T07:31:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:31:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.169970+0000", 2024-09-17T07:31:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000109957 2024-09-17T07:31:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.171850+0000", 2024-09-17T07:31:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001879868 2024-09-17T07:31:12.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172391+0000", 2024-09-17T07:31:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00054084100000000002 2024-09-17T07:31:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172439+0000", 2024-09-17T07:31:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7916e-05 2024-09-17T07:31:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173121+0000", 2024-09-17T07:31:12.991 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00068235799999999999 2024-09-17T07:31:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(client.4483.0:117250 3.5 e128/109 3:a4728dbe:::rbd_data.11832ac10dcc.0000000000000075:head v 128'521, mlcod=126'519)", 2024-09-17T07:31:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:31:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 35.770395170999997, 2024-09-17T07:31:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0034023949999999999, 2024-09-17T07:31:12.992 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:31:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:31:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.993 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172904+0000", 2024-09-17T07:31:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5900000000000002e-07 2024-09-17T07:31:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172910+0000", 2024-09-17T07:31:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2449999999999998e-06 2024-09-17T07:31:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.172919+0000", 2024-09-17T07:31:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5389999999999996e-06 2024-09-17T07:31:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173431+0000", 2024-09-17T07:31:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00051224699999999996 2024-09-17T07:31:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173485+0000", 2024-09-17T07:31:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3936e-05 2024-09-17T07:31:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.173504+0000", 2024-09-17T07:31:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8681e-05 2024-09-17T07:31:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.176259+0000", 2024-09-17T07:31:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0027547719999999999 2024-09-17T07:31:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:37.176306+0000", 2024-09-17T07:31:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7515999999999997e-05 2024-09-17T07:31:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.996 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:118889 3.b 3:d08c2e41:::rbd_data.11839a97b753.0000000000000002:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-17T07:31:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:50.259083+0000", 2024-09-17T07:31:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.684216209999999, 2024-09-17T07:31:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0060418879999999996, 2024-09-17T07:31:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 118889 2024-09-17T07:31:12.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259083+0000", 2024-09-17T07:31:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.998 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259083+0000", 2024-09-17T07:31:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259085+0000", 2024-09-17T07:31:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8509999999999999e-06 2024-09-17T07:31:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259092+0000", 2024-09-17T07:31:12.999 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9500000000000004e-06 2024-09-17T07:31:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259111+0000", 2024-09-17T07:31:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9165e-05 2024-09-17T07:31:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259122+0000", 2024-09-17T07:31:13.000 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0745e-05 2024-09-17T07:31:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259206+0000", 2024-09-17T07:31:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.4741999999999996e-05 2024-09-17T07:31:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259345+0000", 2024-09-17T07:31:13.001 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000138644 2024-09-17T07:31:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:31:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259470+0000", 2024-09-17T07:31:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012535899999999999 2024-09-17T07:31:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.261565+0000", 2024-09-17T07:31:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.002094561 2024-09-17T07:31:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:13.002 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265016+0000", 2024-09-17T07:31:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0034515240000000001 2024-09-17T07:31:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265080+0000", 2024-09-17T07:31:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3927e-05 2024-09-17T07:31:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.265125+0000", 2024-09-17T07:31:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4419999999999998e-05 2024-09-17T07:31:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.003 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:118890 3.2 3:4647e1ae:::rbd_data.11839a97b753.0000000000000003:head [delete] snapc 14=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-17T07:31:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:50.259150+0000", 2024-09-17T07:31:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 22.684148887999999, 2024-09-17T07:31:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003910134, 2024-09-17T07:31:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.004 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 118890 2024-09-17T07:31:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259150+0000", 2024-09-17T07:31:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259150+0000", 2024-09-17T07:31:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259151+0000", 2024-09-17T07:31:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1999999999999999e-07 2024-09-17T07:31:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259154+0000", 2024-09-17T07:31:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5020000000000001e-06 2024-09-17T07:31:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259164+0000", 2024-09-17T07:31:13.006 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.8190000000000003e-06 2024-09-17T07:31:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.259171+0000", 2024-09-17T07:31:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9009999999999997e-06 2024-09-17T07:31:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.260418+0000", 2024-09-17T07:31:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001246891 2024-09-17T07:31:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.260587+0000", 2024-09-17T07:31:13.007 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000168796 2024-09-17T07:31:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:31:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.260773+0000", 2024-09-17T07:31:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000186269 2024-09-17T07:31:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262459+0000", 2024-09-17T07:31:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0016858660000000001 2024-09-17T07:31:13.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.262962+0000", 2024-09-17T07:31:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00050283499999999998 2024-09-17T07:31:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.263015+0000", 2024-09-17T07:31:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3115999999999998e-05 2024-09-17T07:31:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.009 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:50.263060+0000", 2024-09-17T07:31:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5519000000000002e-05 2024-09-17T07:31:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:119884 3.18 3:1a94cd5b:::rbd_header.1183ce4061a0:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T07:31:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:58.210641+0000", 2024-09-17T07:31:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.732657746999999, 2024-09-17T07:31:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.010663832999999999, 2024-09-17T07:31:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.010 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 119884 2024-09-17T07:31:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.210641+0000", 2024-09-17T07:31:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.011 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.210641+0000", 2024-09-17T07:31:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.210644+0000", 2024-09-17T07:31:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2400000000000002e-06 2024-09-17T07:31:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.210650+0000", 2024-09-17T07:31:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9299999999999997e-06 2024-09-17T07:31:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.012 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.210663+0000", 2024-09-17T07:31:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2931e-05 2024-09-17T07:31:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.210671+0000", 2024-09-17T07:31:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7729999999999999e-06 2024-09-17T07:31:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.215471+0000", 2024-09-17T07:31:13.013 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.004799462 2024-09-17T07:31:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.215568+0000", 2024-09-17T07:31:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.7116999999999999e-05 2024-09-17T07:31:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 0", 2024-09-17T07:31:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.215951+0000", 2024-09-17T07:31:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000382791 2024-09-17T07:31:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:13.014 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.217671+0000", 2024-09-17T07:31:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0017203990000000001 2024-09-17T07:31:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.221194+0000", 2024-09-17T07:31:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035229990000000002 2024-09-17T07:31:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.221256+0000", 2024-09-17T07:31:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2114999999999997e-05 2024-09-17T07:31:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.015 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.221305+0000", 2024-09-17T07:31:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9076000000000002e-05 2024-09-17T07:31:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop_reply(osd.3.0:271 3.18 e149/133 ondisk, result = 0)", 2024-09-17T07:31:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:58.213549+0000", 2024-09-17T07:31:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.729749727, 2024-09-17T07:31:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033847119999999998, 2024-09-17T07:31:13.016 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213549+0000", 2024-09-17T07:31:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213549+0000", 2024-09-17T07:31:13.017 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213550+0000", 2024-09-17T07:31:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.06e-07 2024-09-17T07:31:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213554+0000", 2024-09-17T07:31:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-06 2024-09-17T07:31:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.018 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213564+0000", 2024-09-17T07:31:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.5109999999999992e-06 2024-09-17T07:31:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213573+0000", 2024-09-17T07:31:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2790000000000004e-06 2024-09-17T07:31:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.019 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.216900+0000", 2024-09-17T07:31:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0033272559999999998 2024-09-17T07:31:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.216918+0000", 2024-09-17T07:31:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7479e-05 2024-09-17T07:31:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.216934+0000", 2024-09-17T07:31:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6070999999999999e-05 2024-09-17T07:31:13.020 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(osd.1.0:2942 3.19 e149/108 3:9c9ac020:::rbd_data.1183ce4061a0.000000000000002a:e v 149'426, mlcod=139'423)", 2024-09-17T07:31:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:58.213882+0000", 2024-09-17T07:31:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.729417473, 2024-09-17T07:31:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003450023, 2024-09-17T07:31:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.021 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213882+0000", 2024-09-17T07:31:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213882+0000", 2024-09-17T07:31:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.022 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213882+0000", 2024-09-17T07:31:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.4299999999999999e-07 2024-09-17T07:31:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213889+0000", 2024-09-17T07:31:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.5780000000000004e-06 2024-09-17T07:31:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213899+0000", 2024-09-17T07:31:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.012e-05 2024-09-17T07:31:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.213906+0000", 2024-09-17T07:31:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3629999999999996e-06 2024-09-17T07:31:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.215356+0000", 2024-09-17T07:31:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0014503949999999999 2024-09-17T07:31:13.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.215378+0000", 2024-09-17T07:31:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1770999999999999e-05 2024-09-17T07:31:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.217286+0000", 2024-09-17T07:31:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0019081090000000001 2024-09-17T07:31:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.217332+0000", 2024-09-17T07:31:13.025 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5343999999999998e-05 2024-09-17T07:31:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:119974 3.15 3:abe93d35:::rbd_data.1183ce4061a0.0000000000000006:head [delete] snapc e=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T07:31:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:58.298891+0000", 2024-09-17T07:31:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.644407719, 2024-09-17T07:31:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0047785229999999998, 2024-09-17T07:31:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:13.026 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 119974 2024-09-17T07:31:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298891+0000", 2024-09-17T07:31:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298891+0000", 2024-09-17T07:31:13.027 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298893+0000", 2024-09-17T07:31:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4309999999999999e-06 2024-09-17T07:31:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298898+0000", 2024-09-17T07:31:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6169999999999998e-06 2024-09-17T07:31:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.028 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298913+0000", 2024-09-17T07:31:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.4803e-05 2024-09-17T07:31:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298923+0000", 2024-09-17T07:31:13.029 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.9010000000000006e-06 2024-09-17T07:31:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298995+0000", 2024-09-17T07:31:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2278000000000002e-05 2024-09-17T07:31:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.299125+0000", 2024-09-17T07:31:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00012987400000000001 2024-09-17T07:31:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:31:13.030 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.299253+0000", 2024-09-17T07:31:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000127732 2024-09-17T07:31:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.300124+0000", 2024-09-17T07:31:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00087097800000000001 2024-09-17T07:31:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.303580+0000", 2024-09-17T07:31:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0034557469999999999 2024-09-17T07:31:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:13.031 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.303630+0000", 2024-09-17T07:31:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0300000000000003e-05 2024-09-17T07:31:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.303670+0000", 2024-09-17T07:31:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9861999999999999e-05 2024-09-17T07:31:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:119978 3.7 3:e6f882de:::rbd_data.1183ce4061a0.000000000000000b:head [delete] snapc e=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T07:31:13.032 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:58.298977+0000", 2024-09-17T07:31:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.644321680000001, 2024-09-17T07:31:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0046515039999999999, 2024-09-17T07:31:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 119978 2024-09-17T07:31:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.033 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298977+0000", 2024-09-17T07:31:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298977+0000", 2024-09-17T07:31:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.034 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298978+0000", 2024-09-17T07:31:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:31:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298980+0000", 2024-09-17T07:31:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2069999999999998e-06 2024-09-17T07:31:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.298998+0000", 2024-09-17T07:31:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7489000000000001e-05 2024-09-17T07:31:13.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.299003+0000", 2024-09-17T07:31:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8000000000000004e-06 2024-09-17T07:31:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.299065+0000", 2024-09-17T07:31:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1758000000000001e-05 2024-09-17T07:31:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:13.036 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.299174+0000", 2024-09-17T07:31:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000109281 2024-09-17T07:31:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 7", 2024-09-17T07:31:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.299265+0000", 2024-09-17T07:31:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1098000000000001e-05 2024-09-17T07:31:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.300109+0000", 2024-09-17T07:31:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00084360399999999997 2024-09-17T07:31:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.037 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.303538+0000", 2024-09-17T07:31:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0034284910000000001 2024-09-17T07:31:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.303590+0000", 2024-09-17T07:31:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1984999999999998e-05 2024-09-17T07:31:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.303629+0000", 2024-09-17T07:31:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9359e-05 2024-09-17T07:31:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.038 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:119984 3.15 3:a9fb6512:::rbd_data.1183ce4061a0.0000000000000012:head [delete] snapc e=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T07:31:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:58.301188+0000", 2024-09-17T07:31:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.642110855, 2024-09-17T07:31:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00515352, 2024-09-17T07:31:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.039 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 119984 2024-09-17T07:31:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301188+0000", 2024-09-17T07:31:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301188+0000", 2024-09-17T07:31:13.040 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301190+0000", 2024-09-17T07:31:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.342e-06 2024-09-17T07:31:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301195+0000", 2024-09-17T07:31:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.3170000000000001e-06 2024-09-17T07:31:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.041 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301209+0000", 2024-09-17T07:31:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.3716e-05 2024-09-17T07:31:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301216+0000", 2024-09-17T07:31:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9929999999999998e-06 2024-09-17T07:31:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301293+0000", 2024-09-17T07:31:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7775000000000007e-05 2024-09-17T07:31:13.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301382+0000", 2024-09-17T07:31:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9087999999999998e-05 2024-09-17T07:31:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 4", 2024-09-17T07:31:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301500+0000", 2024-09-17T07:31:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000117176 2024-09-17T07:31:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:13.043 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.302158+0000", 2024-09-17T07:31:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00065859200000000005 2024-09-17T07:31:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.306286+0000", 2024-09-17T07:31:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0041275069999999999 2024-09-17T07:31:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.306319+0000", 2024-09-17T07:31:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.3563999999999999e-05 2024-09-17T07:31:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.044 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.306342+0000", 2024-09-17T07:31:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.245e-05 2024-09-17T07:31:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:119988 3.2 3:431863b4:::rbd_data.1183ce4061a0.0000000000000016:head [delete] snapc e=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T07:31:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:58.301677+0000", 2024-09-17T07:31:13.045 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 14.641621755999999, 2024-09-17T07:31:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0045348599999999999, 2024-09-17T07:31:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 119988 2024-09-17T07:31:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.046 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301677+0000", 2024-09-17T07:31:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301677+0000", 2024-09-17T07:31:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.047 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301678+0000", 2024-09-17T07:31:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1500000000000004e-07 2024-09-17T07:31:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301683+0000", 2024-09-17T07:31:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.4689999999999996e-06 2024-09-17T07:31:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301693+0000", 2024-09-17T07:31:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1989999999999995e-06 2024-09-17T07:31:13.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301700+0000", 2024-09-17T07:31:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.9229999999999996e-06 2024-09-17T07:31:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301759+0000", 2024-09-17T07:31:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9197999999999998e-05 2024-09-17T07:31:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.049 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301826+0000", 2024-09-17T07:31:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7096999999999998e-05 2024-09-17T07:31:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 5", 2024-09-17T07:31:13.050 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.301917+0000", 2024-09-17T07:31:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.1668000000000004e-05 2024-09-17T07:31:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.302591+0000", 2024-09-17T07:31:13.051 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00067327800000000003 2024-09-17T07:31:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.306145+0000", 2024-09-17T07:31:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003554504 2024-09-17T07:31:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.306181+0000", 2024-09-17T07:31:13.052 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.5952999999999999e-05 2024-09-17T07:31:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:58.306212+0000", 2024-09-17T07:31:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.0956e-05 2024-09-17T07:31:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(osd.1.0:2948 3.19 e150/108 3:9d0336d1:::rbd_data.11838dd88fc3.0000000000000003:d v 150'431, mlcod=149'428)", 2024-09-17T07:31:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:30:59.215746+0000", 2024-09-17T07:31:13.053 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 13.727552660000001, 2024-09-17T07:31:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0037243380000000002, 2024-09-17T07:31:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:59.215746+0000", 2024-09-17T07:31:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:59.215746+0000", 2024-09-17T07:31:13.054 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:59.215747+0000", 2024-09-17T07:31:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.2800000000000002e-07 2024-09-17T07:31:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:59.215756+0000", 2024-09-17T07:31:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.8259999999999999e-06 2024-09-17T07:31:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.055 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:59.215775+0000", 2024-09-17T07:31:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9417000000000001e-05 2024-09-17T07:31:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:59.215786+0000", 2024-09-17T07:31:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0977e-05 2024-09-17T07:31:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:59.215878+0000", 2024-09-17T07:31:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2274000000000005e-05 2024-09-17T07:31:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.056 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:59.215900+0000", 2024-09-17T07:31:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1746000000000002e-05 2024-09-17T07:31:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:59.219427+0000", 2024-09-17T07:31:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0035272179999999999 2024-09-17T07:31:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:30:59.219471+0000", 2024-09-17T07:31:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.3451999999999997e-05 2024-09-17T07:31:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.057 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_repop(osd.5.0:3974 3.5 e155/109 3:a09e8afc:::rbd_data.1183b59abaf1.0000000000000001:9 v 155'544, mlcod=152'540)", 2024-09-17T07:31:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:31:04.094285+0000", 2024-09-17T07:31:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 8.8490137529999995, 2024-09-17T07:31:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0055875910000000003, 2024-09-17T07:31:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.058 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:04.094285+0000", 2024-09-17T07:31:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:04.094285+0000", 2024-09-17T07:31:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:04.094286+0000", 2024-09-17T07:31:13.059 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.2000000000000001e-07 2024-09-17T07:31:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:04.094291+0000", 2024-09-17T07:31:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9429999999999999e-06 2024-09-17T07:31:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:04.094303+0000", 2024-09-17T07:31:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.2753e-05 2024-09-17T07:31:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.060 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:04.094315+0000", 2024-09-17T07:31:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1617000000000001e-05 2024-09-17T07:31:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:04.094391+0000", 2024-09-17T07:31:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5672000000000003e-05 2024-09-17T07:31:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:04.094418+0000", 2024-09-17T07:31:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.7426000000000001e-05 2024-09-17T07:31:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.061 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:04.099825+0000", 2024-09-17T07:31:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0054065420000000003 2024-09-17T07:31:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:04.099873+0000", 2024-09-17T07:31:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8318000000000001e-05 2024-09-17T07:31:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.062 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:120968 3.1e 3:7d957560:::rbd_header.11833dd73a5d:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-17T07:31:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:31:09.095786+0000", 2024-09-17T07:31:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.847512654, 2024-09-17T07:31:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0055751250000000002, 2024-09-17T07:31:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 120968 2024-09-17T07:31:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.063 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.095786+0000", 2024-09-17T07:31:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.095786+0000", 2024-09-17T07:31:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.064 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.095789+0000", 2024-09-17T07:31:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.88e-06 2024-09-17T07:31:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.095800+0000", 2024-09-17T07:31:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0659000000000001e-05 2024-09-17T07:31:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.095818+0000", 2024-09-17T07:31:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7765e-05 2024-09-17T07:31:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.065 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.095828+0000", 2024-09-17T07:31:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0787e-05 2024-09-17T07:31:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.095911+0000", 2024-09-17T07:31:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2925999999999999e-05 2024-09-17T07:31:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.066 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:31:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.095919+0000", 2024-09-17T07:31:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5530000000000004e-06 2024-09-17T07:31:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.098900+0000", 2024-09-17T07:31:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0029808959999999998 2024-09-17T07:31:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.099009+0000", 2024-09-17T07:31:13.067 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000109338 2024-09-17T07:31:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-17T07:31:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.099588+0000", 2024-09-17T07:31:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00057869099999999999 2024-09-17T07:31:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.100693+0000", 2024-09-17T07:31:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001105576 2024-09-17T07:31:13.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.101250+0000", 2024-09-17T07:31:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00055688500000000002 2024-09-17T07:31:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.101313+0000", 2024-09-17T07:31:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2869999999999999e-05 2024-09-17T07:31:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.069 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.101362+0000", 2024-09-17T07:31:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8299e-05 2024-09-17T07:31:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:120972 3.1e 3:7d957560:::rbd_header.11833dd73a5d:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-17T07:31:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:31:09.106007+0000", 2024-09-17T07:31:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 3.837291767, 2024-09-17T07:31:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0048855030000000002, 2024-09-17T07:31:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.070 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T07:31:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 120972 2024-09-17T07:31:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.106007+0000", 2024-09-17T07:31:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.071 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.106007+0000", 2024-09-17T07:31:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.106009+0000", 2024-09-17T07:31:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.092e-06 2024-09-17T07:31:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.106018+0000", 2024-09-17T07:31:13.072 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.3100000000000001e-06 2024-09-17T07:31:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.106034+0000", 2024-09-17T07:31:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.6351e-05 2024-09-17T07:31:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.106042+0000", 2024-09-17T07:31:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.6729999999999992e-06 2024-09-17T07:31:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.073 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.106103+0000", 2024-09-17T07:31:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.1186000000000007e-05 2024-09-17T07:31:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.106204+0000", 2024-09-17T07:31:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000101463 2024-09-17T07:31:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for subops from 6", 2024-09-17T07:31:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.106407+0000", 2024-09-17T07:31:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00020252200000000001 2024-09-17T07:31:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op_commit", 2024-09-17T07:31:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.107131+0000", 2024-09-17T07:31:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00072377599999999998 2024-09-17T07:31:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "sub_op_commit_rec", 2024-09-17T07:31:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.110794+0000", 2024-09-17T07:31:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.003663211 2024-09-17T07:31:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "commit_sent", 2024-09-17T07:31:13.075 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.110846+0000", 2024-09-17T07:31:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2438999999999997e-05 2024-09-17T07:31:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:31:09.110893+0000", 2024-09-17T07:31:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6480000000000002e-05 2024-09-17T07:31:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.076 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.076 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:13.078 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:13.077+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:13.078 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T07:31:13.100 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-17T07:31:13.179 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:13.178+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:13.219 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:13.219 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:31:13.219 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:31:13.219 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:13.229 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T07:31:13.280 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:13.279+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:13.339 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:13.339 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:31:13.339 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:31:13.339 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:31:13.339 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:13.348 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T07:31:13.381 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:13.381+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:13.482 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:13.480+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:13.497 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: "size": 20, 2024-09-17T07:31:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 600, 2024-09-17T07:31:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [ 2024-09-17T07:31:13.497 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:14510 3.18 3:18102f58:::rbd_header.11837e68aeab:head [watch reconnect cookie 140227462562544 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-17T07:31:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:31:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 135.36388547000001, 2024-09-17T07:31:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.006851223, 2024-09-17T07:31:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 14510 2024-09-17T07:31:13.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:31:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132608+0000", 2024-09-17T07:31:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.499 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132609+0000", 2024-09-17T07:31:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9750000000000001e-06 2024-09-17T07:31:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132617+0000", 2024-09-17T07:31:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8390000000000007e-06 2024-09-17T07:31:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132639+0000", 2024-09-17T07:31:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2161999999999999e-05 2024-09-17T07:31:13.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132657+0000", 2024-09-17T07:31:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7272999999999999e-05 2024-09-17T07:31:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132743+0000", 2024-09-17T07:31:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.5857000000000001e-05 2024-09-17T07:31:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.501 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "op must wait for map", 2024-09-17T07:31:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.132752+0000", 2024-09-17T07:31:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.4129999999999995e-06 2024-09-17T07:31:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137084+0000", 2024-09-17T07:31:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.0043318569999999997 2024-09-17T07:31:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "waiting for peered", 2024-09-17T07:31:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:58.137092+0000", 2024-09-17T07:31:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.8120000000000008e-06 2024-09-17T07:31:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138217+0000", 2024-09-17T07:31:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0011252310000001 2024-09-17T07:31:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.138388+0000", 2024-09-17T07:31:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00017089199999999999 2024-09-17T07:31:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:28:59.139459+0000", 2024-09-17T07:31:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.001070912 2024-09-17T07:31:13.503 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28208 3.18 3:1d6cdae8:::rbd_data.1183ce4061a0.000000000000001f:e [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:31:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.057918766, 2024-09-17T07:31:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032104042999999999, 2024-09-17T07:31:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "started", 2024-09-17T07:31:13.504 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28208 2024-09-17T07:31:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:31:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.505 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438574+0000", 2024-09-17T07:31:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438575+0000", 2024-09-17T07:31:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6899999999999997e-07 2024-09-17T07:31:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438577+0000", 2024-09-17T07:31:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0990000000000001e-06 2024-09-17T07:31:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.506 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438583+0000", 2024-09-17T07:31:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4779999999999996e-06 2024-09-17T07:31:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438588+0000", 2024-09-17T07:31:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7999999999999998e-06 2024-09-17T07:31:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.507 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.469915+0000", 2024-09-17T07:31:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031326568999999999 2024-09-17T07:31:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "started", 2024-09-17T07:31:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470064+0000", 2024-09-17T07:31:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.000149238 2024-09-17T07:31:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470678+0000", 2024-09-17T07:31:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.00061428999999999997 2024-09-17T07:31:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.508 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28210 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:31:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.05789665099999, 2024-09-17T07:31:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032143100000000001, 2024-09-17T07:31:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.509 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28210 2024-09-17T07:31:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:31:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438596+0000", 2024-09-17T07:31:13.510 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438597+0000", 2024-09-17T07:31:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:31:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438599+0000", 2024-09-17T07:31:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2539999999999999e-06 2024-09-17T07:31:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.511 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438605+0000", 2024-09-17T07:31:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2559999999999999e-06 2024-09-17T07:31:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438610+0000", 2024-09-17T07:31:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7469999999999996e-06 2024-09-17T07:31:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470729+0000", 2024-09-17T07:31:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032118563000000003 2024-09-17T07:31:13.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470739+0000", 2024-09-17T07:31:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0834e-05 2024-09-17T07:31:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28211 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:31:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.057872547, 2024-09-17T07:31:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032169573999999999, 2024-09-17T07:31:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.513 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28211 2024-09-17T07:31:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:31:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.514 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438620+0000", 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438621+0000", 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.46e-07 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438623+0000", 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1859999999999999e-06 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438629+0000", 2024-09-17T07:31:13.515 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3489999999999997e-06 2024-09-17T07:31:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438637+0000", 2024-09-17T07:31:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.5809999999999999e-06 2024-09-17T07:31:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470784+0000", 2024-09-17T07:31:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032147000000000002 2024-09-17T07:31:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.516 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470790+0000", 2024-09-17T07:31:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0120000000000002e-06 2024-09-17T07:31:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28212 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:31:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.057848458, 2024-09-17T07:31:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032186353000000001, 2024-09-17T07:31:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.517 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28212 2024-09-17T07:31:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:31:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:31:13.518 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438645+0000", 2024-09-17T07:31:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.06e-07 2024-09-17T07:31:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438647+0000", 2024-09-17T07:31:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.9570000000000001e-06 2024-09-17T07:31:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.519 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438654+0000", 2024-09-17T07:31:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6290000000000004e-06 2024-09-17T07:31:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438662+0000", 2024-09-17T07:31:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0059999999999992e-06 2024-09-17T07:31:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470822+0000", 2024-09-17T07:31:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032159883 2024-09-17T07:31:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.520 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470831+0000", 2024-09-17T07:31:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.3719999999999994e-06 2024-09-17T07:31:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28215 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:31:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.057824039, 2024-09-17T07:31:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032234991999999997, 2024-09-17T07:31:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.521 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28215 2024-09-17T07:31:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:31:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:31:13.522 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438669+0000", 2024-09-17T07:31:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6100000000000001e-07 2024-09-17T07:31:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438672+0000", 2024-09-17T07:31:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.12e-06 2024-09-17T07:31:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438679+0000", 2024-09-17T07:31:13.523 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.0899999999999999e-06 2024-09-17T07:31:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438684+0000", 2024-09-17T07:31:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.1370000000000001e-06 2024-09-17T07:31:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470892+0000", 2024-09-17T07:31:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032208647999999999 2024-09-17T07:31:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470904+0000", 2024-09-17T07:31:13.524 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1535999999999999e-05 2024-09-17T07:31:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28217 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:31:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.05780233900001, 2024-09-17T07:31:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032267135000000002, 2024-09-17T07:31:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.525 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28217 2024-09-17T07:31:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:31:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.526 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:31:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.528 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438691+0000", 2024-09-17T07:31:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4999999999999998e-07 2024-09-17T07:31:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438693+0000", 2024-09-17T07:31:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0020000000000001e-06 2024-09-17T07:31:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438699+0000", 2024-09-17T07:31:13.529 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.3799999999999999e-06 2024-09-17T07:31:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438726+0000", 2024-09-17T07:31:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.6914e-05 2024-09-17T07:31:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470946+0000", 2024-09-17T07:31:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032219614000000001 2024-09-17T07:31:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.470958+0000", 2024-09-17T07:31:13.530 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1775e-05 2024-09-17T07:31:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28220 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:31:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.057757607, 2024-09-17T07:31:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032276283000000003, 2024-09-17T07:31:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.531 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28220 2024-09-17T07:31:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:31:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438735+0000", 2024-09-17T07:31:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.532 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438736+0000", 2024-09-17T07:31:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.2699999999999999e-07 2024-09-17T07:31:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438743+0000", 2024-09-17T07:31:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.2429999999999999e-06 2024-09-17T07:31:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438751+0000", 2024-09-17T07:31:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7770000000000002e-06 2024-09-17T07:31:13.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438756+0000", 2024-09-17T07:31:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5380000000000002e-06 2024-09-17T07:31:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471005+0000", 2024-09-17T07:31:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032248532000000003 2024-09-17T07:31:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.534 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471012+0000", 2024-09-17T07:31:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.6660000000000002e-06 2024-09-17T07:31:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28222 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:31:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.057729141, 2024-09-17T07:31:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032291780999999999, 2024-09-17T07:31:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.535 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28222 2024-09-17T07:31:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:31:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.536 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:31:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438764+0000", 2024-09-17T07:31:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.8299999999999998e-07 2024-09-17T07:31:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438766+0000", 2024-09-17T07:31:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.548e-06 2024-09-17T07:31:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.537 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438769+0000", 2024-09-17T07:31:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6720000000000002e-06 2024-09-17T07:31:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438772+0000", 2024-09-17T07:31:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.615e-06 2024-09-17T07:31:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471047+0000", 2024-09-17T07:31:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032275314999999999 2024-09-17T07:31:13.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471056+0000", 2024-09-17T07:31:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2479999999999996e-06 2024-09-17T07:31:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28227 3.18 3.175b36b8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:31:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.057611889, 2024-09-17T07:31:13.539 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032192408999999998, 2024-09-17T07:31:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28227 2024-09-17T07:31:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:31:13.540 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438881+0000", 2024-09-17T07:31:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438882+0000", 2024-09-17T07:31:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.0999999999999998e-07 2024-09-17T07:31:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.541 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438884+0000", 2024-09-17T07:31:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.114e-06 2024-09-17T07:31:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438890+0000", 2024-09-17T07:31:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.2249999999999997e-06 2024-09-17T07:31:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.438895+0000", 2024-09-17T07:31:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.0159999999999999e-06 2024-09-17T07:31:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.542 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471070+0000", 2024-09-17T07:31:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032174769999999998 2024-09-17T07:31:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471074+0000", 2024-09-17T07:31:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.6739999999999999e-06 2024-09-17T07:31:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.543 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28295 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:31:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.05718578699999, 2024-09-17T07:31:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031955428000000001, 2024-09-17T07:31:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28295 2024-09-17T07:31:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.544 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:31:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439307+0000", 2024-09-17T07:31:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439308+0000", 2024-09-17T07:31:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 3.9999999999999998e-07 2024-09-17T07:31:13.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439309+0000", 2024-09-17T07:31:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8360000000000001e-06 2024-09-17T07:31:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439315+0000", 2024-09-17T07:31:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.5489999999999997e-06 2024-09-17T07:31:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439320+0000", 2024-09-17T07:31:13.546 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5809999999999998e-06 2024-09-17T07:31:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471254+0000", 2024-09-17T07:31:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031934044000000002 2024-09-17T07:31:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471263+0000", 2024-09-17T07:31:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0180000000000007e-06 2024-09-17T07:31:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28296 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:31:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.057166116, 2024-09-17T07:31:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031950098000000003, 2024-09-17T07:31:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28296 2024-09-17T07:31:13.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:31:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:31:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439327+0000", 2024-09-17T07:31:13.549 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.6600000000000002e-07 2024-09-17T07:31:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439330+0000", 2024-09-17T07:31:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.3939999999999999e-06 2024-09-17T07:31:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439338+0000", 2024-09-17T07:31:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.2800000000000003e-06 2024-09-17T07:31:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.550 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439343+0000", 2024-09-17T07:31:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9559999999999996e-06 2024-09-17T07:31:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471273+0000", 2024-09-17T07:31:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031929539999999999 2024-09-17T07:31:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471277+0000", 2024-09-17T07:31:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.4619999999999998e-06 2024-09-17T07:31:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.551 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28297 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:31:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.057143252, 2024-09-17T07:31:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031952113999999997, 2024-09-17T07:31:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.552 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28297 2024-09-17T07:31:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:31:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:31:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439350+0000", 2024-09-17T07:31:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7999999999999995e-07 2024-09-17T07:31:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439352+0000", 2024-09-17T07:31:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8700000000000001e-06 2024-09-17T07:31:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439360+0000", 2024-09-17T07:31:13.554 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.3560000000000004e-06 2024-09-17T07:31:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439365+0000", 2024-09-17T07:31:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.9640000000000002e-06 2024-09-17T07:31:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471293+0000", 2024-09-17T07:31:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031928335000000002 2024-09-17T07:31:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471302+0000", 2024-09-17T07:31:13.555 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.0089999999999996e-06 2024-09-17T07:31:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28299 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:31:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.057121951, 2024-09-17T07:31:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031976220999999999, 2024-09-17T07:31:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.556 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28299 2024-09-17T07:31:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:31:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439371+0000", 2024-09-17T07:31:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439372+0000", 2024-09-17T07:31:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.8899999999999999e-07 2024-09-17T07:31:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439373+0000", 2024-09-17T07:31:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.7799999999999999e-06 2024-09-17T07:31:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.558 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439396+0000", 2024-09-17T07:31:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.2951000000000001e-05 2024-09-17T07:31:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439405+0000", 2024-09-17T07:31:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.6119999999999999e-06 2024-09-17T07:31:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471338+0000", 2024-09-17T07:31:13.559 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031933028000000002 2024-09-17T07:31:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471347+0000", 2024-09-17T07:31:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2609999999999999e-06 2024-09-17T07:31:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28300 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:31:13.560 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.057074163, 2024-09-17T07:31:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031980291000000001, 2024-09-17T07:31:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28300 2024-09-17T07:31:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:31:13.561 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439419+0000", 2024-09-17T07:31:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439420+0000", 2024-09-17T07:31:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.0500000000000002e-07 2024-09-17T07:31:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.562 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439424+0000", 2024-09-17T07:31:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1049999999999997e-06 2024-09-17T07:31:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439433+0000", 2024-09-17T07:31:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 8.9770000000000006e-06 2024-09-17T07:31:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439438+0000", 2024-09-17T07:31:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.6400000000000002e-06 2024-09-17T07:31:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.563 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471388+0000", 2024-09-17T07:31:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031949191000000002 2024-09-17T07:31:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471399+0000", 2024-09-17T07:31:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.1573000000000001e-05 2024-09-17T07:31:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28301 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:31:13.564 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.057046941, 2024-09-17T07:31:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032003998999999998, 2024-09-17T07:31:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28301 2024-09-17T07:31:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:31:13.565 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:31:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439446+0000", 2024-09-17T07:31:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.32e-07 2024-09-17T07:31:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.566 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439449+0000", 2024-09-17T07:31:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1260000000000001e-06 2024-09-17T07:31:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439455+0000", 2024-09-17T07:31:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.7599999999999997e-06 2024-09-17T07:31:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439460+0000", 2024-09-17T07:31:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8389999999999997e-06 2024-09-17T07:31:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.567 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471439+0000", 2024-09-17T07:31:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.031979234000000002 2024-09-17T07:31:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471450+0000", 2024-09-17T07:31:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0608000000000001e-05 2024-09-17T07:31:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28302 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.568 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:31:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.057026554, 2024-09-17T07:31:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032031693999999999, 2024-09-17T07:31:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28302 2024-09-17T07:31:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.569 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:31:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439466+0000", 2024-09-17T07:31:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439467+0000", 2024-09-17T07:31:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.1199999999999998e-07 2024-09-17T07:31:13.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439469+0000", 2024-09-17T07:31:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.911e-06 2024-09-17T07:31:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439475+0000", 2024-09-17T07:31:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9900000000000002e-06 2024-09-17T07:31:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.571 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439480+0000", 2024-09-17T07:31:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8029999999999996e-06 2024-09-17T07:31:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471489+0000", 2024-09-17T07:31:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032009282 2024-09-17T07:31:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471498+0000", 2024-09-17T07:31:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 9.2960000000000004e-06 2024-09-17T07:31:13.572 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28303 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:31:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.057006849, 2024-09-17T07:31:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032070695000000003, 2024-09-17T07:31:13.573 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28303 2024-09-17T07:31:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:31:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439486+0000", 2024-09-17T07:31:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439487+0000", 2024-09-17T07:31:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.51e-07 2024-09-17T07:31:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439489+0000", 2024-09-17T07:31:13.575 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.1390000000000002e-06 2024-09-17T07:31:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439495+0000", 2024-09-17T07:31:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.942e-06 2024-09-17T07:31:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439500+0000", 2024-09-17T07:31:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7479999999999999e-06 2024-09-17T07:31:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.576 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471547+0000", 2024-09-17T07:31:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032046191000000002 2024-09-17T07:31:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471557+0000", 2024-09-17T07:31:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.0224000000000001e-05 2024-09-17T07:31:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.577 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28304 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:31:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.05698377100001, 2024-09-17T07:31:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032082094999999998, 2024-09-17T07:31:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28304 2024-09-17T07:31:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.578 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:31:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439509+0000", 2024-09-17T07:31:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439510+0000", 2024-09-17T07:31:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.5999999999999999e-07 2024-09-17T07:31:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439512+0000", 2024-09-17T07:31:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 2.0619999999999999e-06 2024-09-17T07:31:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439518+0000", 2024-09-17T07:31:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 6.4799999999999998e-06 2024-09-17T07:31:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439523+0000", 2024-09-17T07:31:13.580 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.8969999999999999e-06 2024-09-17T07:31:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471584+0000", 2024-09-17T07:31:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032060456000000001 2024-09-17T07:31:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471591+0000", 2024-09-17T07:31:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.7400000000000004e-06 2024-09-17T07:31:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.581 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "description": "osd_op(client.4483.0:28306 3.18 3.cf27aff8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-17T07:31:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "initiated_at": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:31:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 125.056963209, 2024-09-17T07:31:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032094224999999997, 2024-09-17T07:31:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "continuous": false, 2024-09-17T07:31:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "type_data": { 2024-09-17T07:31:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "flag_point": "reached pg", 2024-09-17T07:31:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "client_info": { 2024-09-17T07:31:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "client": "client.4483", 2024-09-17T07:31:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "client_addr": "172.21.15.45:0/2166897717", 2024-09-17T07:31:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tid": 28306 2024-09-17T07:31:13.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "events": [ 2024-09-17T07:31:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "initiated", 2024-09-17T07:31:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:31:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "header_read", 2024-09-17T07:31:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:31:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0 2024-09-17T07:31:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "throttled", 2024-09-17T07:31:13.583 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439530+0000", 2024-09-17T07:31:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.7700000000000004e-07 2024-09-17T07:31:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "all_read", 2024-09-17T07:31:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439532+0000", 2024-09-17T07:31:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 1.8729999999999999e-06 2024-09-17T07:31:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "dispatched", 2024-09-17T07:31:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439538+0000", 2024-09-17T07:31:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 5.9499999999999998e-06 2024-09-17T07:31:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.584 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "queued_for_pg", 2024-09-17T07:31:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.439543+0000", 2024-09-17T07:31:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 4.7280000000000003e-06 2024-09-17T07:31:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "reached_pg", 2024-09-17T07:31:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471617+0000", 2024-09-17T07:31:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 0.032073628999999999 2024-09-17T07:31:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-17T07:31:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-17T07:31:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "event": "done", 2024-09-17T07:31:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "time": "2024-09-17T07:29:08.471624+0000", 2024-09-17T07:31:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: "duration": 7.4680000000000003e-06 2024-09-17T07:31:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.585 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-17T07:31:13.586 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-17T07:31:13.586 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:13.587 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T07:31:13.587 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:13.586+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:13.688 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:13.687+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:13.736 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:13.736 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:31:13.736 INFO:teuthology.orchestra.run.smithi045.stdout: "num_ops": 0 2024-09-17T07:31:13.736 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:13.744 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T07:31:13.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T07:31:13.829 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-09-17T07:31:13.828+0000 7f29b7152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33356) UID: 0 2024-09-17T07:31:13.890 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-17T07:31:13.889+0000 7fdb74181640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33363) UID: 0 2024-09-17T07:31:13.895 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "ops": [], 2024-09-17T07:31:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "complaint_time": 30, 2024-09-17T07:31:13.895 INFO:teuthology.orchestra.run.smithi045.stdout: "num_blocked_ops": 0 2024-09-17T07:31:13.895 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:13.905 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T07:31:13.911 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0: {} 2024-09-17T07:31:13.912 INFO:teuthology.orchestra.run.smithi045.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T07:31:13.929 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1: {} 2024-09-17T07:31:13.929 INFO:teuthology.orchestra.run.smithi045.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T07:31:13.946 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2: {} 2024-09-17T07:31:13.946 INFO:teuthology.orchestra.run.smithi045.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T07:31:13.963 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3: {} 2024-09-17T07:31:13.963 INFO:teuthology.orchestra.run.smithi045.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T07:31:13.980 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4: {} 2024-09-17T07:31:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T07:31:13.990 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-09-17T07:31:13.989+0000 7f2992473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33346) UID: 0 2024-09-17T07:31:13.998 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5: {} 2024-09-17T07:31:13.998 INFO:teuthology.orchestra.run.smithi045.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T07:31:14.015 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6: {} 2024-09-17T07:31:14.015 INFO:teuthology.orchestra.run.smithi045.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T07:31:14.020 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:14.020 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:14.029 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T07:31:14.033 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7: {} 2024-09-17T07:31:14.033 INFO:teuthology.orchestra.run.smithi045.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T07:31:14.092 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:14.090+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:14.148 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:14.149 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:14.158 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T07:31:14.190 INFO:teuthology.orchestra.run.smithi045.stdout:All operations completed A-OK! 2024-09-17T07:31:14.192 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-09-17T07:31:14.190+0000 7f8ea0d69640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33341) UID: 0 2024-09-17T07:31:14.201 DEBUG:teuthology.parallel:result is None 2024-09-17T07:31:14.201 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-17T07:31:14.211 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-17T07:31:14.218 INFO:tasks.thrashosds:joining thrashosds 2024-09-17T07:31:14.296 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:14.296 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:14.305 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T07:31:14.412 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:14.413 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:14.421 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T07:31:14.534 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:14.534 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:14.543 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T07:31:14.646 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:14.646 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:14.655 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T07:31:14.758 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:14.758 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:14.766 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T07:31:14.781 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:31:14.869 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:14.869 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:14.877 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T07:31:14.979 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:14.980 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:14.988 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T07:31:15.085 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:15.086 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:31:15.091 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:15.091 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:15.098 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-17T07:31:14.782671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":9803596,"num_objects":630,"num_object_clones":0,"num_object_copies":1260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":630,"num_whiteouts":607,"num_read":166040,"num_read_kb":1208192,"num_write":23257,"num_write_kb":794389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6841,"num_bytes_recovered":667554936,"num_keys_recovered":891,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20596,"ondisk_log_size":20596,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":435156,"kb_used_data":16008,"kb_used_omap":27,"kb_used_meta":418788,"kb_avail":754539564,"statfs":{"total":773094113280,"available":772648513536,"internally_reserved":0,"allocated":16392192,"data_stored":9551664,"data_compressed":88196,"data_compressed_allocated":172032,"data_compressed_original":1605632,"omap_allocated":28217,"internal_metadata":428839367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-122346257,"num_objects":-384,"num_object_clones":-156,"num_object_copies":-768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-384,"num_whiteouts":-153,"num_read":1259,"num_read_kb":1079,"num_write":256,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000648"},"pg_stats":[{"pgid":"3.1f","version":"164'873","reported_seq":4566,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-17T07:31:13.236801+0000","last_change":"2024-09-17T07:31:13.138051+0000","last_active":"2024-09-17T07:31:13.236801+0000","last_peered":"2024-09-17T07:31:13.236801+0000","last_clean":"2024-09-17T07:31:13.236801+0000","last_became_active":"2024-09-17T07:30:43.092456+0000","last_became_peered":"2024-09-17T07:30:43.092456+0000","last_unstale":"2024-09-17T07:31:13.236801+0000","last_undegraded":"2024-09-17T07:31:13.236801+0000","last_fullsized":"2024-09-17T07:31:13.236801+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:06.418445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001624108,"stat_sum":{"num_bytes":16,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":5801,"num_read_kb":34310,"num_write":996,"num_write_kb":41977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":26159662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"5"}]},{"pgid":"3.1e","version":"161'551","reported_seq":3224,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-17T07:31:12.098409+0000","last_change":"2024-09-17T07:31:12.100077+0000","last_active":"2024-09-17T07:31:12.098409+0000","last_peered":"2024-09-17T07:31:12.098409+0000","last_clean":"2024-09-17T07:31:12.098409+0000","last_became_active":"2024-09-17T07:30:19.087213+0000","last_became_peered":"2024-09-17T07:30:19.087213+0000","last_unstale":"2024-09-17T07:31:12.098409+0000","last_undegraded":"2024-09-17T07:31:12.098409+0000","last_fullsized":"2024-09-17T07:31:12.098409+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:35:18.715153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0080337510000000004,"stat_sum":{"num_bytes":1154437,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":34,"num_read":3721,"num_read_kb":30298,"num_write":525,"num_write_kb":15594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":15743718,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.1d","version":"163'550","reported_seq":2940,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.107929+0000","last_change":"2024-09-17T07:31:14.107929+0000","last_active":"2024-09-17T07:31:14.107929+0000","last_peered":"2024-09-17T07:31:14.107929+0000","last_clean":"2024-09-17T07:31:14.107929+0000","last_became_active":"2024-09-17T07:30:30.096044+0000","last_became_peered":"2024-09-17T07:30:30.096044+0000","last_unstale":"2024-09-17T07:31:14.107929+0000","last_undegraded":"2024-09-17T07:31:14.107929+0000","last_fullsized":"2024-09-17T07:31:14.107929+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:04.003098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016933180000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":1948,"num_read_kb":28759,"num_write":579,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":27967854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"3.1c","version":"165'813","reported_seq":6166,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.154587+0000","last_change":"2024-09-17T07:31:14.106892+0000","last_active":"2024-09-17T07:31:14.154587+0000","last_peered":"2024-09-17T07:31:14.154587+0000","last_clean":"2024-09-17T07:31:14.154587+0000","last_became_active":"2024-09-17T07:30:31.098495+0000","last_became_peered":"2024-09-17T07:30:31.098495+0000","last_unstale":"2024-09-17T07:31:14.154587+0000","last_undegraded":"2024-09-17T07:31:14.154587+0000","last_fullsized":"2024-09-17T07:31:14.154587+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:50.743389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010655949999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":7523,"num_read_kb":42454,"num_write":1061,"num_write_kb":38721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":26661650,"num_keys_recovered":180,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"3.1b","version":"164'589","reported_seq":5009,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-17T07:31:13.253746+0000","last_change":"2024-09-17T07:31:13.137654+0000","last_active":"2024-09-17T07:31:13.253746+0000","last_peered":"2024-09-17T07:31:13.253746+0000","last_clean":"2024-09-17T07:31:13.253746+0000","last_became_active":"2024-09-17T07:31:03.094058+0000","last_became_peered":"2024-09-17T07:31:03.094058+0000","last_unstale":"2024-09-17T07:31:13.253746+0000","last_undegraded":"2024-09-17T07:31:13.253746+0000","last_fullsized":"2024-09-17T07:31:13.253746+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:37.295997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001281325,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":5709,"num_read_kb":52380,"num_write":585,"num_write_kb":22878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":674,"num_bytes_recovered":91784745,"num_keys_recovered":126,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"5"}]},{"pgid":"3.1a","version":"163'920","reported_seq":14612,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.139478+0000","last_change":"2024-09-17T07:31:14.108273+0000","last_active":"2024-09-17T07:31:14.139478+0000","last_peered":"2024-09-17T07:31:14.139478+0000","last_clean":"2024-09-17T07:31:14.139478+0000","last_became_active":"2024-09-17T07:30:31.090258+0000","last_became_peered":"2024-09-17T07:30:31.090258+0000","last_unstale":"2024-09-17T07:31:14.139478+0000","last_undegraded":"2024-09-17T07:31:14.139478+0000","last_fullsized":"2024-09-17T07:31:14.139478+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:39:33.685952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001697264,"stat_sum":{"num_bytes":19,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":16734,"num_read_kb":60999,"num_write":1188,"num_write_kb":20597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":19561990,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"3.19","version":"164'439","reported_seq":5639,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.188719+0000","last_change":"2024-09-17T07:31:14.189296+0000","last_active":"2024-09-17T07:31:14.188719+0000","last_peered":"2024-09-17T07:31:14.188719+0000","last_clean":"2024-09-17T07:31:14.188719+0000","last_became_active":"2024-09-17T07:30:18.083918+0000","last_became_peered":"2024-09-17T07:30:18.083918+0000","last_unstale":"2024-09-17T07:31:14.188719+0000","last_undegraded":"2024-09-17T07:31:14.188719+0000","last_fullsized":"2024-09-17T07:31:14.188719+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:26.983426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000820972,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4352,"num_read_kb":37354,"num_write":400,"num_write_kb":43089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":25536146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"3.18","version":"161'763","reported_seq":4610,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-17T07:31:12.091813+0000","last_change":"2024-09-17T07:31:12.093440+0000","last_active":"2024-09-17T07:31:12.091813+0000","last_peered":"2024-09-17T07:31:12.091813+0000","last_clean":"2024-09-17T07:31:12.091813+0000","last_became_active":"2024-09-17T07:30:43.091692+0000","last_became_peered":"2024-09-17T07:30:43.091692+0000","last_unstale":"2024-09-17T07:31:12.091813+0000","last_undegraded":"2024-09-17T07:31:12.091813+0000","last_fullsized":"2024-09-17T07:31:12.091813+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:34:10.078196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019830009999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5969,"num_read_kb":25327,"num_write":792,"num_write_kb":16603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":29456184,"num_keys_recovered":176,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.17","version":"163'436","reported_seq":4176,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-17T07:31:13.138350+0000","last_change":"2024-09-17T07:31:13.139008+0000","last_active":"2024-09-17T07:31:13.138350+0000","last_peered":"2024-09-17T07:31:13.138350+0000","last_clean":"2024-09-17T07:31:13.138350+0000","last_became_active":"2024-09-17T07:28:43.080428+0000","last_became_peered":"2024-09-17T07:28:43.080428+0000","last_unstale":"2024-09-17T07:31:13.138350+0000","last_undegraded":"2024-09-17T07:31:13.138350+0000","last_fullsized":"2024-09-17T07:31:13.138350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:00.035256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023802580000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4961,"num_read_kb":22717,"num_write":365,"num_write_kb":13531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"5"}]},{"pgid":"3.16","version":"164'513","reported_seq":4145,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.106362+0000","last_change":"2024-09-17T07:31:14.106949+0000","last_active":"2024-09-17T07:31:14.106362+0000","last_peered":"2024-09-17T07:31:14.106362+0000","last_clean":"2024-09-17T07:31:14.106362+0000","last_became_active":"2024-09-17T07:28:43.082313+0000","last_became_peered":"2024-09-17T07:28:43.082313+0000","last_unstale":"2024-09-17T07:31:14.106362+0000","last_undegraded":"2024-09-17T07:31:14.106362+0000","last_fullsized":"2024-09-17T07:31:14.106362+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:35.352537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045261,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6140,"num_read_kb":58359,"num_write":449,"num_write_kb":31855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"3.15","version":"164'390","reported_seq":4460,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.174491+0000","last_change":"2024-09-17T07:31:14.175336+0000","last_active":"2024-09-17T07:31:14.174491+0000","last_peered":"2024-09-17T07:31:14.174491+0000","last_clean":"2024-09-17T07:31:14.174491+0000","last_became_active":"2024-09-17T07:31:08.094940+0000","last_became_peered":"2024-09-17T07:31:08.094940+0000","last_unstale":"2024-09-17T07:31:14.174491+0000","last_undegraded":"2024-09-17T07:31:14.174491+0000","last_fullsized":"2024-09-17T07:31:14.174491+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:58.292675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011121970000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3586,"num_read_kb":32352,"num_write":371,"num_write_kb":22781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":431,"num_bytes_recovered":60567103,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"3.14","version":"163'465","reported_seq":2081,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.106979+0000","last_change":"2024-09-17T07:31:14.108307+0000","last_active":"2024-09-17T07:31:14.106979+0000","last_peered":"2024-09-17T07:31:14.106979+0000","last_clean":"2024-09-17T07:31:14.106979+0000","last_became_active":"2024-09-17T07:30:30.095364+0000","last_became_peered":"2024-09-17T07:30:30.095364+0000","last_unstale":"2024-09-17T07:31:14.106979+0000","last_undegraded":"2024-09-17T07:31:14.106979+0000","last_fullsized":"2024-09-17T07:31:14.106979+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:46.743327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019072220000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1414,"num_read_kb":17974,"num_write":482,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":16518806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"2.7","version":"0'0","reported_seq":315,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-17T07:31:13.200980+0000","last_change":"2024-09-17T07:31:05.098106+0000","last_active":"2024-09-17T07:31:13.200980+0000","last_peered":"2024-09-17T07:31:13.200980+0000","last_clean":"2024-09-17T07:31:13.200980+0000","last_became_active":"2024-09-17T07:31:05.097821+0000","last_became_peered":"2024-09-17T07:31:05.097821+0000","last_unstale":"2024-09-17T07:31:13.200980+0000","last_undegraded":"2024-09-17T07:31:13.200980+0000","last_fullsized":"2024-09-17T07:31:13.200980+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:12.297890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"164'1029","reported_seq":2710,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-17T07:31:13.208565+0000","last_change":"2024-09-17T07:31:13.138290+0000","last_active":"2024-09-17T07:31:13.208565+0000","last_peered":"2024-09-17T07:31:13.208565+0000","last_clean":"2024-09-17T07:31:13.208565+0000","last_became_active":"2024-09-17T07:28:43.079229+0000","last_became_peered":"2024-09-17T07:28:43.079229+0000","last_unstale":"2024-09-17T07:31:13.208565+0000","last_undegraded":"2024-09-17T07:31:13.208565+0000","last_fullsized":"2024-09-17T07:31:13.208565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:56:29.022619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018600769999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4625,"num_read_kb":48744,"num_write":1479,"num_write_kb":29297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.188395+0000","last_change":"2024-09-17T07:31:01.268641+0000","last_active":"2024-09-17T07:31:14.188395+0000","last_peered":"2024-09-17T07:31:14.188395+0000","last_clean":"2024-09-17T07:31:14.188395+0000","last_became_active":"2024-09-17T07:31:01.268257+0000","last_became_peered":"2024-09-17T07:31:01.268257+0000","last_unstale":"2024-09-17T07:31:14.188395+0000","last_undegraded":"2024-09-17T07:31:14.188395+0000","last_fullsized":"2024-09-17T07:31:14.188395+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:04.188817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"162'575","reported_seq":3640,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-17T07:31:12.093526+0000","last_change":"2024-09-17T07:31:12.094905+0000","last_active":"2024-09-17T07:31:12.093526+0000","last_peered":"2024-09-17T07:31:12.093526+0000","last_clean":"2024-09-17T07:31:12.093526+0000","last_became_active":"2024-09-17T07:30:19.087787+0000","last_became_peered":"2024-09-17T07:30:19.087787+0000","last_unstale":"2024-09-17T07:31:12.093526+0000","last_undegraded":"2024-09-17T07:31:12.093526+0000","last_fullsized":"2024-09-17T07:31:12.093526+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:38.212511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032327290000000002,"stat_sum":{"num_bytes":1703936,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":20,"num_read":3995,"num_read_kb":36995,"num_write":612,"num_write_kb":22622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":15169480,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"2.5","version":"0'0","reported_seq":306,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.174520+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:31:14.174520+0000","last_peered":"2024-09-17T07:31:14.174520+0000","last_clean":"2024-09-17T07:31:14.174520+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:31:14.174520+0000","last_undegraded":"2024-09-17T07:31:14.174520+0000","last_fullsized":"2024-09-17T07:31:14.174520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"163'1063","reported_seq":3557,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.188749+0000","last_change":"2024-09-17T07:31:14.189299+0000","last_active":"2024-09-17T07:31:14.188749+0000","last_peered":"2024-09-17T07:31:14.188749+0000","last_clean":"2024-09-17T07:31:14.188749+0000","last_became_active":"2024-09-17T07:28:43.239387+0000","last_became_peered":"2024-09-17T07:28:43.239387+0000","last_unstale":"2024-09-17T07:31:14.188749+0000","last_undegraded":"2024-09-17T07:31:14.188749+0000","last_fullsized":"2024-09-17T07:31:14.188749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1063,"log_dups_size":0,"ondisk_log_size":1063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:09.407494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080983699999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6807,"num_read_kb":16873,"num_write":1444,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.105615+0000","last_change":"2024-09-17T07:30:15.200774+0000","last_active":"2024-09-17T07:31:14.105615+0000","last_peered":"2024-09-17T07:31:14.105615+0000","last_clean":"2024-09-17T07:31:14.105615+0000","last_became_active":"2024-09-17T07:30:15.200470+0000","last_became_peered":"2024-09-17T07:30:15.200470+0000","last_unstale":"2024-09-17T07:31:14.105615+0000","last_undegraded":"2024-09-17T07:31:14.105615+0000","last_fullsized":"2024-09-17T07:31:14.105615+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:58.984439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"164'569","reported_seq":3257,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.106963+0000","last_change":"2024-09-17T07:31:14.107519+0000","last_active":"2024-09-17T07:31:14.106963+0000","last_peered":"2024-09-17T07:31:14.106963+0000","last_clean":"2024-09-17T07:31:14.106963+0000","last_became_active":"2024-09-17T07:30:19.085184+0000","last_became_peered":"2024-09-17T07:30:19.085184+0000","last_unstale":"2024-09-17T07:31:14.106963+0000","last_undegraded":"2024-09-17T07:31:14.106963+0000","last_fullsized":"2024-09-17T07:31:14.106963+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:37.515132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015649349999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2687,"num_read_kb":46297,"num_write":632,"num_write_kb":27531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":35691417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"3.3","version":"165'668","reported_seq":2866,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.189782+0000","last_change":"2024-09-17T07:31:14.109473+0000","last_active":"2024-09-17T07:31:14.189782+0000","last_peered":"2024-09-17T07:31:14.189782+0000","last_clean":"2024-09-17T07:31:14.189782+0000","last_became_active":"2024-09-17T07:30:31.091522+0000","last_became_peered":"2024-09-17T07:30:31.091522+0000","last_unstale":"2024-09-17T07:31:14.189782+0000","last_undegraded":"2024-09-17T07:31:14.189782+0000","last_fullsized":"2024-09-17T07:31:14.189782+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:21:02.885459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028913630000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":2238,"num_read_kb":35377,"num_write":632,"num_write_kb":16684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":26727758,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"2.2","version":"22'2","reported_seq":309,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.105616+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:31:14.105616+0000","last_peered":"2024-09-17T07:31:14.105616+0000","last_clean":"2024-09-17T07:31:14.105616+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:31:14.105616+0000","last_undegraded":"2024-09-17T07:31:14.105616+0000","last_fullsized":"2024-09-17T07:31:14.105616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"164'825","reported_seq":11299,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.189352+0000","last_change":"2024-09-17T07:31:14.189768+0000","last_active":"2024-09-17T07:31:14.189352+0000","last_peered":"2024-09-17T07:31:14.189352+0000","last_clean":"2024-09-17T07:31:14.189352+0000","last_became_active":"2024-09-17T07:28:43.239409+0000","last_became_peered":"2024-09-17T07:28:43.239409+0000","last_unstale":"2024-09-17T07:31:14.189352+0000","last_undegraded":"2024-09-17T07:31:14.189352+0000","last_fullsized":"2024-09-17T07:31:14.189352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":825,"log_dups_size":0,"ondisk_log_size":825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:13.253211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012368310000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":12617,"num_read_kb":91706,"num_write":1240,"num_write_kb":58366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"2.1","version":"0'0","reported_seq":318,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-17T07:31:13.102365+0000","last_change":"2024-09-17T07:30:15.201999+0000","last_active":"2024-09-17T07:31:13.102365+0000","last_peered":"2024-09-17T07:31:13.102365+0000","last_clean":"2024-09-17T07:31:13.102365+0000","last_became_active":"2024-09-17T07:30:15.201642+0000","last_became_peered":"2024-09-17T07:30:15.201642+0000","last_unstale":"2024-09-17T07:31:13.102365+0000","last_undegraded":"2024-09-17T07:31:13.102365+0000","last_fullsized":"2024-09-17T07:31:13.102365+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:47.871565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"164'427","reported_seq":4858,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-17T07:31:13.137433+0000","last_change":"2024-09-17T07:31:13.137433+0000","last_active":"2024-09-17T07:31:13.137433+0000","last_peered":"2024-09-17T07:31:13.137433+0000","last_clean":"2024-09-17T07:31:13.137433+0000","last_became_active":"2024-09-17T07:31:10.106669+0000","last_became_peered":"2024-09-17T07:31:10.106669+0000","last_unstale":"2024-09-17T07:31:13.137433+0000","last_undegraded":"2024-09-17T07:31:13.137433+0000","last_fullsized":"2024-09-17T07:31:13.137433+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:19:57.884673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001078798,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3788,"num_read_kb":51682,"num_write":424,"num_write_kb":28012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":37615005,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"5"}]},{"pgid":"2.0","version":"0'0","reported_seq":318,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.174087+0000","last_change":"2024-09-17T07:31:01.231085+0000","last_active":"2024-09-17T07:31:14.174087+0000","last_peered":"2024-09-17T07:31:14.174087+0000","last_clean":"2024-09-17T07:31:14.174087+0000","last_became_active":"2024-09-17T07:31:01.230794+0000","last_became_peered":"2024-09-17T07:31:01.230794+0000","last_unstale":"2024-09-17T07:31:14.174087+0000","last_undegraded":"2024-09-17T07:31:14.174087+0000","last_fullsized":"2024-09-17T07:31:14.174087+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:25.321726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"161'398","reported_seq":2429,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-17T07:31:12.091880+0000","last_change":"2024-09-17T07:31:12.093286+0000","last_active":"2024-09-17T07:31:12.091880+0000","last_peered":"2024-09-17T07:31:12.091880+0000","last_clean":"2024-09-17T07:31:12.091880+0000","last_became_active":"2024-09-17T07:30:18.084247+0000","last_became_peered":"2024-09-17T07:30:18.084247+0000","last_unstale":"2024-09-17T07:31:12.091880+0000","last_undegraded":"2024-09-17T07:31:12.091880+0000","last_fullsized":"2024-09-17T07:31:12.091880+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:23.617236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001821745,"stat_sum":{"num_bytes":6197333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":15,"num_read":2955,"num_read_kb":9260,"num_write":357,"num_write_kb":6317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":6805602,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"2.3","version":"20'1","reported_seq":93,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-17T07:31:13.136424+0000","last_change":"2024-09-17T07:30:34.116563+0000","last_active":"2024-09-17T07:31:13.136424+0000","last_peered":"2024-09-17T07:31:13.136424+0000","last_clean":"2024-09-17T07:31:13.136424+0000","last_became_active":"2024-09-17T07:30:34.113598+0000","last_became_peered":"2024-09-17T07:30:34.113598+0000","last_unstale":"2024-09-17T07:31:13.136424+0000","last_undegraded":"2024-09-17T07:31:13.136424+0000","last_fullsized":"2024-09-17T07:31:13.136424+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:52.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":35,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.173995+0000","last_change":"2024-09-17T07:31:05.102195+0000","last_active":"2024-09-17T07:31:14.173995+0000","last_peered":"2024-09-17T07:31:14.173995+0000","last_clean":"2024-09-17T07:31:14.173995+0000","last_became_active":"2024-09-17T07:31:05.096988+0000","last_became_peered":"2024-09-17T07:31:05.096988+0000","last_unstale":"2024-09-17T07:31:14.173995+0000","last_undegraded":"2024-09-17T07:31:14.173995+0000","last_fullsized":"2024-09-17T07:31:14.173995+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:01.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"163'518","reported_seq":2374,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-17T07:31:12.097528+0000","last_change":"2024-09-17T07:31:12.098426+0000","last_active":"2024-09-17T07:31:12.097528+0000","last_peered":"2024-09-17T07:31:12.097528+0000","last_clean":"2024-09-17T07:31:12.097528+0000","last_became_active":"2024-09-17T07:30:19.086777+0000","last_became_peered":"2024-09-17T07:30:19.086777+0000","last_unstale":"2024-09-17T07:31:12.097528+0000","last_undegraded":"2024-09-17T07:31:12.097528+0000","last_fullsized":"2024-09-17T07:31:12.097528+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:48.191360+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0067265060000000002,"stat_sum":{"num_bytes":157468,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":16,"num_read":1640,"num_read_kb":24382,"num_write":525,"num_write_kb":33950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":24920122,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.9","version":"164'551","reported_seq":3633,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.108347+0000","last_change":"2024-09-17T07:31:14.109472+0000","last_active":"2024-09-17T07:31:14.108347+0000","last_peered":"2024-09-17T07:31:14.108347+0000","last_clean":"2024-09-17T07:31:14.108347+0000","last_became_active":"2024-09-17T07:30:30.095518+0000","last_became_peered":"2024-09-17T07:30:30.095518+0000","last_unstale":"2024-09-17T07:31:14.108347+0000","last_undegraded":"2024-09-17T07:31:14.108347+0000","last_fullsized":"2024-09-17T07:31:14.108347+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:13:07.426219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029644179999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4515,"num_read_kb":21074,"num_write":530,"num_write_kb":12196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":11434290,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"3.a","version":"164'739","reported_seq":8935,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-17T07:31:13.206469+0000","last_change":"2024-09-17T07:31:13.202602+0000","last_active":"2024-09-17T07:31:13.206469+0000","last_peered":"2024-09-17T07:31:13.206469+0000","last_clean":"2024-09-17T07:31:13.206469+0000","last_became_active":"2024-09-17T07:30:31.090321+0000","last_became_peered":"2024-09-17T07:30:31.090321+0000","last_unstale":"2024-09-17T07:31:13.206469+0000","last_undegraded":"2024-09-17T07:31:13.206469+0000","last_fullsized":"2024-09-17T07:31:13.206469+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:05:07.589494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015167480000000001,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":26,"num_read":9368,"num_read_kb":52923,"num_write":688,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":24254406,"num_keys_recovered":93,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"5"}]},{"pgid":"3.b","version":"163'559","reported_seq":2852,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-17T07:31:12.103600+0000","last_change":"2024-09-17T07:31:12.104550+0000","last_active":"2024-09-17T07:31:12.103600+0000","last_peered":"2024-09-17T07:31:12.103600+0000","last_clean":"2024-09-17T07:31:12.103600+0000","last_became_active":"2024-09-17T07:30:19.087515+0000","last_became_peered":"2024-09-17T07:30:19.087515+0000","last_unstale":"2024-09-17T07:31:12.103600+0000","last_undegraded":"2024-09-17T07:31:12.103600+0000","last_fullsized":"2024-09-17T07:31:12.103600+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:02.495676+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.012816311,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2408,"num_read_kb":29904,"num_write":665,"num_write_kb":15633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":11178824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"4"}]},{"pgid":"3.c","version":"165'873","reported_seq":5386,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.179506+0000","last_change":"2024-09-17T07:31:14.107447+0000","last_active":"2024-09-17T07:31:14.179506+0000","last_peered":"2024-09-17T07:31:14.179506+0000","last_clean":"2024-09-17T07:31:14.179506+0000","last_became_active":"2024-09-17T07:30:19.085668+0000","last_became_peered":"2024-09-17T07:30:19.085668+0000","last_unstale":"2024-09-17T07:31:14.179506+0000","last_undegraded":"2024-09-17T07:31:14.179506+0000","last_fullsized":"2024-09-17T07:31:14.179506+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:47.391518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001368193,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6070,"num_read_kb":27721,"num_write":1113,"num_write_kb":14827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":15619889,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"3.d","version":"165'725","reported_seq":4568,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.176026+0000","last_change":"2024-09-17T07:31:14.176026+0000","last_active":"2024-09-17T07:31:14.176026+0000","last_peered":"2024-09-17T07:31:14.176026+0000","last_clean":"2024-09-17T07:31:14.176026+0000","last_became_active":"2024-09-17T07:28:43.080816+0000","last_became_peered":"2024-09-17T07:28:43.080816+0000","last_unstale":"2024-09-17T07:31:14.176026+0000","last_undegraded":"2024-09-17T07:31:14.176026+0000","last_fullsized":"2024-09-17T07:31:14.176026+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:05:06.879033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017783269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6005,"num_read_kb":48957,"num_write":832,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"3.e","version":"164'458","reported_seq":7869,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.175931+0000","last_change":"2024-09-17T07:31:14.176641+0000","last_active":"2024-09-17T07:31:14.175931+0000","last_peered":"2024-09-17T07:31:14.175931+0000","last_clean":"2024-09-17T07:31:14.175931+0000","last_became_active":"2024-09-17T07:31:10.103615+0000","last_became_peered":"2024-09-17T07:31:10.103615+0000","last_unstale":"2024-09-17T07:31:14.175931+0000","last_undegraded":"2024-09-17T07:31:14.175931+0000","last_fullsized":"2024-09-17T07:31:14.175931+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:35:23.104359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021346680000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7087,"num_read_kb":55690,"num_write":392,"num_write_kb":41376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":13797349,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"3.f","version":"163'832","reported_seq":5778,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.176091+0000","last_change":"2024-09-17T07:31:14.176707+0000","last_active":"2024-09-17T07:31:14.176091+0000","last_peered":"2024-09-17T07:31:14.176091+0000","last_clean":"2024-09-17T07:31:14.176091+0000","last_became_active":"2024-09-17T07:31:08.095644+0000","last_became_peered":"2024-09-17T07:31:08.095644+0000","last_unstale":"2024-09-17T07:31:14.176091+0000","last_undegraded":"2024-09-17T07:31:14.176091+0000","last_fullsized":"2024-09-17T07:31:14.176091+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:03:50.277964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021103049999999998,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6418,"num_read_kb":23171,"num_write":923,"num_write_kb":18541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":496,"num_bytes_recovered":25525458,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"3.10","version":"164'401","reported_seq":2715,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-17T07:31:13.206291+0000","last_change":"2024-09-17T07:31:13.203313+0000","last_active":"2024-09-17T07:31:13.206291+0000","last_peered":"2024-09-17T07:31:13.206291+0000","last_clean":"2024-09-17T07:31:13.206291+0000","last_became_active":"2024-09-17T07:28:43.077360+0000","last_became_peered":"2024-09-17T07:28:43.077360+0000","last_unstale":"2024-09-17T07:31:13.206291+0000","last_undegraded":"2024-09-17T07:31:13.206291+0000","last_fullsized":"2024-09-17T07:31:13.206291+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:44.670066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022387710000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2841,"num_read_kb":26198,"num_write":450,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"4","length":"5"}]},{"pgid":"3.11","version":"164'802","reported_seq":3805,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.175224+0000","last_change":"2024-09-17T07:31:14.175834+0000","last_active":"2024-09-17T07:31:14.175224+0000","last_peered":"2024-09-17T07:31:14.175224+0000","last_clean":"2024-09-17T07:31:14.175224+0000","last_became_active":"2024-09-17T07:30:31.091943+0000","last_became_peered":"2024-09-17T07:30:31.091943+0000","last_unstale":"2024-09-17T07:31:14.175224+0000","last_undegraded":"2024-09-17T07:31:14.175224+0000","last_fullsized":"2024-09-17T07:31:14.175224+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:41:20.129623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001604722,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5310,"num_read_kb":45214,"num_write":937,"num_write_kb":26421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":31676674,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"6"}]},{"pgid":"3.12","version":"164'435","reported_seq":2695,"reported_epoch":164,"state":"active+clean","last_fresh":"2024-09-17T07:31:13.206509+0000","last_change":"2024-09-17T07:31:13.138694+0000","last_active":"2024-09-17T07:31:13.206509+0000","last_peered":"2024-09-17T07:31:13.206509+0000","last_clean":"2024-09-17T07:31:13.206509+0000","last_became_active":"2024-09-17T07:31:08.096240+0000","last_became_peered":"2024-09-17T07:31:08.096240+0000","last_unstale":"2024-09-17T07:31:13.206509+0000","last_undegraded":"2024-09-17T07:31:13.206509+0000","last_fullsized":"2024-09-17T07:31:13.206509+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:42.355601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022418569999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2231,"num_read_kb":44562,"num_write":412,"num_write_kb":29148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":24460350,"num_keys_recovered":37,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"5"}]},{"pgid":"3.13","version":"164'772","reported_seq":3772,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-17T07:31:14.174552+0000","last_change":"2024-09-17T07:31:14.175163+0000","last_active":"2024-09-17T07:31:14.174552+0000","last_peered":"2024-09-17T07:31:14.174552+0000","last_clean":"2024-09-17T07:31:14.174552+0000","last_became_active":"2024-09-17T07:30:32.085258+0000","last_became_peered":"2024-09-17T07:30:32.085258+0000","last_unstale":"2024-09-17T07:31:14.174552+0000","last_undegraded":"2024-09-17T07:31:14.174552+0000","last_fullsized":"2024-09-17T07:31:14.174552+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:58.241865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093421499999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4511,"num_read_kb":28124,"num_write":1074,"num_write_kb":16615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":21539718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"6"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":9213209,"num_objects":626,"num_object_clones":0,"num_object_copies":1252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":626,"num_whiteouts":607,"num_read":165974,"num_read_kb":1208137,"num_write":23154,"num_write_kb":792759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6836,"num_bytes_recovered":666374200,"num_keys_recovered":891,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":374565,"data_compressed":10194,"data_compressed_allocated":36864,"data_compressed_original":212992,"omap_allocated":7902,"internal_metadata":0},"log_size":20521,"ondisk_log_size":20521,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":101032,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542180,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59632,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312208,"statfs":{"total":96636764160,"available":96575700992,"internally_reserved":0,"allocated":2039808,"data_stored":1558407,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542181,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28960,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342880,"statfs":{"total":96636764160,"available":96607109120,"internally_reserved":0,"allocated":1970176,"data_stored":949661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55392,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316448,"statfs":{"total":96636764160,"available":96580042752,"internally_reserved":0,"allocated":2039808,"data_stored":1001563,"data_compressed":592,"data_compressed_allocated":4096,"data_compressed_original":12288,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542180,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59532,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312308,"statfs":{"total":96636764160,"available":96575803392,"internally_reserved":0,"allocated":1974272,"data_stored":949680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056656,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43212,"kb_used_data":2160,"kb_used_omap":16,"kb_used_meta":41007,"kb_avail":94328628,"statfs":{"total":96636764160,"available":96592515072,"internally_reserved":0,"allocated":2211840,"data_stored":1362546,"data_compressed":12225,"data_compressed_allocated":45056,"data_compressed_original":253952,"omap_allocated":17058,"internal_metadata":41991518},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542180,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51208,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320632,"statfs":{"total":96636764160,"available":96584327168,"internally_reserved":0,"allocated":2011136,"data_stored":1000151,"data_compressed":1205,"data_compressed_allocated":8192,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542180,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77540,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294300,"statfs":{"total":96636764160,"available":96557363200,"internally_reserved":0,"allocated":1978368,"data_stored":949699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542180,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59680,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312160,"statfs":{"total":96636764160,"available":96575651840,"internally_reserved":0,"allocated":2166784,"data_stored":1779957,"data_compressed":39957,"data_compressed_allocated":69632,"data_compressed_original":724992,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":374511,"data_compressed":10194,"data_compressed_allocated":36864,"data_compressed_original":212992,"omap_allocated":7902,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:31:15.100 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T07:31:15.100 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T07:31:15.100 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 1, 2, 5, 6, 7, 3, 4] 2024-09-17T07:31:15.100 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-09-17T07:31:15.106 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T07:31:15.117 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T07:31:15.209 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:15.209 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:15.217 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T07:31:15.321 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:15.321 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:15.330 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T07:31:15.416 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-17T07:31:15.415+0000 7f6d53957640 -1 mon.a@0(leader).osd e165 definitely_dead 0 2024-09-17T07:31:15.432 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:15.433 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:15.441 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T07:31:15.545 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:15.545 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:15.553 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T07:31:15.657 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:15.658 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:15.667 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T07:31:15.769 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:15.769 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:15.778 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T07:31:15.889 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:15.889 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:15.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T07:31:16.005 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:16.005 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:16.015 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T07:31:16.115 INFO:teuthology.orchestra.run.smithi045.stderr:noscrub is unset 2024-09-17T07:31:16.118 INFO:teuthology.orchestra.run.smithi045.stderr:marked out osd.4. 2024-09-17T07:31:16.127 INFO:teuthology.orchestra.run.smithi045.stdout:ERROR: (22) Invalid argument 2024-09-17T07:31:16.127 INFO:teuthology.orchestra.run.smithi045.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T07:31:16.137 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T07:31:17.091 INFO:teuthology.orchestra.run.smithi045.stderr:nodeep-scrub is unset 2024-09-17T07:31:21.140 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-17T07:31:21.140 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-17T07:31:21.141 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-09-17T07:31:21.463 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-09-17T07:31:21.461+0000 7f6d53957640 -1 mon.a@0(leader).osd e171 definitely_dead 0 2024-09-17T07:31:22.092 INFO:teuthology.orchestra.run.smithi045.stderr:marked in osd.4. 2024-09-17T07:31:22.111 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-17T07:31:22.111 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-17T07:31:22.232 INFO:teuthology.orchestra.run.smithi045.stdout:mon.a: {} 2024-09-17T07:31:22.232 INFO:teuthology.orchestra.run.smithi045.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T07:31:22.242 INFO:teuthology.orchestra.run.smithi045.stdout:mon.b: {} 2024-09-17T07:31:22.243 INFO:teuthology.orchestra.run.smithi045.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T07:31:22.251 INFO:teuthology.orchestra.run.smithi045.stdout:mon.c: {} 2024-09-17T07:31:22.251 INFO:teuthology.orchestra.run.smithi045.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T07:31:22.262 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T07:31:23.097 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T07:31:23.113 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-17T07:31:24.095 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-17T07:31:24.118 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T07:31:25.143 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T07:31:25.158 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-17T07:31:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-17T07:31:26.159 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T07:31:27.092 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T07:31:27.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-17T07:31:28.094 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-17T07:31:28.113 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T07:31:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T07:31:29.115 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-17T07:31:30.161 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-17T07:31:30.178 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T07:31:31.165 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T07:31:31.184 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-17T07:31:32.176 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-17T07:31:32.193 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-17T07:31:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T07:31:33.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-17T07:31:34.188 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-17T07:31:34.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-17T07:31:35.194 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T07:31:35.212 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-17T07:31:36.200 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-17T07:31:36.217 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T07:31:37.205 INFO:teuthology.orchestra.run.smithi045.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T07:31:37.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-17T07:31:38.211 INFO:teuthology.orchestra.run.smithi045.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-17T07:31:38.229 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-17T07:31:38.229 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-17T07:31:38.229 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-17T07:31:38.229 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-17T07:31:38.229 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T07:31:38.230 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T07:31:38.536 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:38.536 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":188,"fsid":"b283e6d5-0f4a-4271-a443-15e695e35364","created":"2024-09-17T07:28:12.276015+0000","modified":"2024-09-17T07:31:38.206026+0000","last_up_change":"2024-09-17T07:29:58.120417+0000","last_in_change":"2024-09-17T07:31:22.090468+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T07:28:24.767696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T07:28:30.249487+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T07:28:41.831004+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":165,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7794e6a8-ebf4-4b7c-bf3d-63cd015234ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6801","nonce":2039818160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6803","nonce":2039818160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6807","nonce":2039818160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6805","nonce":2039818160}]},"public_addr":"172.21.15.45:6801/2039818160","cluster_addr":"172.21.15.45:6803/2039818160","heartbeat_back_addr":"172.21.15.45:6807/2039818160","heartbeat_front_addr":"172.21.15.45:6805/2039818160","state":["exists","up"]},{"osd":1,"uuid":"f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6809","nonce":1158817756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6811","nonce":1158817756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6815","nonce":1158817756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6813","nonce":1158817756}]},"public_addr":"172.21.15.45:6809/1158817756","cluster_addr":"172.21.15.45:6811/1158817756","heartbeat_back_addr":"172.21.15.45:6815/1158817756","heartbeat_front_addr":"172.21.15.45:6813/1158817756","state":["exists","up"]},{"osd":2,"uuid":"195a9db1-568d-459a-bb50-2a1bb2445f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6825","nonce":3298886784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6827","nonce":3298886784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6831","nonce":3298886784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6829","nonce":3298886784}]},"public_addr":"172.21.15.45:6825/3298886784","cluster_addr":"172.21.15.45:6827/3298886784","heartbeat_back_addr":"172.21.15.45:6831/3298886784","heartbeat_front_addr":"172.21.15.45:6829/3298886784","state":["exists","up"]},{"osd":3,"uuid":"b3fda98b-a10a-4354-905e-1b37a5124b98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":36,"up_from":90,"up_thru":157,"down_at":37,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":724784918},{"type":"v1","addr":"172.21.15.45:6817","nonce":724784918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":724784918},{"type":"v1","addr":"172.21.15.45:6819","nonce":724784918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":724784918},{"type":"v1","addr":"172.21.15.45:6823","nonce":724784918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":724784918},{"type":"v1","addr":"172.21.15.45:6821","nonce":724784918}]},"public_addr":"172.21.15.45:6817/724784918","cluster_addr":"172.21.15.45:6819/724784918","heartbeat_back_addr":"172.21.15.45:6823/724784918","heartbeat_front_addr":"172.21.15.45:6821/724784918","state":["exists","up"]},{"osd":4,"uuid":"6ab67b48-2d83-4e68-8898-302a1adc0a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6825","nonce":1644784047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6827","nonce":1644784047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6831","nonce":1644784047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6829","nonce":1644784047}]},"public_addr":"172.21.15.192:6825/1644784047","cluster_addr":"172.21.15.192:6827/1644784047","heartbeat_back_addr":"172.21.15.192:6831/1644784047","heartbeat_front_addr":"172.21.15.192:6829/1644784047","state":["exists","up"]},{"osd":5,"uuid":"ae3c787c-63d2-4e88-8f99-6d4f19837a3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6817","nonce":4077426583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6819","nonce":4077426583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6823","nonce":4077426583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6821","nonce":4077426583}]},"public_addr":"172.21.15.192:6817/4077426583","cluster_addr":"172.21.15.192:6819/4077426583","heartbeat_back_addr":"172.21.15.192:6823/4077426583","heartbeat_front_addr":"172.21.15.192:6821/4077426583","state":["exists","up"]},{"osd":6,"uuid":"f792f18d-ad9e-49a8-b194-24522398fb25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6809","nonce":2922229639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6811","nonce":2922229639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6815","nonce":2922229639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6813","nonce":2922229639}]},"public_addr":"172.21.15.192:6809/2922229639","cluster_addr":"172.21.15.192:6811/2922229639","heartbeat_back_addr":"172.21.15.192:6815/2922229639","heartbeat_front_addr":"172.21.15.192:6813/2922229639","state":["exists","up"]},{"osd":7,"uuid":"4a589d43-b0b9-42de-989c-46928d693c24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6801","nonce":780298944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6803","nonce":780298944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6807","nonce":780298944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6805","nonce":780298944}]},"public_addr":"172.21.15.192:6801/780298944","cluster_addr":"172.21.15.192:6803/780298944","heartbeat_back_addr":"172.21.15.192:6807/780298944","heartbeat_front_addr":"172.21.15.192:6805/780298944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.586514+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.833505+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:21.059899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.499470+0000","dead_epoch":37},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:21.353342+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.586673+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.767847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.859615+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.4","osds":[5,3]}],"pg_upmap_items":[{"pgid":"2.3","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T07:31:38.549 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T07:31:38.549 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T07:31:38.849 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:38.849 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":188,"fsid":"b283e6d5-0f4a-4271-a443-15e695e35364","created":"2024-09-17T07:28:12.276015+0000","modified":"2024-09-17T07:31:38.206026+0000","last_up_change":"2024-09-17T07:29:58.120417+0000","last_in_change":"2024-09-17T07:31:22.090468+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T07:28:24.767696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T07:28:30.249487+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T07:28:41.831004+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":165,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7794e6a8-ebf4-4b7c-bf3d-63cd015234ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6801","nonce":2039818160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6803","nonce":2039818160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6807","nonce":2039818160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6805","nonce":2039818160}]},"public_addr":"172.21.15.45:6801/2039818160","cluster_addr":"172.21.15.45:6803/2039818160","heartbeat_back_addr":"172.21.15.45:6807/2039818160","heartbeat_front_addr":"172.21.15.45:6805/2039818160","state":["exists","up"]},{"osd":1,"uuid":"f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6809","nonce":1158817756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6811","nonce":1158817756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6815","nonce":1158817756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6813","nonce":1158817756}]},"public_addr":"172.21.15.45:6809/1158817756","cluster_addr":"172.21.15.45:6811/1158817756","heartbeat_back_addr":"172.21.15.45:6815/1158817756","heartbeat_front_addr":"172.21.15.45:6813/1158817756","state":["exists","up"]},{"osd":2,"uuid":"195a9db1-568d-459a-bb50-2a1bb2445f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6825","nonce":3298886784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6827","nonce":3298886784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6831","nonce":3298886784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6829","nonce":3298886784}]},"public_addr":"172.21.15.45:6825/3298886784","cluster_addr":"172.21.15.45:6827/3298886784","heartbeat_back_addr":"172.21.15.45:6831/3298886784","heartbeat_front_addr":"172.21.15.45:6829/3298886784","state":["exists","up"]},{"osd":3,"uuid":"b3fda98b-a10a-4354-905e-1b37a5124b98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":36,"up_from":90,"up_thru":157,"down_at":37,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":724784918},{"type":"v1","addr":"172.21.15.45:6817","nonce":724784918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":724784918},{"type":"v1","addr":"172.21.15.45:6819","nonce":724784918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":724784918},{"type":"v1","addr":"172.21.15.45:6823","nonce":724784918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":724784918},{"type":"v1","addr":"172.21.15.45:6821","nonce":724784918}]},"public_addr":"172.21.15.45:6817/724784918","cluster_addr":"172.21.15.45:6819/724784918","heartbeat_back_addr":"172.21.15.45:6823/724784918","heartbeat_front_addr":"172.21.15.45:6821/724784918","state":["exists","up"]},{"osd":4,"uuid":"6ab67b48-2d83-4e68-8898-302a1adc0a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6825","nonce":1644784047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6827","nonce":1644784047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6831","nonce":1644784047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6829","nonce":1644784047}]},"public_addr":"172.21.15.192:6825/1644784047","cluster_addr":"172.21.15.192:6827/1644784047","heartbeat_back_addr":"172.21.15.192:6831/1644784047","heartbeat_front_addr":"172.21.15.192:6829/1644784047","state":["exists","up"]},{"osd":5,"uuid":"ae3c787c-63d2-4e88-8f99-6d4f19837a3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6817","nonce":4077426583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6819","nonce":4077426583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6823","nonce":4077426583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6821","nonce":4077426583}]},"public_addr":"172.21.15.192:6817/4077426583","cluster_addr":"172.21.15.192:6819/4077426583","heartbeat_back_addr":"172.21.15.192:6823/4077426583","heartbeat_front_addr":"172.21.15.192:6821/4077426583","state":["exists","up"]},{"osd":6,"uuid":"f792f18d-ad9e-49a8-b194-24522398fb25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6809","nonce":2922229639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6811","nonce":2922229639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6815","nonce":2922229639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6813","nonce":2922229639}]},"public_addr":"172.21.15.192:6809/2922229639","cluster_addr":"172.21.15.192:6811/2922229639","heartbeat_back_addr":"172.21.15.192:6815/2922229639","heartbeat_front_addr":"172.21.15.192:6813/2922229639","state":["exists","up"]},{"osd":7,"uuid":"4a589d43-b0b9-42de-989c-46928d693c24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6801","nonce":780298944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6803","nonce":780298944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6807","nonce":780298944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6805","nonce":780298944}]},"public_addr":"172.21.15.192:6801/780298944","cluster_addr":"172.21.15.192:6803/780298944","heartbeat_back_addr":"172.21.15.192:6807/780298944","heartbeat_front_addr":"172.21.15.192:6805/780298944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.586514+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.833505+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:21.059899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.499470+0000","dead_epoch":37},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:21.353342+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.586673+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.767847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.859615+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.4","osds":[5,3]}],"pg_upmap_items":[{"pgid":"2.3","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T07:31:38.862 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T07:31:38.862 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T07:31:38.863 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T07:31:38.863 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T07:31:38.864 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T07:31:38.864 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T07:31:38.864 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T07:31:38.865 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T07:31:39.116 INFO:teuthology.orchestra.run.smithi045.stdout:60129542185 2024-09-17T07:31:39.116 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T07:31:39.118 INFO:teuthology.orchestra.run.smithi045.stdout:60129542185 2024-09-17T07:31:39.119 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T07:31:39.119 INFO:teuthology.orchestra.run.smithi045.stdout:60129542185 2024-09-17T07:31:39.120 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T07:31:39.120 INFO:teuthology.orchestra.run.smithi045.stdout:60129542186 2024-09-17T07:31:39.121 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T07:31:39.122 INFO:teuthology.orchestra.run.smithi045.stdout:386547056662 2024-09-17T07:31:39.122 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T07:31:39.124 INFO:teuthology.orchestra.run.smithi045.stdout:60129542185 2024-09-17T07:31:39.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T07:31:39.125 INFO:teuthology.orchestra.run.smithi045.stdout:60129542186 2024-09-17T07:31:39.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T07:31:39.127 INFO:teuthology.orchestra.run.smithi045.stdout:60129542186 2024-09-17T07:31:39.127 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T07:31:39.745 INFO:teuthology.orchestra.run.smithi045.stdout:60129542184 2024-09-17T07:31:39.756 INFO:teuthology.orchestra.run.smithi045.stdout:386547056661 2024-09-17T07:31:39.760 INFO:teuthology.orchestra.run.smithi045.stdout:60129542185 2024-09-17T07:31:39.761 INFO:teuthology.orchestra.run.smithi045.stdout:60129542184 2024-09-17T07:31:39.764 INFO:teuthology.orchestra.run.smithi045.stdout:60129542184 2024-09-17T07:31:39.766 INFO:teuthology.orchestra.run.smithi045.stdout:60129542184 2024-09-17T07:31:39.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542184 for osd.1 2024-09-17T07:31:39.772 INFO:teuthology.orchestra.run.smithi045.stdout:60129542185 2024-09-17T07:31:39.782 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542185 for osd.2 2024-09-17T07:31:39.783 INFO:tasks.ceph.ceph_manager.ceph:need seq 386547056662 got 386547056661 for osd.3 2024-09-17T07:31:39.786 INFO:teuthology.orchestra.run.smithi045.stdout:60129542185 2024-09-17T07:31:39.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542184 for osd.4 2024-09-17T07:31:39.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542184 for osd.5 2024-09-17T07:31:39.791 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542184 for osd.7 2024-09-17T07:31:39.794 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542185 for osd.6 2024-09-17T07:31:39.800 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542185 for osd.0 2024-09-17T07:31:40.768 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T07:31:40.783 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T07:31:40.784 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T07:31:40.788 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T07:31:40.790 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T07:31:40.792 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T07:31:40.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T07:31:40.801 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T07:31:41.357 INFO:teuthology.orchestra.run.smithi045.stdout:60129542186 2024-09-17T07:31:41.376 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542186 for osd.1 2024-09-17T07:31:41.376 DEBUG:teuthology.parallel:result is None 2024-09-17T07:31:41.412 INFO:teuthology.orchestra.run.smithi045.stdout:60129542186 2024-09-17T07:31:41.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542186 for osd.5 2024-09-17T07:31:41.426 DEBUG:teuthology.parallel:result is None 2024-09-17T07:31:41.449 INFO:teuthology.orchestra.run.smithi045.stdout:60129542187 2024-09-17T07:31:41.453 INFO:teuthology.orchestra.run.smithi045.stdout:60129542186 2024-09-17T07:31:41.456 INFO:teuthology.orchestra.run.smithi045.stdout:386547056662 2024-09-17T07:31:41.460 INFO:teuthology.orchestra.run.smithi045.stdout:60129542186 2024-09-17T07:31:41.460 INFO:teuthology.orchestra.run.smithi045.stdout:60129542186 2024-09-17T07:31:41.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542187 for osd.6 2024-09-17T07:31:41.469 DEBUG:teuthology.parallel:result is None 2024-09-17T07:31:41.472 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542186 for osd.2 2024-09-17T07:31:41.472 DEBUG:teuthology.parallel:result is None 2024-09-17T07:31:41.475 INFO:teuthology.orchestra.run.smithi045.stdout:60129542186 2024-09-17T07:31:41.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 386547056662 got 386547056662 for osd.3 2024-09-17T07:31:41.478 DEBUG:teuthology.parallel:result is None 2024-09-17T07:31:41.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542186 for osd.4 2024-09-17T07:31:41.481 DEBUG:teuthology.parallel:result is None 2024-09-17T07:31:41.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542185 got 60129542186 for osd.7 2024-09-17T07:31:41.482 DEBUG:teuthology.parallel:result is None 2024-09-17T07:31:41.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542186 got 60129542186 for osd.0 2024-09-17T07:31:41.489 DEBUG:teuthology.parallel:result is None 2024-09-17T07:31:41.490 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T07:31:41.490 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:31:41.829 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:41.830 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:31:41.841 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-17T07:31:40.788312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":607,"num_read":166114,"num_read_kb":1208257,"num_write":23282,"num_write_kb":794390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9832,"num_bytes_recovered":667554974,"num_keys_recovered":891,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20628,"ondisk_log_size":20628,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439548,"kb_used_data":17776,"kb_used_omap":27,"kb_used_meta":421540,"kb_avail":754535172,"statfs":{"total":773094113280,"available":772644016128,"internally_reserved":0,"allocated":18202624,"data_stored":9917992,"data_compressed":66815,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":28217,"internal_metadata":431657415},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.593244"},"pg_stats":[{"pgid":"3.1f","version":"165'874","reported_seq":4617,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208112+0000","last_change":"2024-09-17T07:31:14.105507+0000","last_active":"2024-09-17T07:31:37.208112+0000","last_peered":"2024-09-17T07:31:37.208112+0000","last_clean":"2024-09-17T07:31:37.208112+0000","last_became_active":"2024-09-17T07:30:43.092456+0000","last_became_peered":"2024-09-17T07:30:43.092456+0000","last_unstale":"2024-09-17T07:31:37.208112+0000","last_undegraded":"2024-09-17T07:31:37.208112+0000","last_fullsized":"2024-09-17T07:31:37.208112+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:26.811072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016105900000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5801,"num_read_kb":34310,"num_write":997,"num_write_kb":41977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":26159662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"164'553","reported_seq":3285,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214114+0000","last_change":"2024-09-17T07:31:14.115590+0000","last_active":"2024-09-17T07:31:38.214114+0000","last_peered":"2024-09-17T07:31:38.214114+0000","last_clean":"2024-09-17T07:31:38.214114+0000","last_became_active":"2024-09-17T07:30:19.087213+0000","last_became_peered":"2024-09-17T07:30:19.087213+0000","last_unstale":"2024-09-17T07:31:38.214114+0000","last_undegraded":"2024-09-17T07:31:38.214114+0000","last_fullsized":"2024-09-17T07:31:38.214114+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:15.610729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029487620000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3721,"num_read_kb":30298,"num_write":527,"num_write_kb":15594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":15743718,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"163'550","reported_seq":3498,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214875+0000","last_change":"2024-09-17T07:31:26.150683+0000","last_active":"2024-09-17T07:31:38.214875+0000","last_peered":"2024-09-17T07:31:38.214875+0000","last_clean":"2024-09-17T07:31:38.214875+0000","last_became_active":"2024-09-17T07:31:26.149428+0000","last_became_peered":"2024-09-17T07:31:26.149428+0000","last_unstale":"2024-09-17T07:31:38.214875+0000","last_undegraded":"2024-09-17T07:31:38.214875+0000","last_fullsized":"2024-09-17T07:31:38.214875+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:58.834946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016933180000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":1948,"num_read_kb":28759,"num_write":579,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":27967854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"165'813","reported_seq":6628,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215109+0000","last_change":"2024-09-17T07:31:28.101214+0000","last_active":"2024-09-17T07:31:38.215109+0000","last_peered":"2024-09-17T07:31:38.215109+0000","last_clean":"2024-09-17T07:31:38.215109+0000","last_became_active":"2024-09-17T07:31:28.100048+0000","last_became_peered":"2024-09-17T07:31:28.100048+0000","last_unstale":"2024-09-17T07:31:38.215109+0000","last_undegraded":"2024-09-17T07:31:38.215109+0000","last_fullsized":"2024-09-17T07:31:38.215109+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:35.020631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010655949999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":7523,"num_read_kb":42454,"num_write":1061,"num_write_kb":38721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":26661650,"num_keys_recovered":180,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"165'606","reported_seq":5639,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208349+0000","last_change":"2024-09-17T07:31:28.104670+0000","last_active":"2024-09-17T07:31:37.208349+0000","last_peered":"2024-09-17T07:31:37.208349+0000","last_clean":"2024-09-17T07:31:37.208349+0000","last_became_active":"2024-09-17T07:31:28.102058+0000","last_became_peered":"2024-09-17T07:31:28.102058+0000","last_unstale":"2024-09-17T07:31:37.208349+0000","last_undegraded":"2024-09-17T07:31:37.208349+0000","last_fullsized":"2024-09-17T07:31:37.208349+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:07.797309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028238529999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5800,"num_read_kb":52463,"num_write":596,"num_write_kb":22879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":91784745,"num_keys_recovered":126,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"163'920","reported_seq":14894,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372319+0000","last_change":"2024-09-17T07:31:27.105439+0000","last_active":"2024-09-17T07:31:37.372319+0000","last_peered":"2024-09-17T07:31:37.372319+0000","last_clean":"2024-09-17T07:31:37.372319+0000","last_became_active":"2024-09-17T07:31:27.103933+0000","last_became_peered":"2024-09-17T07:31:27.103933+0000","last_unstale":"2024-09-17T07:31:37.372319+0000","last_undegraded":"2024-09-17T07:31:37.372319+0000","last_fullsized":"2024-09-17T07:31:37.372319+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:06:20.908980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096620079999999997,"stat_sum":{"num_bytes":19,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":16715,"num_read_kb":60981,"num_write":1188,"num_write_kb":20597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":19562028,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"164'439","reported_seq":5685,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145330+0000","last_change":"2024-09-17T07:31:14.189296+0000","last_active":"2024-09-17T07:31:39.145330+0000","last_peered":"2024-09-17T07:31:39.145330+0000","last_clean":"2024-09-17T07:31:39.145330+0000","last_became_active":"2024-09-17T07:30:18.083918+0000","last_became_peered":"2024-09-17T07:30:18.083918+0000","last_unstale":"2024-09-17T07:31:39.145330+0000","last_undegraded":"2024-09-17T07:31:39.145330+0000","last_fullsized":"2024-09-17T07:31:39.145330+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:26.983426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000820972,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4352,"num_read_kb":37354,"num_write":400,"num_write_kb":43089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":25536146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'763","reported_seq":4669,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214008+0000","last_change":"2024-09-17T07:31:14.113881+0000","last_active":"2024-09-17T07:31:38.214008+0000","last_peered":"2024-09-17T07:31:38.214008+0000","last_clean":"2024-09-17T07:31:38.214008+0000","last_became_active":"2024-09-17T07:30:43.091692+0000","last_became_peered":"2024-09-17T07:30:43.091692+0000","last_unstale":"2024-09-17T07:31:38.214008+0000","last_undegraded":"2024-09-17T07:31:38.214008+0000","last_fullsized":"2024-09-17T07:31:38.214008+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:24.550163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014749730000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5969,"num_read_kb":25327,"num_write":792,"num_write_kb":16603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":29456184,"num_keys_recovered":176,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"163'436","reported_seq":4226,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208396+0000","last_change":"2024-09-17T07:31:14.106118+0000","last_active":"2024-09-17T07:31:37.208396+0000","last_peered":"2024-09-17T07:31:37.208396+0000","last_clean":"2024-09-17T07:31:37.208396+0000","last_became_active":"2024-09-17T07:28:43.080428+0000","last_became_peered":"2024-09-17T07:28:43.080428+0000","last_unstale":"2024-09-17T07:31:37.208396+0000","last_undegraded":"2024-09-17T07:31:37.208396+0000","last_fullsized":"2024-09-17T07:31:37.208396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:12:14.112234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020783120000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4961,"num_read_kb":22717,"num_write":365,"num_write_kb":13531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"164'513","reported_seq":4191,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214897+0000","last_change":"2024-09-17T07:31:14.106949+0000","last_active":"2024-09-17T07:31:38.214897+0000","last_peered":"2024-09-17T07:31:38.214897+0000","last_clean":"2024-09-17T07:31:38.214897+0000","last_became_active":"2024-09-17T07:28:43.082313+0000","last_became_peered":"2024-09-17T07:28:43.082313+0000","last_unstale":"2024-09-17T07:31:38.214897+0000","last_undegraded":"2024-09-17T07:31:38.214897+0000","last_fullsized":"2024-09-17T07:31:38.214897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:35.352537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045261,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6140,"num_read_kb":58359,"num_write":449,"num_write_kb":31855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"164'390","reported_seq":4504,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372480+0000","last_change":"2024-09-17T07:31:14.175336+0000","last_active":"2024-09-17T07:31:37.372480+0000","last_peered":"2024-09-17T07:31:37.372480+0000","last_clean":"2024-09-17T07:31:37.372480+0000","last_became_active":"2024-09-17T07:31:08.094940+0000","last_became_peered":"2024-09-17T07:31:08.094940+0000","last_unstale":"2024-09-17T07:31:37.372480+0000","last_undegraded":"2024-09-17T07:31:37.372480+0000","last_fullsized":"2024-09-17T07:31:37.372480+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:58.292675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011121970000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3586,"num_read_kb":32352,"num_write":371,"num_write_kb":22781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":431,"num_bytes_recovered":60567103,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"163'465","reported_seq":2104,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372357+0000","last_change":"2024-09-17T07:31:26.153704+0000","last_active":"2024-09-17T07:31:37.372357+0000","last_peered":"2024-09-17T07:31:37.372357+0000","last_clean":"2024-09-17T07:31:37.372357+0000","last_became_active":"2024-09-17T07:31:26.152423+0000","last_became_peered":"2024-09-17T07:31:26.152423+0000","last_unstale":"2024-09-17T07:31:37.372357+0000","last_undegraded":"2024-09-17T07:31:37.372357+0000","last_fullsized":"2024-09-17T07:31:37.372357+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:46.055274+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015069525,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1414,"num_read_kb":17974,"num_write":482,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":16518806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":363,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216133+0000","last_change":"2024-09-17T07:31:05.098106+0000","last_active":"2024-09-17T07:31:38.216133+0000","last_peered":"2024-09-17T07:31:38.216133+0000","last_clean":"2024-09-17T07:31:38.216133+0000","last_became_active":"2024-09-17T07:31:05.097821+0000","last_became_peered":"2024-09-17T07:31:05.097821+0000","last_unstale":"2024-09-17T07:31:38.216133+0000","last_undegraded":"2024-09-17T07:31:38.216133+0000","last_fullsized":"2024-09-17T07:31:38.216133+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:12.297890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"165'1030","reported_seq":2761,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208278+0000","last_change":"2024-09-17T07:31:14.105440+0000","last_active":"2024-09-17T07:31:37.208278+0000","last_peered":"2024-09-17T07:31:37.208278+0000","last_clean":"2024-09-17T07:31:37.208278+0000","last_became_active":"2024-09-17T07:28:43.079229+0000","last_became_peered":"2024-09-17T07:28:43.079229+0000","last_unstale":"2024-09-17T07:31:37.208278+0000","last_undegraded":"2024-09-17T07:31:37.208278+0000","last_fullsized":"2024-09-17T07:31:37.208278+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1030,"log_dups_size":0,"ondisk_log_size":1030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:32.835400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015639079999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4625,"num_read_kb":48744,"num_write":1479,"num_write_kb":29297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":83,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145446+0000","last_change":"2024-09-17T07:31:01.268641+0000","last_active":"2024-09-17T07:31:39.145446+0000","last_peered":"2024-09-17T07:31:39.145446+0000","last_clean":"2024-09-17T07:31:39.145446+0000","last_became_active":"2024-09-17T07:31:01.268257+0000","last_became_peered":"2024-09-17T07:31:01.268257+0000","last_unstale":"2024-09-17T07:31:39.145446+0000","last_undegraded":"2024-09-17T07:31:39.145446+0000","last_fullsized":"2024-09-17T07:31:39.145446+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:04.188817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"164'577","reported_seq":3701,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214250+0000","last_change":"2024-09-17T07:31:14.115776+0000","last_active":"2024-09-17T07:31:38.214250+0000","last_peered":"2024-09-17T07:31:38.214250+0000","last_clean":"2024-09-17T07:31:38.214250+0000","last_became_active":"2024-09-17T07:30:19.087787+0000","last_became_peered":"2024-09-17T07:30:19.087787+0000","last_unstale":"2024-09-17T07:31:38.214250+0000","last_undegraded":"2024-09-17T07:31:38.214250+0000","last_fullsized":"2024-09-17T07:31:38.214250+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:26:55.414577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029713869999999998,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":3995,"num_read_kb":36995,"num_write":614,"num_write_kb":22622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":15169480,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":350,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372624+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:31:37.372624+0000","last_peered":"2024-09-17T07:31:37.372624+0000","last_clean":"2024-09-17T07:31:37.372624+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:31:37.372624+0000","last_undegraded":"2024-09-17T07:31:37.372624+0000","last_fullsized":"2024-09-17T07:31:37.372624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"163'1063","reported_seq":3603,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145306+0000","last_change":"2024-09-17T07:31:14.189299+0000","last_active":"2024-09-17T07:31:39.145306+0000","last_peered":"2024-09-17T07:31:39.145306+0000","last_clean":"2024-09-17T07:31:39.145306+0000","last_became_active":"2024-09-17T07:28:43.239387+0000","last_became_peered":"2024-09-17T07:28:43.239387+0000","last_unstale":"2024-09-17T07:31:39.145306+0000","last_undegraded":"2024-09-17T07:31:39.145306+0000","last_fullsized":"2024-09-17T07:31:39.145306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1063,"log_dups_size":0,"ondisk_log_size":1063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:09.407494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080983699999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6807,"num_read_kb":16873,"num_write":1444,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214910+0000","last_change":"2024-09-17T07:30:15.200774+0000","last_active":"2024-09-17T07:31:38.214910+0000","last_peered":"2024-09-17T07:31:38.214910+0000","last_clean":"2024-09-17T07:31:38.214910+0000","last_became_active":"2024-09-17T07:30:15.200470+0000","last_became_peered":"2024-09-17T07:30:15.200470+0000","last_unstale":"2024-09-17T07:31:38.214910+0000","last_undegraded":"2024-09-17T07:31:38.214910+0000","last_fullsized":"2024-09-17T07:31:38.214910+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:58.984439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"164'569","reported_seq":3303,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215021+0000","last_change":"2024-09-17T07:31:14.107519+0000","last_active":"2024-09-17T07:31:38.215021+0000","last_peered":"2024-09-17T07:31:38.215021+0000","last_clean":"2024-09-17T07:31:38.215021+0000","last_became_active":"2024-09-17T07:30:19.085184+0000","last_became_peered":"2024-09-17T07:30:19.085184+0000","last_unstale":"2024-09-17T07:31:38.215021+0000","last_undegraded":"2024-09-17T07:31:38.215021+0000","last_fullsized":"2024-09-17T07:31:38.215021+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:37.515132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015649349999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2687,"num_read_kb":46297,"num_write":632,"num_write_kb":27531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":35691417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"165'668","reported_seq":3157,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372462+0000","last_change":"2024-09-17T07:31:27.102404+0000","last_active":"2024-09-17T07:31:37.372462+0000","last_peered":"2024-09-17T07:31:37.372462+0000","last_clean":"2024-09-17T07:31:37.372462+0000","last_became_active":"2024-09-17T07:31:27.101204+0000","last_became_peered":"2024-09-17T07:31:27.101204+0000","last_unstale":"2024-09-17T07:31:37.372462+0000","last_undegraded":"2024-09-17T07:31:37.372462+0000","last_fullsized":"2024-09-17T07:31:37.372462+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:55.417631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028913630000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":2237,"num_read_kb":35376,"num_write":632,"num_write_kb":16684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":26727758,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":355,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214818+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:31:38.214818+0000","last_peered":"2024-09-17T07:31:38.214818+0000","last_clean":"2024-09-17T07:31:38.214818+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:31:38.214818+0000","last_undegraded":"2024-09-17T07:31:38.214818+0000","last_fullsized":"2024-09-17T07:31:38.214818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"164'825","reported_seq":11345,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145432+0000","last_change":"2024-09-17T07:31:14.189768+0000","last_active":"2024-09-17T07:31:39.145432+0000","last_peered":"2024-09-17T07:31:39.145432+0000","last_clean":"2024-09-17T07:31:39.145432+0000","last_became_active":"2024-09-17T07:28:43.239409+0000","last_became_peered":"2024-09-17T07:28:43.239409+0000","last_unstale":"2024-09-17T07:31:39.145432+0000","last_undegraded":"2024-09-17T07:31:39.145432+0000","last_fullsized":"2024-09-17T07:31:39.145432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":825,"log_dups_size":0,"ondisk_log_size":825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:13.253211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012368310000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":12617,"num_read_kb":91706,"num_write":1240,"num_write_kb":58366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":365,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.380263+0000","last_change":"2024-09-17T07:30:15.201999+0000","last_active":"2024-09-17T07:31:38.380263+0000","last_peered":"2024-09-17T07:31:38.380263+0000","last_clean":"2024-09-17T07:31:38.380263+0000","last_became_active":"2024-09-17T07:30:15.201642+0000","last_became_peered":"2024-09-17T07:30:15.201642+0000","last_unstale":"2024-09-17T07:31:38.380263+0000","last_undegraded":"2024-09-17T07:31:38.380263+0000","last_fullsized":"2024-09-17T07:31:38.380263+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:47.871565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"164'427","reported_seq":5376,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208231+0000","last_change":"2024-09-17T07:31:29.105514+0000","last_active":"2024-09-17T07:31:37.208231+0000","last_peered":"2024-09-17T07:31:37.208231+0000","last_clean":"2024-09-17T07:31:37.208231+0000","last_became_active":"2024-09-17T07:31:29.103782+0000","last_became_peered":"2024-09-17T07:31:29.103782+0000","last_unstale":"2024-09-17T07:31:37.208231+0000","last_undegraded":"2024-09-17T07:31:37.208231+0000","last_fullsized":"2024-09-17T07:31:37.208231+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:52.018922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094238099999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3788,"num_read_kb":51682,"num_write":424,"num_write_kb":28012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":37615005,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":362,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372172+0000","last_change":"2024-09-17T07:31:01.231085+0000","last_active":"2024-09-17T07:31:37.372172+0000","last_peered":"2024-09-17T07:31:37.372172+0000","last_clean":"2024-09-17T07:31:37.372172+0000","last_became_active":"2024-09-17T07:31:01.230794+0000","last_became_peered":"2024-09-17T07:31:01.230794+0000","last_unstale":"2024-09-17T07:31:37.372172+0000","last_undegraded":"2024-09-17T07:31:37.372172+0000","last_fullsized":"2024-09-17T07:31:37.372172+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:25.321726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"165'405","reported_seq":2497,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214012+0000","last_change":"2024-09-17T07:31:14.114890+0000","last_active":"2024-09-17T07:31:38.214012+0000","last_peered":"2024-09-17T07:31:38.214012+0000","last_clean":"2024-09-17T07:31:38.214012+0000","last_became_active":"2024-09-17T07:30:18.084247+0000","last_became_peered":"2024-09-17T07:30:18.084247+0000","last_unstale":"2024-09-17T07:31:38.214012+0000","last_undegraded":"2024-09-17T07:31:38.214012+0000","last_fullsized":"2024-09-17T07:31:38.214012+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:54:33.014267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002225922,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2958,"num_read_kb":9261,"num_write":364,"num_write_kb":6317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":6805602,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":138,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208474+0000","last_change":"2024-09-17T07:30:34.116563+0000","last_active":"2024-09-17T07:31:37.208474+0000","last_peered":"2024-09-17T07:31:37.208474+0000","last_clean":"2024-09-17T07:31:37.208474+0000","last_became_active":"2024-09-17T07:30:34.113598+0000","last_became_peered":"2024-09-17T07:30:34.113598+0000","last_unstale":"2024-09-17T07:31:37.208474+0000","last_undegraded":"2024-09-17T07:31:37.208474+0000","last_fullsized":"2024-09-17T07:31:37.208474+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:52.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":79,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372168+0000","last_change":"2024-09-17T07:31:05.102195+0000","last_active":"2024-09-17T07:31:37.372168+0000","last_peered":"2024-09-17T07:31:37.372168+0000","last_clean":"2024-09-17T07:31:37.372168+0000","last_became_active":"2024-09-17T07:31:05.096988+0000","last_became_peered":"2024-09-17T07:31:05.096988+0000","last_unstale":"2024-09-17T07:31:37.372168+0000","last_undegraded":"2024-09-17T07:31:37.372168+0000","last_fullsized":"2024-09-17T07:31:37.372168+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:01.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"164'520","reported_seq":2435,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214008+0000","last_change":"2024-09-17T07:31:14.113864+0000","last_active":"2024-09-17T07:31:38.214008+0000","last_peered":"2024-09-17T07:31:38.214008+0000","last_clean":"2024-09-17T07:31:38.214008+0000","last_became_active":"2024-09-17T07:30:19.086777+0000","last_became_peered":"2024-09-17T07:30:19.086777+0000","last_unstale":"2024-09-17T07:31:38.214008+0000","last_undegraded":"2024-09-17T07:31:38.214008+0000","last_fullsized":"2024-09-17T07:31:38.214008+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:55.561658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014795710000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1640,"num_read_kb":24382,"num_write":527,"num_write_kb":33950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":24920122,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"164'551","reported_seq":3917,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372211+0000","last_change":"2024-09-17T07:31:27.105903+0000","last_active":"2024-09-17T07:31:37.372211+0000","last_peered":"2024-09-17T07:31:37.372211+0000","last_clean":"2024-09-17T07:31:37.372211+0000","last_became_active":"2024-09-17T07:31:27.103978+0000","last_became_peered":"2024-09-17T07:31:27.103978+0000","last_unstale":"2024-09-17T07:31:37.372211+0000","last_undegraded":"2024-09-17T07:31:37.372211+0000","last_fullsized":"2024-09-17T07:31:37.372211+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:13.427259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001973519,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4515,"num_read_kb":21074,"num_write":530,"num_write_kb":12196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":11434290,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"164'739","reported_seq":9612,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216065+0000","last_change":"2024-09-17T07:31:28.103185+0000","last_active":"2024-09-17T07:31:38.216065+0000","last_peered":"2024-09-17T07:31:38.216065+0000","last_clean":"2024-09-17T07:31:38.216065+0000","last_became_active":"2024-09-17T07:31:28.101926+0000","last_became_peered":"2024-09-17T07:31:28.101926+0000","last_unstale":"2024-09-17T07:31:38.216065+0000","last_undegraded":"2024-09-17T07:31:38.216065+0000","last_fullsized":"2024-09-17T07:31:38.216065+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:17:26.947693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000464964,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":26,"num_read":9368,"num_read_kb":52923,"num_write":688,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":604,"num_bytes_recovered":24254406,"num_keys_recovered":93,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"163'559","reported_seq":2911,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214086+0000","last_change":"2024-09-17T07:31:14.114917+0000","last_active":"2024-09-17T07:31:38.214086+0000","last_peered":"2024-09-17T07:31:38.214086+0000","last_clean":"2024-09-17T07:31:38.214086+0000","last_became_active":"2024-09-17T07:30:19.087515+0000","last_became_peered":"2024-09-17T07:30:19.087515+0000","last_unstale":"2024-09-17T07:31:38.214086+0000","last_undegraded":"2024-09-17T07:31:38.214086+0000","last_fullsized":"2024-09-17T07:31:38.214086+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:42.469193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023177290000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2408,"num_read_kb":29904,"num_write":665,"num_write_kb":15633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":11178824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"165'873","reported_seq":5432,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215146+0000","last_change":"2024-09-17T07:31:14.107447+0000","last_active":"2024-09-17T07:31:38.215146+0000","last_peered":"2024-09-17T07:31:38.215146+0000","last_clean":"2024-09-17T07:31:38.215146+0000","last_became_active":"2024-09-17T07:30:19.085668+0000","last_became_peered":"2024-09-17T07:30:19.085668+0000","last_unstale":"2024-09-17T07:31:38.215146+0000","last_undegraded":"2024-09-17T07:31:38.215146+0000","last_fullsized":"2024-09-17T07:31:38.215146+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:47.391518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001368193,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6070,"num_read_kb":27721,"num_write":1113,"num_write_kb":14827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":15619889,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"165'725","reported_seq":4612,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372484+0000","last_change":"2024-09-17T07:31:14.176026+0000","last_active":"2024-09-17T07:31:37.372484+0000","last_peered":"2024-09-17T07:31:37.372484+0000","last_clean":"2024-09-17T07:31:37.372484+0000","last_became_active":"2024-09-17T07:28:43.080816+0000","last_became_peered":"2024-09-17T07:28:43.080816+0000","last_unstale":"2024-09-17T07:31:37.372484+0000","last_undegraded":"2024-09-17T07:31:37.372484+0000","last_fullsized":"2024-09-17T07:31:37.372484+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:05:06.879033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017783269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6005,"num_read_kb":48957,"num_write":832,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"164'458","reported_seq":8425,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372431+0000","last_change":"2024-09-17T07:31:26.155667+0000","last_active":"2024-09-17T07:31:37.372431+0000","last_peered":"2024-09-17T07:31:37.372431+0000","last_clean":"2024-09-17T07:31:37.372431+0000","last_became_active":"2024-09-17T07:31:26.154754+0000","last_became_peered":"2024-09-17T07:31:26.154754+0000","last_unstale":"2024-09-17T07:31:37.372431+0000","last_undegraded":"2024-09-17T07:31:37.372431+0000","last_fullsized":"2024-09-17T07:31:37.372431+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:47.550445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021346680000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7087,"num_read_kb":55690,"num_write":392,"num_write_kb":41376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":13797349,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"163'832","reported_seq":6338,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372621+0000","last_change":"2024-09-17T07:31:28.103546+0000","last_active":"2024-09-17T07:31:37.372621+0000","last_peered":"2024-09-17T07:31:37.372621+0000","last_clean":"2024-09-17T07:31:37.372621+0000","last_became_active":"2024-09-17T07:31:28.101799+0000","last_became_peered":"2024-09-17T07:31:28.101799+0000","last_unstale":"2024-09-17T07:31:37.372621+0000","last_undegraded":"2024-09-17T07:31:37.372621+0000","last_fullsized":"2024-09-17T07:31:37.372621+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:28.873325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021103049999999998,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6418,"num_read_kb":23171,"num_write":923,"num_write_kb":18541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":744,"num_bytes_recovered":25525458,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"164'401","reported_seq":2768,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216051+0000","last_change":"2024-09-17T07:31:14.720102+0000","last_active":"2024-09-17T07:31:38.216051+0000","last_peered":"2024-09-17T07:31:38.216051+0000","last_clean":"2024-09-17T07:31:38.216051+0000","last_became_active":"2024-09-17T07:28:43.077360+0000","last_became_peered":"2024-09-17T07:28:43.077360+0000","last_unstale":"2024-09-17T07:31:38.216051+0000","last_undegraded":"2024-09-17T07:31:38.216051+0000","last_fullsized":"2024-09-17T07:31:38.216051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:17.256040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061413699999999995,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2841,"num_read_kb":26198,"num_write":450,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"164'802","reported_seq":4337,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372196+0000","last_change":"2024-09-17T07:31:29.105135+0000","last_active":"2024-09-17T07:31:37.372196+0000","last_peered":"2024-09-17T07:31:37.372196+0000","last_clean":"2024-09-17T07:31:37.372196+0000","last_became_active":"2024-09-17T07:31:29.103626+0000","last_became_peered":"2024-09-17T07:31:29.103626+0000","last_unstale":"2024-09-17T07:31:37.372196+0000","last_undegraded":"2024-09-17T07:31:37.372196+0000","last_fullsized":"2024-09-17T07:31:37.372196+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:10.689986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001604722,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5310,"num_read_kb":45214,"num_write":937,"num_write_kb":26421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":468,"num_bytes_recovered":31676674,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"164'435","reported_seq":2745,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208126+0000","last_change":"2024-09-17T07:31:14.104829+0000","last_active":"2024-09-17T07:31:37.208126+0000","last_peered":"2024-09-17T07:31:37.208126+0000","last_clean":"2024-09-17T07:31:37.208126+0000","last_became_active":"2024-09-17T07:31:08.096240+0000","last_became_peered":"2024-09-17T07:31:08.096240+0000","last_unstale":"2024-09-17T07:31:37.208126+0000","last_undegraded":"2024-09-17T07:31:37.208126+0000","last_fullsized":"2024-09-17T07:31:37.208126+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:34.448388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098890999999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2231,"num_read_kb":44562,"num_write":412,"num_write_kb":29148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":24460350,"num_keys_recovered":37,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"164'772","reported_seq":4240,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372356+0000","last_change":"2024-09-17T07:31:29.104804+0000","last_active":"2024-09-17T07:31:37.372356+0000","last_peered":"2024-09-17T07:31:37.372356+0000","last_clean":"2024-09-17T07:31:37.372356+0000","last_became_active":"2024-09-17T07:31:29.103168+0000","last_became_peered":"2024-09-17T07:31:29.103168+0000","last_unstale":"2024-09-17T07:31:37.372356+0000","last_undegraded":"2024-09-17T07:31:37.372356+0000","last_fullsized":"2024-09-17T07:31:37.372356+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:31.292655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093421499999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4511,"num_read_kb":28124,"num_write":1074,"num_write_kb":16615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":21539718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":607,"num_read":166048,"num_read_kb":1208202,"num_write":23179,"num_write_kb":792760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9827,"num_bytes_recovered":666374238,"num_keys_recovered":891,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7902,"internal_metadata":0},"log_size":20553,"ondisk_log_size":20553,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":101032,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542186,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47324,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324516,"statfs":{"total":96636764160,"available":96588304384,"internally_reserved":0,"allocated":2301952,"data_stored":1679580,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33412,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338428,"statfs":{"total":96636764160,"available":96602550272,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55616,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316224,"statfs":{"total":96636764160,"available":96579813376,"internally_reserved":0,"allocated":2269184,"data_stored":1095102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542186,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91640,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280200,"statfs":{"total":96636764160,"available":96542924800,"internally_reserved":0,"allocated":2256896,"data_stored":1089231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056662,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43268,"kb_used_data":2212,"kb_used_omap":16,"kb_used_meta":41007,"kb_avail":94328572,"statfs":{"total":96636764160,"available":96592457728,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17058,"internal_metadata":41991518},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542186,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55612,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316228,"statfs":{"total":96636764160,"available":96579817472,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65336,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306504,"statfs":{"total":96636764160,"available":96569860096,"internally_reserved":0,"allocated":2260992,"data_stored":1089250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47340,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324500,"statfs":{"total":96636764160,"available":96588288000,"internally_reserved":0,"allocated":2318336,"data_stored":1679580,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7902,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:31:41.843 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:31:42.143 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:42.144 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:31:42.155 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-17T07:31:40.788312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":607,"num_read":166114,"num_read_kb":1208257,"num_write":23282,"num_write_kb":794390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9832,"num_bytes_recovered":667554974,"num_keys_recovered":891,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20628,"ondisk_log_size":20628,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439548,"kb_used_data":17776,"kb_used_omap":27,"kb_used_meta":421540,"kb_avail":754535172,"statfs":{"total":773094113280,"available":772644016128,"internally_reserved":0,"allocated":18202624,"data_stored":9917992,"data_compressed":66815,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":28217,"internal_metadata":431657415},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.593244"},"pg_stats":[{"pgid":"3.1f","version":"165'874","reported_seq":4617,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208112+0000","last_change":"2024-09-17T07:31:14.105507+0000","last_active":"2024-09-17T07:31:37.208112+0000","last_peered":"2024-09-17T07:31:37.208112+0000","last_clean":"2024-09-17T07:31:37.208112+0000","last_became_active":"2024-09-17T07:30:43.092456+0000","last_became_peered":"2024-09-17T07:30:43.092456+0000","last_unstale":"2024-09-17T07:31:37.208112+0000","last_undegraded":"2024-09-17T07:31:37.208112+0000","last_fullsized":"2024-09-17T07:31:37.208112+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:26.811072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016105900000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5801,"num_read_kb":34310,"num_write":997,"num_write_kb":41977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":26159662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"164'553","reported_seq":3285,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214114+0000","last_change":"2024-09-17T07:31:14.115590+0000","last_active":"2024-09-17T07:31:38.214114+0000","last_peered":"2024-09-17T07:31:38.214114+0000","last_clean":"2024-09-17T07:31:38.214114+0000","last_became_active":"2024-09-17T07:30:19.087213+0000","last_became_peered":"2024-09-17T07:30:19.087213+0000","last_unstale":"2024-09-17T07:31:38.214114+0000","last_undegraded":"2024-09-17T07:31:38.214114+0000","last_fullsized":"2024-09-17T07:31:38.214114+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:15.610729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029487620000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3721,"num_read_kb":30298,"num_write":527,"num_write_kb":15594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":15743718,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"163'550","reported_seq":3498,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214875+0000","last_change":"2024-09-17T07:31:26.150683+0000","last_active":"2024-09-17T07:31:38.214875+0000","last_peered":"2024-09-17T07:31:38.214875+0000","last_clean":"2024-09-17T07:31:38.214875+0000","last_became_active":"2024-09-17T07:31:26.149428+0000","last_became_peered":"2024-09-17T07:31:26.149428+0000","last_unstale":"2024-09-17T07:31:38.214875+0000","last_undegraded":"2024-09-17T07:31:38.214875+0000","last_fullsized":"2024-09-17T07:31:38.214875+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:58.834946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016933180000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":1948,"num_read_kb":28759,"num_write":579,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":27967854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"165'813","reported_seq":6628,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215109+0000","last_change":"2024-09-17T07:31:28.101214+0000","last_active":"2024-09-17T07:31:38.215109+0000","last_peered":"2024-09-17T07:31:38.215109+0000","last_clean":"2024-09-17T07:31:38.215109+0000","last_became_active":"2024-09-17T07:31:28.100048+0000","last_became_peered":"2024-09-17T07:31:28.100048+0000","last_unstale":"2024-09-17T07:31:38.215109+0000","last_undegraded":"2024-09-17T07:31:38.215109+0000","last_fullsized":"2024-09-17T07:31:38.215109+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:35.020631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010655949999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":7523,"num_read_kb":42454,"num_write":1061,"num_write_kb":38721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":26661650,"num_keys_recovered":180,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"165'606","reported_seq":5639,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208349+0000","last_change":"2024-09-17T07:31:28.104670+0000","last_active":"2024-09-17T07:31:37.208349+0000","last_peered":"2024-09-17T07:31:37.208349+0000","last_clean":"2024-09-17T07:31:37.208349+0000","last_became_active":"2024-09-17T07:31:28.102058+0000","last_became_peered":"2024-09-17T07:31:28.102058+0000","last_unstale":"2024-09-17T07:31:37.208349+0000","last_undegraded":"2024-09-17T07:31:37.208349+0000","last_fullsized":"2024-09-17T07:31:37.208349+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:07.797309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028238529999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5800,"num_read_kb":52463,"num_write":596,"num_write_kb":22879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":91784745,"num_keys_recovered":126,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"163'920","reported_seq":14894,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372319+0000","last_change":"2024-09-17T07:31:27.105439+0000","last_active":"2024-09-17T07:31:37.372319+0000","last_peered":"2024-09-17T07:31:37.372319+0000","last_clean":"2024-09-17T07:31:37.372319+0000","last_became_active":"2024-09-17T07:31:27.103933+0000","last_became_peered":"2024-09-17T07:31:27.103933+0000","last_unstale":"2024-09-17T07:31:37.372319+0000","last_undegraded":"2024-09-17T07:31:37.372319+0000","last_fullsized":"2024-09-17T07:31:37.372319+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:06:20.908980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096620079999999997,"stat_sum":{"num_bytes":19,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":16715,"num_read_kb":60981,"num_write":1188,"num_write_kb":20597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":19562028,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"164'439","reported_seq":5685,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145330+0000","last_change":"2024-09-17T07:31:14.189296+0000","last_active":"2024-09-17T07:31:39.145330+0000","last_peered":"2024-09-17T07:31:39.145330+0000","last_clean":"2024-09-17T07:31:39.145330+0000","last_became_active":"2024-09-17T07:30:18.083918+0000","last_became_peered":"2024-09-17T07:30:18.083918+0000","last_unstale":"2024-09-17T07:31:39.145330+0000","last_undegraded":"2024-09-17T07:31:39.145330+0000","last_fullsized":"2024-09-17T07:31:39.145330+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:26.983426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000820972,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4352,"num_read_kb":37354,"num_write":400,"num_write_kb":43089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":25536146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'763","reported_seq":4669,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214008+0000","last_change":"2024-09-17T07:31:14.113881+0000","last_active":"2024-09-17T07:31:38.214008+0000","last_peered":"2024-09-17T07:31:38.214008+0000","last_clean":"2024-09-17T07:31:38.214008+0000","last_became_active":"2024-09-17T07:30:43.091692+0000","last_became_peered":"2024-09-17T07:30:43.091692+0000","last_unstale":"2024-09-17T07:31:38.214008+0000","last_undegraded":"2024-09-17T07:31:38.214008+0000","last_fullsized":"2024-09-17T07:31:38.214008+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:24.550163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014749730000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5969,"num_read_kb":25327,"num_write":792,"num_write_kb":16603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":29456184,"num_keys_recovered":176,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"163'436","reported_seq":4226,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208396+0000","last_change":"2024-09-17T07:31:14.106118+0000","last_active":"2024-09-17T07:31:37.208396+0000","last_peered":"2024-09-17T07:31:37.208396+0000","last_clean":"2024-09-17T07:31:37.208396+0000","last_became_active":"2024-09-17T07:28:43.080428+0000","last_became_peered":"2024-09-17T07:28:43.080428+0000","last_unstale":"2024-09-17T07:31:37.208396+0000","last_undegraded":"2024-09-17T07:31:37.208396+0000","last_fullsized":"2024-09-17T07:31:37.208396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:12:14.112234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020783120000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4961,"num_read_kb":22717,"num_write":365,"num_write_kb":13531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"164'513","reported_seq":4191,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214897+0000","last_change":"2024-09-17T07:31:14.106949+0000","last_active":"2024-09-17T07:31:38.214897+0000","last_peered":"2024-09-17T07:31:38.214897+0000","last_clean":"2024-09-17T07:31:38.214897+0000","last_became_active":"2024-09-17T07:28:43.082313+0000","last_became_peered":"2024-09-17T07:28:43.082313+0000","last_unstale":"2024-09-17T07:31:38.214897+0000","last_undegraded":"2024-09-17T07:31:38.214897+0000","last_fullsized":"2024-09-17T07:31:38.214897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:35.352537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045261,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6140,"num_read_kb":58359,"num_write":449,"num_write_kb":31855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"164'390","reported_seq":4504,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372480+0000","last_change":"2024-09-17T07:31:14.175336+0000","last_active":"2024-09-17T07:31:37.372480+0000","last_peered":"2024-09-17T07:31:37.372480+0000","last_clean":"2024-09-17T07:31:37.372480+0000","last_became_active":"2024-09-17T07:31:08.094940+0000","last_became_peered":"2024-09-17T07:31:08.094940+0000","last_unstale":"2024-09-17T07:31:37.372480+0000","last_undegraded":"2024-09-17T07:31:37.372480+0000","last_fullsized":"2024-09-17T07:31:37.372480+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:58.292675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011121970000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3586,"num_read_kb":32352,"num_write":371,"num_write_kb":22781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":431,"num_bytes_recovered":60567103,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"163'465","reported_seq":2104,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372357+0000","last_change":"2024-09-17T07:31:26.153704+0000","last_active":"2024-09-17T07:31:37.372357+0000","last_peered":"2024-09-17T07:31:37.372357+0000","last_clean":"2024-09-17T07:31:37.372357+0000","last_became_active":"2024-09-17T07:31:26.152423+0000","last_became_peered":"2024-09-17T07:31:26.152423+0000","last_unstale":"2024-09-17T07:31:37.372357+0000","last_undegraded":"2024-09-17T07:31:37.372357+0000","last_fullsized":"2024-09-17T07:31:37.372357+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:46.055274+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015069525,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1414,"num_read_kb":17974,"num_write":482,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":16518806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":363,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216133+0000","last_change":"2024-09-17T07:31:05.098106+0000","last_active":"2024-09-17T07:31:38.216133+0000","last_peered":"2024-09-17T07:31:38.216133+0000","last_clean":"2024-09-17T07:31:38.216133+0000","last_became_active":"2024-09-17T07:31:05.097821+0000","last_became_peered":"2024-09-17T07:31:05.097821+0000","last_unstale":"2024-09-17T07:31:38.216133+0000","last_undegraded":"2024-09-17T07:31:38.216133+0000","last_fullsized":"2024-09-17T07:31:38.216133+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:12.297890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"165'1030","reported_seq":2761,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208278+0000","last_change":"2024-09-17T07:31:14.105440+0000","last_active":"2024-09-17T07:31:37.208278+0000","last_peered":"2024-09-17T07:31:37.208278+0000","last_clean":"2024-09-17T07:31:37.208278+0000","last_became_active":"2024-09-17T07:28:43.079229+0000","last_became_peered":"2024-09-17T07:28:43.079229+0000","last_unstale":"2024-09-17T07:31:37.208278+0000","last_undegraded":"2024-09-17T07:31:37.208278+0000","last_fullsized":"2024-09-17T07:31:37.208278+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1030,"log_dups_size":0,"ondisk_log_size":1030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:32.835400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015639079999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4625,"num_read_kb":48744,"num_write":1479,"num_write_kb":29297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":83,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145446+0000","last_change":"2024-09-17T07:31:01.268641+0000","last_active":"2024-09-17T07:31:39.145446+0000","last_peered":"2024-09-17T07:31:39.145446+0000","last_clean":"2024-09-17T07:31:39.145446+0000","last_became_active":"2024-09-17T07:31:01.268257+0000","last_became_peered":"2024-09-17T07:31:01.268257+0000","last_unstale":"2024-09-17T07:31:39.145446+0000","last_undegraded":"2024-09-17T07:31:39.145446+0000","last_fullsized":"2024-09-17T07:31:39.145446+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:04.188817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"164'577","reported_seq":3701,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214250+0000","last_change":"2024-09-17T07:31:14.115776+0000","last_active":"2024-09-17T07:31:38.214250+0000","last_peered":"2024-09-17T07:31:38.214250+0000","last_clean":"2024-09-17T07:31:38.214250+0000","last_became_active":"2024-09-17T07:30:19.087787+0000","last_became_peered":"2024-09-17T07:30:19.087787+0000","last_unstale":"2024-09-17T07:31:38.214250+0000","last_undegraded":"2024-09-17T07:31:38.214250+0000","last_fullsized":"2024-09-17T07:31:38.214250+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:26:55.414577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029713869999999998,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":3995,"num_read_kb":36995,"num_write":614,"num_write_kb":22622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":15169480,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":350,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372624+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:31:37.372624+0000","last_peered":"2024-09-17T07:31:37.372624+0000","last_clean":"2024-09-17T07:31:37.372624+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:31:37.372624+0000","last_undegraded":"2024-09-17T07:31:37.372624+0000","last_fullsized":"2024-09-17T07:31:37.372624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"163'1063","reported_seq":3603,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145306+0000","last_change":"2024-09-17T07:31:14.189299+0000","last_active":"2024-09-17T07:31:39.145306+0000","last_peered":"2024-09-17T07:31:39.145306+0000","last_clean":"2024-09-17T07:31:39.145306+0000","last_became_active":"2024-09-17T07:28:43.239387+0000","last_became_peered":"2024-09-17T07:28:43.239387+0000","last_unstale":"2024-09-17T07:31:39.145306+0000","last_undegraded":"2024-09-17T07:31:39.145306+0000","last_fullsized":"2024-09-17T07:31:39.145306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1063,"log_dups_size":0,"ondisk_log_size":1063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:09.407494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080983699999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6807,"num_read_kb":16873,"num_write":1444,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214910+0000","last_change":"2024-09-17T07:30:15.200774+0000","last_active":"2024-09-17T07:31:38.214910+0000","last_peered":"2024-09-17T07:31:38.214910+0000","last_clean":"2024-09-17T07:31:38.214910+0000","last_became_active":"2024-09-17T07:30:15.200470+0000","last_became_peered":"2024-09-17T07:30:15.200470+0000","last_unstale":"2024-09-17T07:31:38.214910+0000","last_undegraded":"2024-09-17T07:31:38.214910+0000","last_fullsized":"2024-09-17T07:31:38.214910+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:58.984439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"164'569","reported_seq":3303,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215021+0000","last_change":"2024-09-17T07:31:14.107519+0000","last_active":"2024-09-17T07:31:38.215021+0000","last_peered":"2024-09-17T07:31:38.215021+0000","last_clean":"2024-09-17T07:31:38.215021+0000","last_became_active":"2024-09-17T07:30:19.085184+0000","last_became_peered":"2024-09-17T07:30:19.085184+0000","last_unstale":"2024-09-17T07:31:38.215021+0000","last_undegraded":"2024-09-17T07:31:38.215021+0000","last_fullsized":"2024-09-17T07:31:38.215021+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:37.515132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015649349999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2687,"num_read_kb":46297,"num_write":632,"num_write_kb":27531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":35691417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"165'668","reported_seq":3157,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372462+0000","last_change":"2024-09-17T07:31:27.102404+0000","last_active":"2024-09-17T07:31:37.372462+0000","last_peered":"2024-09-17T07:31:37.372462+0000","last_clean":"2024-09-17T07:31:37.372462+0000","last_became_active":"2024-09-17T07:31:27.101204+0000","last_became_peered":"2024-09-17T07:31:27.101204+0000","last_unstale":"2024-09-17T07:31:37.372462+0000","last_undegraded":"2024-09-17T07:31:37.372462+0000","last_fullsized":"2024-09-17T07:31:37.372462+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:55.417631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028913630000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":2237,"num_read_kb":35376,"num_write":632,"num_write_kb":16684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":26727758,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":355,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214818+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:31:38.214818+0000","last_peered":"2024-09-17T07:31:38.214818+0000","last_clean":"2024-09-17T07:31:38.214818+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:31:38.214818+0000","last_undegraded":"2024-09-17T07:31:38.214818+0000","last_fullsized":"2024-09-17T07:31:38.214818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"164'825","reported_seq":11345,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145432+0000","last_change":"2024-09-17T07:31:14.189768+0000","last_active":"2024-09-17T07:31:39.145432+0000","last_peered":"2024-09-17T07:31:39.145432+0000","last_clean":"2024-09-17T07:31:39.145432+0000","last_became_active":"2024-09-17T07:28:43.239409+0000","last_became_peered":"2024-09-17T07:28:43.239409+0000","last_unstale":"2024-09-17T07:31:39.145432+0000","last_undegraded":"2024-09-17T07:31:39.145432+0000","last_fullsized":"2024-09-17T07:31:39.145432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":825,"log_dups_size":0,"ondisk_log_size":825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:13.253211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012368310000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":12617,"num_read_kb":91706,"num_write":1240,"num_write_kb":58366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":365,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.380263+0000","last_change":"2024-09-17T07:30:15.201999+0000","last_active":"2024-09-17T07:31:38.380263+0000","last_peered":"2024-09-17T07:31:38.380263+0000","last_clean":"2024-09-17T07:31:38.380263+0000","last_became_active":"2024-09-17T07:30:15.201642+0000","last_became_peered":"2024-09-17T07:30:15.201642+0000","last_unstale":"2024-09-17T07:31:38.380263+0000","last_undegraded":"2024-09-17T07:31:38.380263+0000","last_fullsized":"2024-09-17T07:31:38.380263+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:47.871565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"164'427","reported_seq":5376,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208231+0000","last_change":"2024-09-17T07:31:29.105514+0000","last_active":"2024-09-17T07:31:37.208231+0000","last_peered":"2024-09-17T07:31:37.208231+0000","last_clean":"2024-09-17T07:31:37.208231+0000","last_became_active":"2024-09-17T07:31:29.103782+0000","last_became_peered":"2024-09-17T07:31:29.103782+0000","last_unstale":"2024-09-17T07:31:37.208231+0000","last_undegraded":"2024-09-17T07:31:37.208231+0000","last_fullsized":"2024-09-17T07:31:37.208231+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:52.018922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094238099999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3788,"num_read_kb":51682,"num_write":424,"num_write_kb":28012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":37615005,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":362,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372172+0000","last_change":"2024-09-17T07:31:01.231085+0000","last_active":"2024-09-17T07:31:37.372172+0000","last_peered":"2024-09-17T07:31:37.372172+0000","last_clean":"2024-09-17T07:31:37.372172+0000","last_became_active":"2024-09-17T07:31:01.230794+0000","last_became_peered":"2024-09-17T07:31:01.230794+0000","last_unstale":"2024-09-17T07:31:37.372172+0000","last_undegraded":"2024-09-17T07:31:37.372172+0000","last_fullsized":"2024-09-17T07:31:37.372172+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:25.321726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"165'405","reported_seq":2497,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214012+0000","last_change":"2024-09-17T07:31:14.114890+0000","last_active":"2024-09-17T07:31:38.214012+0000","last_peered":"2024-09-17T07:31:38.214012+0000","last_clean":"2024-09-17T07:31:38.214012+0000","last_became_active":"2024-09-17T07:30:18.084247+0000","last_became_peered":"2024-09-17T07:30:18.084247+0000","last_unstale":"2024-09-17T07:31:38.214012+0000","last_undegraded":"2024-09-17T07:31:38.214012+0000","last_fullsized":"2024-09-17T07:31:38.214012+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:54:33.014267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002225922,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2958,"num_read_kb":9261,"num_write":364,"num_write_kb":6317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":6805602,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":138,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208474+0000","last_change":"2024-09-17T07:30:34.116563+0000","last_active":"2024-09-17T07:31:37.208474+0000","last_peered":"2024-09-17T07:31:37.208474+0000","last_clean":"2024-09-17T07:31:37.208474+0000","last_became_active":"2024-09-17T07:30:34.113598+0000","last_became_peered":"2024-09-17T07:30:34.113598+0000","last_unstale":"2024-09-17T07:31:37.208474+0000","last_undegraded":"2024-09-17T07:31:37.208474+0000","last_fullsized":"2024-09-17T07:31:37.208474+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:52.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":79,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372168+0000","last_change":"2024-09-17T07:31:05.102195+0000","last_active":"2024-09-17T07:31:37.372168+0000","last_peered":"2024-09-17T07:31:37.372168+0000","last_clean":"2024-09-17T07:31:37.372168+0000","last_became_active":"2024-09-17T07:31:05.096988+0000","last_became_peered":"2024-09-17T07:31:05.096988+0000","last_unstale":"2024-09-17T07:31:37.372168+0000","last_undegraded":"2024-09-17T07:31:37.372168+0000","last_fullsized":"2024-09-17T07:31:37.372168+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:01.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"164'520","reported_seq":2435,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214008+0000","last_change":"2024-09-17T07:31:14.113864+0000","last_active":"2024-09-17T07:31:38.214008+0000","last_peered":"2024-09-17T07:31:38.214008+0000","last_clean":"2024-09-17T07:31:38.214008+0000","last_became_active":"2024-09-17T07:30:19.086777+0000","last_became_peered":"2024-09-17T07:30:19.086777+0000","last_unstale":"2024-09-17T07:31:38.214008+0000","last_undegraded":"2024-09-17T07:31:38.214008+0000","last_fullsized":"2024-09-17T07:31:38.214008+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:55.561658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014795710000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1640,"num_read_kb":24382,"num_write":527,"num_write_kb":33950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":24920122,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"164'551","reported_seq":3917,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372211+0000","last_change":"2024-09-17T07:31:27.105903+0000","last_active":"2024-09-17T07:31:37.372211+0000","last_peered":"2024-09-17T07:31:37.372211+0000","last_clean":"2024-09-17T07:31:37.372211+0000","last_became_active":"2024-09-17T07:31:27.103978+0000","last_became_peered":"2024-09-17T07:31:27.103978+0000","last_unstale":"2024-09-17T07:31:37.372211+0000","last_undegraded":"2024-09-17T07:31:37.372211+0000","last_fullsized":"2024-09-17T07:31:37.372211+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:13.427259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001973519,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4515,"num_read_kb":21074,"num_write":530,"num_write_kb":12196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":11434290,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"164'739","reported_seq":9612,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216065+0000","last_change":"2024-09-17T07:31:28.103185+0000","last_active":"2024-09-17T07:31:38.216065+0000","last_peered":"2024-09-17T07:31:38.216065+0000","last_clean":"2024-09-17T07:31:38.216065+0000","last_became_active":"2024-09-17T07:31:28.101926+0000","last_became_peered":"2024-09-17T07:31:28.101926+0000","last_unstale":"2024-09-17T07:31:38.216065+0000","last_undegraded":"2024-09-17T07:31:38.216065+0000","last_fullsized":"2024-09-17T07:31:38.216065+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:17:26.947693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000464964,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":26,"num_read":9368,"num_read_kb":52923,"num_write":688,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":604,"num_bytes_recovered":24254406,"num_keys_recovered":93,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"163'559","reported_seq":2911,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214086+0000","last_change":"2024-09-17T07:31:14.114917+0000","last_active":"2024-09-17T07:31:38.214086+0000","last_peered":"2024-09-17T07:31:38.214086+0000","last_clean":"2024-09-17T07:31:38.214086+0000","last_became_active":"2024-09-17T07:30:19.087515+0000","last_became_peered":"2024-09-17T07:30:19.087515+0000","last_unstale":"2024-09-17T07:31:38.214086+0000","last_undegraded":"2024-09-17T07:31:38.214086+0000","last_fullsized":"2024-09-17T07:31:38.214086+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:42.469193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023177290000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2408,"num_read_kb":29904,"num_write":665,"num_write_kb":15633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":11178824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"165'873","reported_seq":5432,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215146+0000","last_change":"2024-09-17T07:31:14.107447+0000","last_active":"2024-09-17T07:31:38.215146+0000","last_peered":"2024-09-17T07:31:38.215146+0000","last_clean":"2024-09-17T07:31:38.215146+0000","last_became_active":"2024-09-17T07:30:19.085668+0000","last_became_peered":"2024-09-17T07:30:19.085668+0000","last_unstale":"2024-09-17T07:31:38.215146+0000","last_undegraded":"2024-09-17T07:31:38.215146+0000","last_fullsized":"2024-09-17T07:31:38.215146+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:47.391518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001368193,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6070,"num_read_kb":27721,"num_write":1113,"num_write_kb":14827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":15619889,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"165'725","reported_seq":4612,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372484+0000","last_change":"2024-09-17T07:31:14.176026+0000","last_active":"2024-09-17T07:31:37.372484+0000","last_peered":"2024-09-17T07:31:37.372484+0000","last_clean":"2024-09-17T07:31:37.372484+0000","last_became_active":"2024-09-17T07:28:43.080816+0000","last_became_peered":"2024-09-17T07:28:43.080816+0000","last_unstale":"2024-09-17T07:31:37.372484+0000","last_undegraded":"2024-09-17T07:31:37.372484+0000","last_fullsized":"2024-09-17T07:31:37.372484+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:05:06.879033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017783269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6005,"num_read_kb":48957,"num_write":832,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"164'458","reported_seq":8425,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372431+0000","last_change":"2024-09-17T07:31:26.155667+0000","last_active":"2024-09-17T07:31:37.372431+0000","last_peered":"2024-09-17T07:31:37.372431+0000","last_clean":"2024-09-17T07:31:37.372431+0000","last_became_active":"2024-09-17T07:31:26.154754+0000","last_became_peered":"2024-09-17T07:31:26.154754+0000","last_unstale":"2024-09-17T07:31:37.372431+0000","last_undegraded":"2024-09-17T07:31:37.372431+0000","last_fullsized":"2024-09-17T07:31:37.372431+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:47.550445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021346680000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7087,"num_read_kb":55690,"num_write":392,"num_write_kb":41376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":13797349,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"163'832","reported_seq":6338,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372621+0000","last_change":"2024-09-17T07:31:28.103546+0000","last_active":"2024-09-17T07:31:37.372621+0000","last_peered":"2024-09-17T07:31:37.372621+0000","last_clean":"2024-09-17T07:31:37.372621+0000","last_became_active":"2024-09-17T07:31:28.101799+0000","last_became_peered":"2024-09-17T07:31:28.101799+0000","last_unstale":"2024-09-17T07:31:37.372621+0000","last_undegraded":"2024-09-17T07:31:37.372621+0000","last_fullsized":"2024-09-17T07:31:37.372621+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:28.873325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021103049999999998,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6418,"num_read_kb":23171,"num_write":923,"num_write_kb":18541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":744,"num_bytes_recovered":25525458,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"164'401","reported_seq":2768,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216051+0000","last_change":"2024-09-17T07:31:14.720102+0000","last_active":"2024-09-17T07:31:38.216051+0000","last_peered":"2024-09-17T07:31:38.216051+0000","last_clean":"2024-09-17T07:31:38.216051+0000","last_became_active":"2024-09-17T07:28:43.077360+0000","last_became_peered":"2024-09-17T07:28:43.077360+0000","last_unstale":"2024-09-17T07:31:38.216051+0000","last_undegraded":"2024-09-17T07:31:38.216051+0000","last_fullsized":"2024-09-17T07:31:38.216051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:17.256040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061413699999999995,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2841,"num_read_kb":26198,"num_write":450,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"164'802","reported_seq":4337,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372196+0000","last_change":"2024-09-17T07:31:29.105135+0000","last_active":"2024-09-17T07:31:37.372196+0000","last_peered":"2024-09-17T07:31:37.372196+0000","last_clean":"2024-09-17T07:31:37.372196+0000","last_became_active":"2024-09-17T07:31:29.103626+0000","last_became_peered":"2024-09-17T07:31:29.103626+0000","last_unstale":"2024-09-17T07:31:37.372196+0000","last_undegraded":"2024-09-17T07:31:37.372196+0000","last_fullsized":"2024-09-17T07:31:37.372196+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:10.689986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001604722,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5310,"num_read_kb":45214,"num_write":937,"num_write_kb":26421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":468,"num_bytes_recovered":31676674,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"164'435","reported_seq":2745,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208126+0000","last_change":"2024-09-17T07:31:14.104829+0000","last_active":"2024-09-17T07:31:37.208126+0000","last_peered":"2024-09-17T07:31:37.208126+0000","last_clean":"2024-09-17T07:31:37.208126+0000","last_became_active":"2024-09-17T07:31:08.096240+0000","last_became_peered":"2024-09-17T07:31:08.096240+0000","last_unstale":"2024-09-17T07:31:37.208126+0000","last_undegraded":"2024-09-17T07:31:37.208126+0000","last_fullsized":"2024-09-17T07:31:37.208126+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:34.448388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098890999999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2231,"num_read_kb":44562,"num_write":412,"num_write_kb":29148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":24460350,"num_keys_recovered":37,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"164'772","reported_seq":4240,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372356+0000","last_change":"2024-09-17T07:31:29.104804+0000","last_active":"2024-09-17T07:31:37.372356+0000","last_peered":"2024-09-17T07:31:37.372356+0000","last_clean":"2024-09-17T07:31:37.372356+0000","last_became_active":"2024-09-17T07:31:29.103168+0000","last_became_peered":"2024-09-17T07:31:29.103168+0000","last_unstale":"2024-09-17T07:31:37.372356+0000","last_undegraded":"2024-09-17T07:31:37.372356+0000","last_fullsized":"2024-09-17T07:31:37.372356+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:31.292655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093421499999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4511,"num_read_kb":28124,"num_write":1074,"num_write_kb":16615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":21539718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":607,"num_read":166048,"num_read_kb":1208202,"num_write":23179,"num_write_kb":792760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9827,"num_bytes_recovered":666374238,"num_keys_recovered":891,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7902,"internal_metadata":0},"log_size":20553,"ondisk_log_size":20553,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":101032,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542186,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47324,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324516,"statfs":{"total":96636764160,"available":96588304384,"internally_reserved":0,"allocated":2301952,"data_stored":1679580,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33412,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338428,"statfs":{"total":96636764160,"available":96602550272,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55616,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316224,"statfs":{"total":96636764160,"available":96579813376,"internally_reserved":0,"allocated":2269184,"data_stored":1095102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542186,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91640,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280200,"statfs":{"total":96636764160,"available":96542924800,"internally_reserved":0,"allocated":2256896,"data_stored":1089231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056662,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43268,"kb_used_data":2212,"kb_used_omap":16,"kb_used_meta":41007,"kb_avail":94328572,"statfs":{"total":96636764160,"available":96592457728,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17058,"internal_metadata":41991518},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542186,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55612,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316228,"statfs":{"total":96636764160,"available":96579817472,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65336,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306504,"statfs":{"total":96636764160,"available":96569860096,"internally_reserved":0,"allocated":2260992,"data_stored":1089250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47340,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324500,"statfs":{"total":96636764160,"available":96588288000,"internally_reserved":0,"allocated":2318336,"data_stored":1679580,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7902,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:31:42.158 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T07:31:42.158 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-17T07:31:42.476 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T07:31:42.486 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T07:31:42.486 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:31:42.783 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:42.783 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:31:42.795 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-17T07:31:40.788312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":607,"num_read":166114,"num_read_kb":1208257,"num_write":23282,"num_write_kb":794390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9832,"num_bytes_recovered":667554974,"num_keys_recovered":891,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20628,"ondisk_log_size":20628,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439548,"kb_used_data":17776,"kb_used_omap":27,"kb_used_meta":421540,"kb_avail":754535172,"statfs":{"total":773094113280,"available":772644016128,"internally_reserved":0,"allocated":18202624,"data_stored":9917992,"data_compressed":66815,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":28217,"internal_metadata":431657415},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.593244"},"pg_stats":[{"pgid":"3.1f","version":"165'874","reported_seq":4617,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208112+0000","last_change":"2024-09-17T07:31:14.105507+0000","last_active":"2024-09-17T07:31:37.208112+0000","last_peered":"2024-09-17T07:31:37.208112+0000","last_clean":"2024-09-17T07:31:37.208112+0000","last_became_active":"2024-09-17T07:30:43.092456+0000","last_became_peered":"2024-09-17T07:30:43.092456+0000","last_unstale":"2024-09-17T07:31:37.208112+0000","last_undegraded":"2024-09-17T07:31:37.208112+0000","last_fullsized":"2024-09-17T07:31:37.208112+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:26.811072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016105900000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5801,"num_read_kb":34310,"num_write":997,"num_write_kb":41977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":26159662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"164'553","reported_seq":3285,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214114+0000","last_change":"2024-09-17T07:31:14.115590+0000","last_active":"2024-09-17T07:31:38.214114+0000","last_peered":"2024-09-17T07:31:38.214114+0000","last_clean":"2024-09-17T07:31:38.214114+0000","last_became_active":"2024-09-17T07:30:19.087213+0000","last_became_peered":"2024-09-17T07:30:19.087213+0000","last_unstale":"2024-09-17T07:31:38.214114+0000","last_undegraded":"2024-09-17T07:31:38.214114+0000","last_fullsized":"2024-09-17T07:31:38.214114+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:15.610729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029487620000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3721,"num_read_kb":30298,"num_write":527,"num_write_kb":15594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":15743718,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"163'550","reported_seq":3498,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214875+0000","last_change":"2024-09-17T07:31:26.150683+0000","last_active":"2024-09-17T07:31:38.214875+0000","last_peered":"2024-09-17T07:31:38.214875+0000","last_clean":"2024-09-17T07:31:38.214875+0000","last_became_active":"2024-09-17T07:31:26.149428+0000","last_became_peered":"2024-09-17T07:31:26.149428+0000","last_unstale":"2024-09-17T07:31:38.214875+0000","last_undegraded":"2024-09-17T07:31:38.214875+0000","last_fullsized":"2024-09-17T07:31:38.214875+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:58.834946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016933180000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":1948,"num_read_kb":28759,"num_write":579,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":27967854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"165'813","reported_seq":6628,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215109+0000","last_change":"2024-09-17T07:31:28.101214+0000","last_active":"2024-09-17T07:31:38.215109+0000","last_peered":"2024-09-17T07:31:38.215109+0000","last_clean":"2024-09-17T07:31:38.215109+0000","last_became_active":"2024-09-17T07:31:28.100048+0000","last_became_peered":"2024-09-17T07:31:28.100048+0000","last_unstale":"2024-09-17T07:31:38.215109+0000","last_undegraded":"2024-09-17T07:31:38.215109+0000","last_fullsized":"2024-09-17T07:31:38.215109+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:35.020631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010655949999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":7523,"num_read_kb":42454,"num_write":1061,"num_write_kb":38721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":26661650,"num_keys_recovered":180,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"165'606","reported_seq":5639,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208349+0000","last_change":"2024-09-17T07:31:28.104670+0000","last_active":"2024-09-17T07:31:37.208349+0000","last_peered":"2024-09-17T07:31:37.208349+0000","last_clean":"2024-09-17T07:31:37.208349+0000","last_became_active":"2024-09-17T07:31:28.102058+0000","last_became_peered":"2024-09-17T07:31:28.102058+0000","last_unstale":"2024-09-17T07:31:37.208349+0000","last_undegraded":"2024-09-17T07:31:37.208349+0000","last_fullsized":"2024-09-17T07:31:37.208349+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:07.797309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028238529999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5800,"num_read_kb":52463,"num_write":596,"num_write_kb":22879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":91784745,"num_keys_recovered":126,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"163'920","reported_seq":14894,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372319+0000","last_change":"2024-09-17T07:31:27.105439+0000","last_active":"2024-09-17T07:31:37.372319+0000","last_peered":"2024-09-17T07:31:37.372319+0000","last_clean":"2024-09-17T07:31:37.372319+0000","last_became_active":"2024-09-17T07:31:27.103933+0000","last_became_peered":"2024-09-17T07:31:27.103933+0000","last_unstale":"2024-09-17T07:31:37.372319+0000","last_undegraded":"2024-09-17T07:31:37.372319+0000","last_fullsized":"2024-09-17T07:31:37.372319+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:06:20.908980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096620079999999997,"stat_sum":{"num_bytes":19,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":16715,"num_read_kb":60981,"num_write":1188,"num_write_kb":20597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":19562028,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"164'439","reported_seq":5685,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145330+0000","last_change":"2024-09-17T07:31:14.189296+0000","last_active":"2024-09-17T07:31:39.145330+0000","last_peered":"2024-09-17T07:31:39.145330+0000","last_clean":"2024-09-17T07:31:39.145330+0000","last_became_active":"2024-09-17T07:30:18.083918+0000","last_became_peered":"2024-09-17T07:30:18.083918+0000","last_unstale":"2024-09-17T07:31:39.145330+0000","last_undegraded":"2024-09-17T07:31:39.145330+0000","last_fullsized":"2024-09-17T07:31:39.145330+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:26.983426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000820972,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4352,"num_read_kb":37354,"num_write":400,"num_write_kb":43089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":25536146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'763","reported_seq":4669,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214008+0000","last_change":"2024-09-17T07:31:14.113881+0000","last_active":"2024-09-17T07:31:38.214008+0000","last_peered":"2024-09-17T07:31:38.214008+0000","last_clean":"2024-09-17T07:31:38.214008+0000","last_became_active":"2024-09-17T07:30:43.091692+0000","last_became_peered":"2024-09-17T07:30:43.091692+0000","last_unstale":"2024-09-17T07:31:38.214008+0000","last_undegraded":"2024-09-17T07:31:38.214008+0000","last_fullsized":"2024-09-17T07:31:38.214008+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:24.550163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014749730000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5969,"num_read_kb":25327,"num_write":792,"num_write_kb":16603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":29456184,"num_keys_recovered":176,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"163'436","reported_seq":4226,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208396+0000","last_change":"2024-09-17T07:31:14.106118+0000","last_active":"2024-09-17T07:31:37.208396+0000","last_peered":"2024-09-17T07:31:37.208396+0000","last_clean":"2024-09-17T07:31:37.208396+0000","last_became_active":"2024-09-17T07:28:43.080428+0000","last_became_peered":"2024-09-17T07:28:43.080428+0000","last_unstale":"2024-09-17T07:31:37.208396+0000","last_undegraded":"2024-09-17T07:31:37.208396+0000","last_fullsized":"2024-09-17T07:31:37.208396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:12:14.112234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020783120000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4961,"num_read_kb":22717,"num_write":365,"num_write_kb":13531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"164'513","reported_seq":4191,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214897+0000","last_change":"2024-09-17T07:31:14.106949+0000","last_active":"2024-09-17T07:31:38.214897+0000","last_peered":"2024-09-17T07:31:38.214897+0000","last_clean":"2024-09-17T07:31:38.214897+0000","last_became_active":"2024-09-17T07:28:43.082313+0000","last_became_peered":"2024-09-17T07:28:43.082313+0000","last_unstale":"2024-09-17T07:31:38.214897+0000","last_undegraded":"2024-09-17T07:31:38.214897+0000","last_fullsized":"2024-09-17T07:31:38.214897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:35.352537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045261,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6140,"num_read_kb":58359,"num_write":449,"num_write_kb":31855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"164'390","reported_seq":4504,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372480+0000","last_change":"2024-09-17T07:31:14.175336+0000","last_active":"2024-09-17T07:31:37.372480+0000","last_peered":"2024-09-17T07:31:37.372480+0000","last_clean":"2024-09-17T07:31:37.372480+0000","last_became_active":"2024-09-17T07:31:08.094940+0000","last_became_peered":"2024-09-17T07:31:08.094940+0000","last_unstale":"2024-09-17T07:31:37.372480+0000","last_undegraded":"2024-09-17T07:31:37.372480+0000","last_fullsized":"2024-09-17T07:31:37.372480+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:58.292675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011121970000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3586,"num_read_kb":32352,"num_write":371,"num_write_kb":22781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":431,"num_bytes_recovered":60567103,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"163'465","reported_seq":2104,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372357+0000","last_change":"2024-09-17T07:31:26.153704+0000","last_active":"2024-09-17T07:31:37.372357+0000","last_peered":"2024-09-17T07:31:37.372357+0000","last_clean":"2024-09-17T07:31:37.372357+0000","last_became_active":"2024-09-17T07:31:26.152423+0000","last_became_peered":"2024-09-17T07:31:26.152423+0000","last_unstale":"2024-09-17T07:31:37.372357+0000","last_undegraded":"2024-09-17T07:31:37.372357+0000","last_fullsized":"2024-09-17T07:31:37.372357+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:46.055274+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015069525,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1414,"num_read_kb":17974,"num_write":482,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":16518806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":363,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216133+0000","last_change":"2024-09-17T07:31:05.098106+0000","last_active":"2024-09-17T07:31:38.216133+0000","last_peered":"2024-09-17T07:31:38.216133+0000","last_clean":"2024-09-17T07:31:38.216133+0000","last_became_active":"2024-09-17T07:31:05.097821+0000","last_became_peered":"2024-09-17T07:31:05.097821+0000","last_unstale":"2024-09-17T07:31:38.216133+0000","last_undegraded":"2024-09-17T07:31:38.216133+0000","last_fullsized":"2024-09-17T07:31:38.216133+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:12.297890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"165'1030","reported_seq":2761,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208278+0000","last_change":"2024-09-17T07:31:14.105440+0000","last_active":"2024-09-17T07:31:37.208278+0000","last_peered":"2024-09-17T07:31:37.208278+0000","last_clean":"2024-09-17T07:31:37.208278+0000","last_became_active":"2024-09-17T07:28:43.079229+0000","last_became_peered":"2024-09-17T07:28:43.079229+0000","last_unstale":"2024-09-17T07:31:37.208278+0000","last_undegraded":"2024-09-17T07:31:37.208278+0000","last_fullsized":"2024-09-17T07:31:37.208278+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1030,"log_dups_size":0,"ondisk_log_size":1030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:32.835400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015639079999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4625,"num_read_kb":48744,"num_write":1479,"num_write_kb":29297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":83,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145446+0000","last_change":"2024-09-17T07:31:01.268641+0000","last_active":"2024-09-17T07:31:39.145446+0000","last_peered":"2024-09-17T07:31:39.145446+0000","last_clean":"2024-09-17T07:31:39.145446+0000","last_became_active":"2024-09-17T07:31:01.268257+0000","last_became_peered":"2024-09-17T07:31:01.268257+0000","last_unstale":"2024-09-17T07:31:39.145446+0000","last_undegraded":"2024-09-17T07:31:39.145446+0000","last_fullsized":"2024-09-17T07:31:39.145446+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:04.188817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"164'577","reported_seq":3701,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214250+0000","last_change":"2024-09-17T07:31:14.115776+0000","last_active":"2024-09-17T07:31:38.214250+0000","last_peered":"2024-09-17T07:31:38.214250+0000","last_clean":"2024-09-17T07:31:38.214250+0000","last_became_active":"2024-09-17T07:30:19.087787+0000","last_became_peered":"2024-09-17T07:30:19.087787+0000","last_unstale":"2024-09-17T07:31:38.214250+0000","last_undegraded":"2024-09-17T07:31:38.214250+0000","last_fullsized":"2024-09-17T07:31:38.214250+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:26:55.414577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029713869999999998,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":3995,"num_read_kb":36995,"num_write":614,"num_write_kb":22622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":15169480,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":350,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372624+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:31:37.372624+0000","last_peered":"2024-09-17T07:31:37.372624+0000","last_clean":"2024-09-17T07:31:37.372624+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:31:37.372624+0000","last_undegraded":"2024-09-17T07:31:37.372624+0000","last_fullsized":"2024-09-17T07:31:37.372624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"163'1063","reported_seq":3603,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145306+0000","last_change":"2024-09-17T07:31:14.189299+0000","last_active":"2024-09-17T07:31:39.145306+0000","last_peered":"2024-09-17T07:31:39.145306+0000","last_clean":"2024-09-17T07:31:39.145306+0000","last_became_active":"2024-09-17T07:28:43.239387+0000","last_became_peered":"2024-09-17T07:28:43.239387+0000","last_unstale":"2024-09-17T07:31:39.145306+0000","last_undegraded":"2024-09-17T07:31:39.145306+0000","last_fullsized":"2024-09-17T07:31:39.145306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1063,"log_dups_size":0,"ondisk_log_size":1063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:09.407494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080983699999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6807,"num_read_kb":16873,"num_write":1444,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214910+0000","last_change":"2024-09-17T07:30:15.200774+0000","last_active":"2024-09-17T07:31:38.214910+0000","last_peered":"2024-09-17T07:31:38.214910+0000","last_clean":"2024-09-17T07:31:38.214910+0000","last_became_active":"2024-09-17T07:30:15.200470+0000","last_became_peered":"2024-09-17T07:30:15.200470+0000","last_unstale":"2024-09-17T07:31:38.214910+0000","last_undegraded":"2024-09-17T07:31:38.214910+0000","last_fullsized":"2024-09-17T07:31:38.214910+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:58.984439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"164'569","reported_seq":3303,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215021+0000","last_change":"2024-09-17T07:31:14.107519+0000","last_active":"2024-09-17T07:31:38.215021+0000","last_peered":"2024-09-17T07:31:38.215021+0000","last_clean":"2024-09-17T07:31:38.215021+0000","last_became_active":"2024-09-17T07:30:19.085184+0000","last_became_peered":"2024-09-17T07:30:19.085184+0000","last_unstale":"2024-09-17T07:31:38.215021+0000","last_undegraded":"2024-09-17T07:31:38.215021+0000","last_fullsized":"2024-09-17T07:31:38.215021+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:37.515132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015649349999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2687,"num_read_kb":46297,"num_write":632,"num_write_kb":27531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":35691417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"165'668","reported_seq":3157,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372462+0000","last_change":"2024-09-17T07:31:27.102404+0000","last_active":"2024-09-17T07:31:37.372462+0000","last_peered":"2024-09-17T07:31:37.372462+0000","last_clean":"2024-09-17T07:31:37.372462+0000","last_became_active":"2024-09-17T07:31:27.101204+0000","last_became_peered":"2024-09-17T07:31:27.101204+0000","last_unstale":"2024-09-17T07:31:37.372462+0000","last_undegraded":"2024-09-17T07:31:37.372462+0000","last_fullsized":"2024-09-17T07:31:37.372462+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:55.417631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028913630000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":2237,"num_read_kb":35376,"num_write":632,"num_write_kb":16684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":26727758,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":355,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214818+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:31:38.214818+0000","last_peered":"2024-09-17T07:31:38.214818+0000","last_clean":"2024-09-17T07:31:38.214818+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:31:38.214818+0000","last_undegraded":"2024-09-17T07:31:38.214818+0000","last_fullsized":"2024-09-17T07:31:38.214818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"164'825","reported_seq":11345,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145432+0000","last_change":"2024-09-17T07:31:14.189768+0000","last_active":"2024-09-17T07:31:39.145432+0000","last_peered":"2024-09-17T07:31:39.145432+0000","last_clean":"2024-09-17T07:31:39.145432+0000","last_became_active":"2024-09-17T07:28:43.239409+0000","last_became_peered":"2024-09-17T07:28:43.239409+0000","last_unstale":"2024-09-17T07:31:39.145432+0000","last_undegraded":"2024-09-17T07:31:39.145432+0000","last_fullsized":"2024-09-17T07:31:39.145432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":825,"log_dups_size":0,"ondisk_log_size":825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:13.253211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012368310000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":12617,"num_read_kb":91706,"num_write":1240,"num_write_kb":58366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":365,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.380263+0000","last_change":"2024-09-17T07:30:15.201999+0000","last_active":"2024-09-17T07:31:38.380263+0000","last_peered":"2024-09-17T07:31:38.380263+0000","last_clean":"2024-09-17T07:31:38.380263+0000","last_became_active":"2024-09-17T07:30:15.201642+0000","last_became_peered":"2024-09-17T07:30:15.201642+0000","last_unstale":"2024-09-17T07:31:38.380263+0000","last_undegraded":"2024-09-17T07:31:38.380263+0000","last_fullsized":"2024-09-17T07:31:38.380263+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:47.871565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"164'427","reported_seq":5376,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208231+0000","last_change":"2024-09-17T07:31:29.105514+0000","last_active":"2024-09-17T07:31:37.208231+0000","last_peered":"2024-09-17T07:31:37.208231+0000","last_clean":"2024-09-17T07:31:37.208231+0000","last_became_active":"2024-09-17T07:31:29.103782+0000","last_became_peered":"2024-09-17T07:31:29.103782+0000","last_unstale":"2024-09-17T07:31:37.208231+0000","last_undegraded":"2024-09-17T07:31:37.208231+0000","last_fullsized":"2024-09-17T07:31:37.208231+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:52.018922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094238099999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3788,"num_read_kb":51682,"num_write":424,"num_write_kb":28012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":37615005,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":362,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372172+0000","last_change":"2024-09-17T07:31:01.231085+0000","last_active":"2024-09-17T07:31:37.372172+0000","last_peered":"2024-09-17T07:31:37.372172+0000","last_clean":"2024-09-17T07:31:37.372172+0000","last_became_active":"2024-09-17T07:31:01.230794+0000","last_became_peered":"2024-09-17T07:31:01.230794+0000","last_unstale":"2024-09-17T07:31:37.372172+0000","last_undegraded":"2024-09-17T07:31:37.372172+0000","last_fullsized":"2024-09-17T07:31:37.372172+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:25.321726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"165'405","reported_seq":2497,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214012+0000","last_change":"2024-09-17T07:31:14.114890+0000","last_active":"2024-09-17T07:31:38.214012+0000","last_peered":"2024-09-17T07:31:38.214012+0000","last_clean":"2024-09-17T07:31:38.214012+0000","last_became_active":"2024-09-17T07:30:18.084247+0000","last_became_peered":"2024-09-17T07:30:18.084247+0000","last_unstale":"2024-09-17T07:31:38.214012+0000","last_undegraded":"2024-09-17T07:31:38.214012+0000","last_fullsized":"2024-09-17T07:31:38.214012+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:54:33.014267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002225922,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2958,"num_read_kb":9261,"num_write":364,"num_write_kb":6317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":6805602,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":138,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208474+0000","last_change":"2024-09-17T07:30:34.116563+0000","last_active":"2024-09-17T07:31:37.208474+0000","last_peered":"2024-09-17T07:31:37.208474+0000","last_clean":"2024-09-17T07:31:37.208474+0000","last_became_active":"2024-09-17T07:30:34.113598+0000","last_became_peered":"2024-09-17T07:30:34.113598+0000","last_unstale":"2024-09-17T07:31:37.208474+0000","last_undegraded":"2024-09-17T07:31:37.208474+0000","last_fullsized":"2024-09-17T07:31:37.208474+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:52.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":79,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372168+0000","last_change":"2024-09-17T07:31:05.102195+0000","last_active":"2024-09-17T07:31:37.372168+0000","last_peered":"2024-09-17T07:31:37.372168+0000","last_clean":"2024-09-17T07:31:37.372168+0000","last_became_active":"2024-09-17T07:31:05.096988+0000","last_became_peered":"2024-09-17T07:31:05.096988+0000","last_unstale":"2024-09-17T07:31:37.372168+0000","last_undegraded":"2024-09-17T07:31:37.372168+0000","last_fullsized":"2024-09-17T07:31:37.372168+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:01.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"164'520","reported_seq":2435,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214008+0000","last_change":"2024-09-17T07:31:14.113864+0000","last_active":"2024-09-17T07:31:38.214008+0000","last_peered":"2024-09-17T07:31:38.214008+0000","last_clean":"2024-09-17T07:31:38.214008+0000","last_became_active":"2024-09-17T07:30:19.086777+0000","last_became_peered":"2024-09-17T07:30:19.086777+0000","last_unstale":"2024-09-17T07:31:38.214008+0000","last_undegraded":"2024-09-17T07:31:38.214008+0000","last_fullsized":"2024-09-17T07:31:38.214008+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:55.561658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014795710000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1640,"num_read_kb":24382,"num_write":527,"num_write_kb":33950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":24920122,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"164'551","reported_seq":3917,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372211+0000","last_change":"2024-09-17T07:31:27.105903+0000","last_active":"2024-09-17T07:31:37.372211+0000","last_peered":"2024-09-17T07:31:37.372211+0000","last_clean":"2024-09-17T07:31:37.372211+0000","last_became_active":"2024-09-17T07:31:27.103978+0000","last_became_peered":"2024-09-17T07:31:27.103978+0000","last_unstale":"2024-09-17T07:31:37.372211+0000","last_undegraded":"2024-09-17T07:31:37.372211+0000","last_fullsized":"2024-09-17T07:31:37.372211+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:13.427259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001973519,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4515,"num_read_kb":21074,"num_write":530,"num_write_kb":12196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":11434290,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"164'739","reported_seq":9612,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216065+0000","last_change":"2024-09-17T07:31:28.103185+0000","last_active":"2024-09-17T07:31:38.216065+0000","last_peered":"2024-09-17T07:31:38.216065+0000","last_clean":"2024-09-17T07:31:38.216065+0000","last_became_active":"2024-09-17T07:31:28.101926+0000","last_became_peered":"2024-09-17T07:31:28.101926+0000","last_unstale":"2024-09-17T07:31:38.216065+0000","last_undegraded":"2024-09-17T07:31:38.216065+0000","last_fullsized":"2024-09-17T07:31:38.216065+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:17:26.947693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000464964,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":26,"num_read":9368,"num_read_kb":52923,"num_write":688,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":604,"num_bytes_recovered":24254406,"num_keys_recovered":93,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"163'559","reported_seq":2911,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214086+0000","last_change":"2024-09-17T07:31:14.114917+0000","last_active":"2024-09-17T07:31:38.214086+0000","last_peered":"2024-09-17T07:31:38.214086+0000","last_clean":"2024-09-17T07:31:38.214086+0000","last_became_active":"2024-09-17T07:30:19.087515+0000","last_became_peered":"2024-09-17T07:30:19.087515+0000","last_unstale":"2024-09-17T07:31:38.214086+0000","last_undegraded":"2024-09-17T07:31:38.214086+0000","last_fullsized":"2024-09-17T07:31:38.214086+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:42.469193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023177290000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2408,"num_read_kb":29904,"num_write":665,"num_write_kb":15633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":11178824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"165'873","reported_seq":5432,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215146+0000","last_change":"2024-09-17T07:31:14.107447+0000","last_active":"2024-09-17T07:31:38.215146+0000","last_peered":"2024-09-17T07:31:38.215146+0000","last_clean":"2024-09-17T07:31:38.215146+0000","last_became_active":"2024-09-17T07:30:19.085668+0000","last_became_peered":"2024-09-17T07:30:19.085668+0000","last_unstale":"2024-09-17T07:31:38.215146+0000","last_undegraded":"2024-09-17T07:31:38.215146+0000","last_fullsized":"2024-09-17T07:31:38.215146+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:47.391518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001368193,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6070,"num_read_kb":27721,"num_write":1113,"num_write_kb":14827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":15619889,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"165'725","reported_seq":4612,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372484+0000","last_change":"2024-09-17T07:31:14.176026+0000","last_active":"2024-09-17T07:31:37.372484+0000","last_peered":"2024-09-17T07:31:37.372484+0000","last_clean":"2024-09-17T07:31:37.372484+0000","last_became_active":"2024-09-17T07:28:43.080816+0000","last_became_peered":"2024-09-17T07:28:43.080816+0000","last_unstale":"2024-09-17T07:31:37.372484+0000","last_undegraded":"2024-09-17T07:31:37.372484+0000","last_fullsized":"2024-09-17T07:31:37.372484+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:05:06.879033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017783269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6005,"num_read_kb":48957,"num_write":832,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"164'458","reported_seq":8425,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372431+0000","last_change":"2024-09-17T07:31:26.155667+0000","last_active":"2024-09-17T07:31:37.372431+0000","last_peered":"2024-09-17T07:31:37.372431+0000","last_clean":"2024-09-17T07:31:37.372431+0000","last_became_active":"2024-09-17T07:31:26.154754+0000","last_became_peered":"2024-09-17T07:31:26.154754+0000","last_unstale":"2024-09-17T07:31:37.372431+0000","last_undegraded":"2024-09-17T07:31:37.372431+0000","last_fullsized":"2024-09-17T07:31:37.372431+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:47.550445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021346680000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7087,"num_read_kb":55690,"num_write":392,"num_write_kb":41376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":13797349,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"163'832","reported_seq":6338,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372621+0000","last_change":"2024-09-17T07:31:28.103546+0000","last_active":"2024-09-17T07:31:37.372621+0000","last_peered":"2024-09-17T07:31:37.372621+0000","last_clean":"2024-09-17T07:31:37.372621+0000","last_became_active":"2024-09-17T07:31:28.101799+0000","last_became_peered":"2024-09-17T07:31:28.101799+0000","last_unstale":"2024-09-17T07:31:37.372621+0000","last_undegraded":"2024-09-17T07:31:37.372621+0000","last_fullsized":"2024-09-17T07:31:37.372621+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:28.873325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021103049999999998,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6418,"num_read_kb":23171,"num_write":923,"num_write_kb":18541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":744,"num_bytes_recovered":25525458,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"164'401","reported_seq":2768,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216051+0000","last_change":"2024-09-17T07:31:14.720102+0000","last_active":"2024-09-17T07:31:38.216051+0000","last_peered":"2024-09-17T07:31:38.216051+0000","last_clean":"2024-09-17T07:31:38.216051+0000","last_became_active":"2024-09-17T07:28:43.077360+0000","last_became_peered":"2024-09-17T07:28:43.077360+0000","last_unstale":"2024-09-17T07:31:38.216051+0000","last_undegraded":"2024-09-17T07:31:38.216051+0000","last_fullsized":"2024-09-17T07:31:38.216051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:17.256040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061413699999999995,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2841,"num_read_kb":26198,"num_write":450,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"164'802","reported_seq":4337,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372196+0000","last_change":"2024-09-17T07:31:29.105135+0000","last_active":"2024-09-17T07:31:37.372196+0000","last_peered":"2024-09-17T07:31:37.372196+0000","last_clean":"2024-09-17T07:31:37.372196+0000","last_became_active":"2024-09-17T07:31:29.103626+0000","last_became_peered":"2024-09-17T07:31:29.103626+0000","last_unstale":"2024-09-17T07:31:37.372196+0000","last_undegraded":"2024-09-17T07:31:37.372196+0000","last_fullsized":"2024-09-17T07:31:37.372196+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:10.689986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001604722,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5310,"num_read_kb":45214,"num_write":937,"num_write_kb":26421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":468,"num_bytes_recovered":31676674,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"164'435","reported_seq":2745,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208126+0000","last_change":"2024-09-17T07:31:14.104829+0000","last_active":"2024-09-17T07:31:37.208126+0000","last_peered":"2024-09-17T07:31:37.208126+0000","last_clean":"2024-09-17T07:31:37.208126+0000","last_became_active":"2024-09-17T07:31:08.096240+0000","last_became_peered":"2024-09-17T07:31:08.096240+0000","last_unstale":"2024-09-17T07:31:37.208126+0000","last_undegraded":"2024-09-17T07:31:37.208126+0000","last_fullsized":"2024-09-17T07:31:37.208126+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:34.448388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098890999999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2231,"num_read_kb":44562,"num_write":412,"num_write_kb":29148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":24460350,"num_keys_recovered":37,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"164'772","reported_seq":4240,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372356+0000","last_change":"2024-09-17T07:31:29.104804+0000","last_active":"2024-09-17T07:31:37.372356+0000","last_peered":"2024-09-17T07:31:37.372356+0000","last_clean":"2024-09-17T07:31:37.372356+0000","last_became_active":"2024-09-17T07:31:29.103168+0000","last_became_peered":"2024-09-17T07:31:29.103168+0000","last_unstale":"2024-09-17T07:31:37.372356+0000","last_undegraded":"2024-09-17T07:31:37.372356+0000","last_fullsized":"2024-09-17T07:31:37.372356+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:31.292655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093421499999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4511,"num_read_kb":28124,"num_write":1074,"num_write_kb":16615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":21539718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":607,"num_read":166048,"num_read_kb":1208202,"num_write":23179,"num_write_kb":792760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9827,"num_bytes_recovered":666374238,"num_keys_recovered":891,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7902,"internal_metadata":0},"log_size":20553,"ondisk_log_size":20553,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":101032,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542186,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47324,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324516,"statfs":{"total":96636764160,"available":96588304384,"internally_reserved":0,"allocated":2301952,"data_stored":1679580,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33412,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338428,"statfs":{"total":96636764160,"available":96602550272,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55616,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316224,"statfs":{"total":96636764160,"available":96579813376,"internally_reserved":0,"allocated":2269184,"data_stored":1095102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542186,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91640,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280200,"statfs":{"total":96636764160,"available":96542924800,"internally_reserved":0,"allocated":2256896,"data_stored":1089231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056662,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43268,"kb_used_data":2212,"kb_used_omap":16,"kb_used_meta":41007,"kb_avail":94328572,"statfs":{"total":96636764160,"available":96592457728,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17058,"internal_metadata":41991518},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542186,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55612,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316228,"statfs":{"total":96636764160,"available":96579817472,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65336,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306504,"statfs":{"total":96636764160,"available":96569860096,"internally_reserved":0,"allocated":2260992,"data_stored":1089250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47340,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324500,"statfs":{"total":96636764160,"available":96588288000,"internally_reserved":0,"allocated":2318336,"data_stored":1679580,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7902,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:31:42.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:31:43.094 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:31:43.106 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-09-17T07:31:42.788723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":607,"num_read":166114,"num_read_kb":1208257,"num_write":23282,"num_write_kb":794390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9832,"num_bytes_recovered":667554974,"num_keys_recovered":891,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20628,"ondisk_log_size":20628,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439548,"kb_used_data":17776,"kb_used_omap":27,"kb_used_meta":421540,"kb_avail":754535172,"statfs":{"total":773094113280,"available":772644016128,"internally_reserved":0,"allocated":18202624,"data_stored":9917992,"data_compressed":66815,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":28217,"internal_metadata":431657415},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.592133"},"pg_stats":[{"pgid":"3.1f","version":"165'874","reported_seq":4617,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208112+0000","last_change":"2024-09-17T07:31:14.105507+0000","last_active":"2024-09-17T07:31:37.208112+0000","last_peered":"2024-09-17T07:31:37.208112+0000","last_clean":"2024-09-17T07:31:37.208112+0000","last_became_active":"2024-09-17T07:30:43.092456+0000","last_became_peered":"2024-09-17T07:30:43.092456+0000","last_unstale":"2024-09-17T07:31:37.208112+0000","last_undegraded":"2024-09-17T07:31:37.208112+0000","last_fullsized":"2024-09-17T07:31:37.208112+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:26.811072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016105900000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5801,"num_read_kb":34310,"num_write":997,"num_write_kb":41977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":26159662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"164'553","reported_seq":3285,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214114+0000","last_change":"2024-09-17T07:31:14.115590+0000","last_active":"2024-09-17T07:31:38.214114+0000","last_peered":"2024-09-17T07:31:38.214114+0000","last_clean":"2024-09-17T07:31:38.214114+0000","last_became_active":"2024-09-17T07:30:19.087213+0000","last_became_peered":"2024-09-17T07:30:19.087213+0000","last_unstale":"2024-09-17T07:31:38.214114+0000","last_undegraded":"2024-09-17T07:31:38.214114+0000","last_fullsized":"2024-09-17T07:31:38.214114+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:15.610729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029487620000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3721,"num_read_kb":30298,"num_write":527,"num_write_kb":15594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":15743718,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"163'550","reported_seq":3498,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214875+0000","last_change":"2024-09-17T07:31:26.150683+0000","last_active":"2024-09-17T07:31:38.214875+0000","last_peered":"2024-09-17T07:31:38.214875+0000","last_clean":"2024-09-17T07:31:38.214875+0000","last_became_active":"2024-09-17T07:31:26.149428+0000","last_became_peered":"2024-09-17T07:31:26.149428+0000","last_unstale":"2024-09-17T07:31:38.214875+0000","last_undegraded":"2024-09-17T07:31:38.214875+0000","last_fullsized":"2024-09-17T07:31:38.214875+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:58.834946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016933180000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":1948,"num_read_kb":28759,"num_write":579,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":27967854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"165'813","reported_seq":6628,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215109+0000","last_change":"2024-09-17T07:31:28.101214+0000","last_active":"2024-09-17T07:31:38.215109+0000","last_peered":"2024-09-17T07:31:38.215109+0000","last_clean":"2024-09-17T07:31:38.215109+0000","last_became_active":"2024-09-17T07:31:28.100048+0000","last_became_peered":"2024-09-17T07:31:28.100048+0000","last_unstale":"2024-09-17T07:31:38.215109+0000","last_undegraded":"2024-09-17T07:31:38.215109+0000","last_fullsized":"2024-09-17T07:31:38.215109+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:35.020631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010655949999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":7523,"num_read_kb":42454,"num_write":1061,"num_write_kb":38721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":26661650,"num_keys_recovered":180,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"165'606","reported_seq":5639,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208349+0000","last_change":"2024-09-17T07:31:28.104670+0000","last_active":"2024-09-17T07:31:37.208349+0000","last_peered":"2024-09-17T07:31:37.208349+0000","last_clean":"2024-09-17T07:31:37.208349+0000","last_became_active":"2024-09-17T07:31:28.102058+0000","last_became_peered":"2024-09-17T07:31:28.102058+0000","last_unstale":"2024-09-17T07:31:37.208349+0000","last_undegraded":"2024-09-17T07:31:37.208349+0000","last_fullsized":"2024-09-17T07:31:37.208349+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:07.797309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028238529999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5800,"num_read_kb":52463,"num_write":596,"num_write_kb":22879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":91784745,"num_keys_recovered":126,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"163'920","reported_seq":14894,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372319+0000","last_change":"2024-09-17T07:31:27.105439+0000","last_active":"2024-09-17T07:31:37.372319+0000","last_peered":"2024-09-17T07:31:37.372319+0000","last_clean":"2024-09-17T07:31:37.372319+0000","last_became_active":"2024-09-17T07:31:27.103933+0000","last_became_peered":"2024-09-17T07:31:27.103933+0000","last_unstale":"2024-09-17T07:31:37.372319+0000","last_undegraded":"2024-09-17T07:31:37.372319+0000","last_fullsized":"2024-09-17T07:31:37.372319+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:06:20.908980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096620079999999997,"stat_sum":{"num_bytes":19,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":16715,"num_read_kb":60981,"num_write":1188,"num_write_kb":20597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":19562028,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"164'439","reported_seq":5685,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145330+0000","last_change":"2024-09-17T07:31:14.189296+0000","last_active":"2024-09-17T07:31:39.145330+0000","last_peered":"2024-09-17T07:31:39.145330+0000","last_clean":"2024-09-17T07:31:39.145330+0000","last_became_active":"2024-09-17T07:30:18.083918+0000","last_became_peered":"2024-09-17T07:30:18.083918+0000","last_unstale":"2024-09-17T07:31:39.145330+0000","last_undegraded":"2024-09-17T07:31:39.145330+0000","last_fullsized":"2024-09-17T07:31:39.145330+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:26.983426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000820972,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4352,"num_read_kb":37354,"num_write":400,"num_write_kb":43089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":25536146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'763","reported_seq":4669,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214008+0000","last_change":"2024-09-17T07:31:14.113881+0000","last_active":"2024-09-17T07:31:38.214008+0000","last_peered":"2024-09-17T07:31:38.214008+0000","last_clean":"2024-09-17T07:31:38.214008+0000","last_became_active":"2024-09-17T07:30:43.091692+0000","last_became_peered":"2024-09-17T07:30:43.091692+0000","last_unstale":"2024-09-17T07:31:38.214008+0000","last_undegraded":"2024-09-17T07:31:38.214008+0000","last_fullsized":"2024-09-17T07:31:38.214008+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:24.550163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014749730000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5969,"num_read_kb":25327,"num_write":792,"num_write_kb":16603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":29456184,"num_keys_recovered":176,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"163'436","reported_seq":4226,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208396+0000","last_change":"2024-09-17T07:31:14.106118+0000","last_active":"2024-09-17T07:31:37.208396+0000","last_peered":"2024-09-17T07:31:37.208396+0000","last_clean":"2024-09-17T07:31:37.208396+0000","last_became_active":"2024-09-17T07:28:43.080428+0000","last_became_peered":"2024-09-17T07:28:43.080428+0000","last_unstale":"2024-09-17T07:31:37.208396+0000","last_undegraded":"2024-09-17T07:31:37.208396+0000","last_fullsized":"2024-09-17T07:31:37.208396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:12:14.112234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020783120000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4961,"num_read_kb":22717,"num_write":365,"num_write_kb":13531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"164'513","reported_seq":4191,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214897+0000","last_change":"2024-09-17T07:31:14.106949+0000","last_active":"2024-09-17T07:31:38.214897+0000","last_peered":"2024-09-17T07:31:38.214897+0000","last_clean":"2024-09-17T07:31:38.214897+0000","last_became_active":"2024-09-17T07:28:43.082313+0000","last_became_peered":"2024-09-17T07:28:43.082313+0000","last_unstale":"2024-09-17T07:31:38.214897+0000","last_undegraded":"2024-09-17T07:31:38.214897+0000","last_fullsized":"2024-09-17T07:31:38.214897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:35.352537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045261,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6140,"num_read_kb":58359,"num_write":449,"num_write_kb":31855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"164'390","reported_seq":4504,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372480+0000","last_change":"2024-09-17T07:31:14.175336+0000","last_active":"2024-09-17T07:31:37.372480+0000","last_peered":"2024-09-17T07:31:37.372480+0000","last_clean":"2024-09-17T07:31:37.372480+0000","last_became_active":"2024-09-17T07:31:08.094940+0000","last_became_peered":"2024-09-17T07:31:08.094940+0000","last_unstale":"2024-09-17T07:31:37.372480+0000","last_undegraded":"2024-09-17T07:31:37.372480+0000","last_fullsized":"2024-09-17T07:31:37.372480+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:58.292675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011121970000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3586,"num_read_kb":32352,"num_write":371,"num_write_kb":22781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":431,"num_bytes_recovered":60567103,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"163'465","reported_seq":2104,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372357+0000","last_change":"2024-09-17T07:31:26.153704+0000","last_active":"2024-09-17T07:31:37.372357+0000","last_peered":"2024-09-17T07:31:37.372357+0000","last_clean":"2024-09-17T07:31:37.372357+0000","last_became_active":"2024-09-17T07:31:26.152423+0000","last_became_peered":"2024-09-17T07:31:26.152423+0000","last_unstale":"2024-09-17T07:31:37.372357+0000","last_undegraded":"2024-09-17T07:31:37.372357+0000","last_fullsized":"2024-09-17T07:31:37.372357+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:46.055274+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015069525,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1414,"num_read_kb":17974,"num_write":482,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":16518806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":363,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216133+0000","last_change":"2024-09-17T07:31:05.098106+0000","last_active":"2024-09-17T07:31:38.216133+0000","last_peered":"2024-09-17T07:31:38.216133+0000","last_clean":"2024-09-17T07:31:38.216133+0000","last_became_active":"2024-09-17T07:31:05.097821+0000","last_became_peered":"2024-09-17T07:31:05.097821+0000","last_unstale":"2024-09-17T07:31:38.216133+0000","last_undegraded":"2024-09-17T07:31:38.216133+0000","last_fullsized":"2024-09-17T07:31:38.216133+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:12.297890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"165'1030","reported_seq":2761,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208278+0000","last_change":"2024-09-17T07:31:14.105440+0000","last_active":"2024-09-17T07:31:37.208278+0000","last_peered":"2024-09-17T07:31:37.208278+0000","last_clean":"2024-09-17T07:31:37.208278+0000","last_became_active":"2024-09-17T07:28:43.079229+0000","last_became_peered":"2024-09-17T07:28:43.079229+0000","last_unstale":"2024-09-17T07:31:37.208278+0000","last_undegraded":"2024-09-17T07:31:37.208278+0000","last_fullsized":"2024-09-17T07:31:37.208278+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1030,"log_dups_size":0,"ondisk_log_size":1030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:32.835400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015639079999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4625,"num_read_kb":48744,"num_write":1479,"num_write_kb":29297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":83,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145446+0000","last_change":"2024-09-17T07:31:01.268641+0000","last_active":"2024-09-17T07:31:39.145446+0000","last_peered":"2024-09-17T07:31:39.145446+0000","last_clean":"2024-09-17T07:31:39.145446+0000","last_became_active":"2024-09-17T07:31:01.268257+0000","last_became_peered":"2024-09-17T07:31:01.268257+0000","last_unstale":"2024-09-17T07:31:39.145446+0000","last_undegraded":"2024-09-17T07:31:39.145446+0000","last_fullsized":"2024-09-17T07:31:39.145446+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:04.188817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"164'577","reported_seq":3701,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214250+0000","last_change":"2024-09-17T07:31:14.115776+0000","last_active":"2024-09-17T07:31:38.214250+0000","last_peered":"2024-09-17T07:31:38.214250+0000","last_clean":"2024-09-17T07:31:38.214250+0000","last_became_active":"2024-09-17T07:30:19.087787+0000","last_became_peered":"2024-09-17T07:30:19.087787+0000","last_unstale":"2024-09-17T07:31:38.214250+0000","last_undegraded":"2024-09-17T07:31:38.214250+0000","last_fullsized":"2024-09-17T07:31:38.214250+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:26:55.414577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029713869999999998,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":3995,"num_read_kb":36995,"num_write":614,"num_write_kb":22622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":15169480,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":350,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372624+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:31:37.372624+0000","last_peered":"2024-09-17T07:31:37.372624+0000","last_clean":"2024-09-17T07:31:37.372624+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:31:37.372624+0000","last_undegraded":"2024-09-17T07:31:37.372624+0000","last_fullsized":"2024-09-17T07:31:37.372624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"163'1063","reported_seq":3603,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145306+0000","last_change":"2024-09-17T07:31:14.189299+0000","last_active":"2024-09-17T07:31:39.145306+0000","last_peered":"2024-09-17T07:31:39.145306+0000","last_clean":"2024-09-17T07:31:39.145306+0000","last_became_active":"2024-09-17T07:28:43.239387+0000","last_became_peered":"2024-09-17T07:28:43.239387+0000","last_unstale":"2024-09-17T07:31:39.145306+0000","last_undegraded":"2024-09-17T07:31:39.145306+0000","last_fullsized":"2024-09-17T07:31:39.145306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1063,"log_dups_size":0,"ondisk_log_size":1063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:09.407494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080983699999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6807,"num_read_kb":16873,"num_write":1444,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214910+0000","last_change":"2024-09-17T07:30:15.200774+0000","last_active":"2024-09-17T07:31:38.214910+0000","last_peered":"2024-09-17T07:31:38.214910+0000","last_clean":"2024-09-17T07:31:38.214910+0000","last_became_active":"2024-09-17T07:30:15.200470+0000","last_became_peered":"2024-09-17T07:30:15.200470+0000","last_unstale":"2024-09-17T07:31:38.214910+0000","last_undegraded":"2024-09-17T07:31:38.214910+0000","last_fullsized":"2024-09-17T07:31:38.214910+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:58.984439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"164'569","reported_seq":3303,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215021+0000","last_change":"2024-09-17T07:31:14.107519+0000","last_active":"2024-09-17T07:31:38.215021+0000","last_peered":"2024-09-17T07:31:38.215021+0000","last_clean":"2024-09-17T07:31:38.215021+0000","last_became_active":"2024-09-17T07:30:19.085184+0000","last_became_peered":"2024-09-17T07:30:19.085184+0000","last_unstale":"2024-09-17T07:31:38.215021+0000","last_undegraded":"2024-09-17T07:31:38.215021+0000","last_fullsized":"2024-09-17T07:31:38.215021+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:37.515132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015649349999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2687,"num_read_kb":46297,"num_write":632,"num_write_kb":27531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":35691417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"165'668","reported_seq":3157,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372462+0000","last_change":"2024-09-17T07:31:27.102404+0000","last_active":"2024-09-17T07:31:37.372462+0000","last_peered":"2024-09-17T07:31:37.372462+0000","last_clean":"2024-09-17T07:31:37.372462+0000","last_became_active":"2024-09-17T07:31:27.101204+0000","last_became_peered":"2024-09-17T07:31:27.101204+0000","last_unstale":"2024-09-17T07:31:37.372462+0000","last_undegraded":"2024-09-17T07:31:37.372462+0000","last_fullsized":"2024-09-17T07:31:37.372462+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:55.417631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028913630000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":2237,"num_read_kb":35376,"num_write":632,"num_write_kb":16684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":26727758,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":355,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214818+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:31:38.214818+0000","last_peered":"2024-09-17T07:31:38.214818+0000","last_clean":"2024-09-17T07:31:38.214818+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:31:38.214818+0000","last_undegraded":"2024-09-17T07:31:38.214818+0000","last_fullsized":"2024-09-17T07:31:38.214818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"164'825","reported_seq":11345,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145432+0000","last_change":"2024-09-17T07:31:14.189768+0000","last_active":"2024-09-17T07:31:39.145432+0000","last_peered":"2024-09-17T07:31:39.145432+0000","last_clean":"2024-09-17T07:31:39.145432+0000","last_became_active":"2024-09-17T07:28:43.239409+0000","last_became_peered":"2024-09-17T07:28:43.239409+0000","last_unstale":"2024-09-17T07:31:39.145432+0000","last_undegraded":"2024-09-17T07:31:39.145432+0000","last_fullsized":"2024-09-17T07:31:39.145432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":825,"log_dups_size":0,"ondisk_log_size":825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:13.253211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012368310000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":12617,"num_read_kb":91706,"num_write":1240,"num_write_kb":58366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":365,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.380263+0000","last_change":"2024-09-17T07:30:15.201999+0000","last_active":"2024-09-17T07:31:38.380263+0000","last_peered":"2024-09-17T07:31:38.380263+0000","last_clean":"2024-09-17T07:31:38.380263+0000","last_became_active":"2024-09-17T07:30:15.201642+0000","last_became_peered":"2024-09-17T07:30:15.201642+0000","last_unstale":"2024-09-17T07:31:38.380263+0000","last_undegraded":"2024-09-17T07:31:38.380263+0000","last_fullsized":"2024-09-17T07:31:38.380263+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:47.871565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"164'427","reported_seq":5376,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208231+0000","last_change":"2024-09-17T07:31:29.105514+0000","last_active":"2024-09-17T07:31:37.208231+0000","last_peered":"2024-09-17T07:31:37.208231+0000","last_clean":"2024-09-17T07:31:37.208231+0000","last_became_active":"2024-09-17T07:31:29.103782+0000","last_became_peered":"2024-09-17T07:31:29.103782+0000","last_unstale":"2024-09-17T07:31:37.208231+0000","last_undegraded":"2024-09-17T07:31:37.208231+0000","last_fullsized":"2024-09-17T07:31:37.208231+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:52.018922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094238099999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3788,"num_read_kb":51682,"num_write":424,"num_write_kb":28012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":37615005,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":362,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372172+0000","last_change":"2024-09-17T07:31:01.231085+0000","last_active":"2024-09-17T07:31:37.372172+0000","last_peered":"2024-09-17T07:31:37.372172+0000","last_clean":"2024-09-17T07:31:37.372172+0000","last_became_active":"2024-09-17T07:31:01.230794+0000","last_became_peered":"2024-09-17T07:31:01.230794+0000","last_unstale":"2024-09-17T07:31:37.372172+0000","last_undegraded":"2024-09-17T07:31:37.372172+0000","last_fullsized":"2024-09-17T07:31:37.372172+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:25.321726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"165'405","reported_seq":2497,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214012+0000","last_change":"2024-09-17T07:31:14.114890+0000","last_active":"2024-09-17T07:31:38.214012+0000","last_peered":"2024-09-17T07:31:38.214012+0000","last_clean":"2024-09-17T07:31:38.214012+0000","last_became_active":"2024-09-17T07:30:18.084247+0000","last_became_peered":"2024-09-17T07:30:18.084247+0000","last_unstale":"2024-09-17T07:31:38.214012+0000","last_undegraded":"2024-09-17T07:31:38.214012+0000","last_fullsized":"2024-09-17T07:31:38.214012+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:54:33.014267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002225922,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2958,"num_read_kb":9261,"num_write":364,"num_write_kb":6317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":6805602,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":138,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208474+0000","last_change":"2024-09-17T07:30:34.116563+0000","last_active":"2024-09-17T07:31:37.208474+0000","last_peered":"2024-09-17T07:31:37.208474+0000","last_clean":"2024-09-17T07:31:37.208474+0000","last_became_active":"2024-09-17T07:30:34.113598+0000","last_became_peered":"2024-09-17T07:30:34.113598+0000","last_unstale":"2024-09-17T07:31:37.208474+0000","last_undegraded":"2024-09-17T07:31:37.208474+0000","last_fullsized":"2024-09-17T07:31:37.208474+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:52.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":79,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372168+0000","last_change":"2024-09-17T07:31:05.102195+0000","last_active":"2024-09-17T07:31:37.372168+0000","last_peered":"2024-09-17T07:31:37.372168+0000","last_clean":"2024-09-17T07:31:37.372168+0000","last_became_active":"2024-09-17T07:31:05.096988+0000","last_became_peered":"2024-09-17T07:31:05.096988+0000","last_unstale":"2024-09-17T07:31:37.372168+0000","last_undegraded":"2024-09-17T07:31:37.372168+0000","last_fullsized":"2024-09-17T07:31:37.372168+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:01.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"164'520","reported_seq":2435,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214008+0000","last_change":"2024-09-17T07:31:14.113864+0000","last_active":"2024-09-17T07:31:38.214008+0000","last_peered":"2024-09-17T07:31:38.214008+0000","last_clean":"2024-09-17T07:31:38.214008+0000","last_became_active":"2024-09-17T07:30:19.086777+0000","last_became_peered":"2024-09-17T07:30:19.086777+0000","last_unstale":"2024-09-17T07:31:38.214008+0000","last_undegraded":"2024-09-17T07:31:38.214008+0000","last_fullsized":"2024-09-17T07:31:38.214008+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:55.561658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014795710000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1640,"num_read_kb":24382,"num_write":527,"num_write_kb":33950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":24920122,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"164'551","reported_seq":3917,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372211+0000","last_change":"2024-09-17T07:31:27.105903+0000","last_active":"2024-09-17T07:31:37.372211+0000","last_peered":"2024-09-17T07:31:37.372211+0000","last_clean":"2024-09-17T07:31:37.372211+0000","last_became_active":"2024-09-17T07:31:27.103978+0000","last_became_peered":"2024-09-17T07:31:27.103978+0000","last_unstale":"2024-09-17T07:31:37.372211+0000","last_undegraded":"2024-09-17T07:31:37.372211+0000","last_fullsized":"2024-09-17T07:31:37.372211+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:13.427259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001973519,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4515,"num_read_kb":21074,"num_write":530,"num_write_kb":12196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":11434290,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"164'739","reported_seq":9612,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216065+0000","last_change":"2024-09-17T07:31:28.103185+0000","last_active":"2024-09-17T07:31:38.216065+0000","last_peered":"2024-09-17T07:31:38.216065+0000","last_clean":"2024-09-17T07:31:38.216065+0000","last_became_active":"2024-09-17T07:31:28.101926+0000","last_became_peered":"2024-09-17T07:31:28.101926+0000","last_unstale":"2024-09-17T07:31:38.216065+0000","last_undegraded":"2024-09-17T07:31:38.216065+0000","last_fullsized":"2024-09-17T07:31:38.216065+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:17:26.947693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000464964,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":26,"num_read":9368,"num_read_kb":52923,"num_write":688,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":604,"num_bytes_recovered":24254406,"num_keys_recovered":93,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"163'559","reported_seq":2911,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214086+0000","last_change":"2024-09-17T07:31:14.114917+0000","last_active":"2024-09-17T07:31:38.214086+0000","last_peered":"2024-09-17T07:31:38.214086+0000","last_clean":"2024-09-17T07:31:38.214086+0000","last_became_active":"2024-09-17T07:30:19.087515+0000","last_became_peered":"2024-09-17T07:30:19.087515+0000","last_unstale":"2024-09-17T07:31:38.214086+0000","last_undegraded":"2024-09-17T07:31:38.214086+0000","last_fullsized":"2024-09-17T07:31:38.214086+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:42.469193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023177290000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2408,"num_read_kb":29904,"num_write":665,"num_write_kb":15633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":11178824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"165'873","reported_seq":5432,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215146+0000","last_change":"2024-09-17T07:31:14.107447+0000","last_active":"2024-09-17T07:31:38.215146+0000","last_peered":"2024-09-17T07:31:38.215146+0000","last_clean":"2024-09-17T07:31:38.215146+0000","last_became_active":"2024-09-17T07:30:19.085668+0000","last_became_peered":"2024-09-17T07:30:19.085668+0000","last_unstale":"2024-09-17T07:31:38.215146+0000","last_undegraded":"2024-09-17T07:31:38.215146+0000","last_fullsized":"2024-09-17T07:31:38.215146+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:47.391518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001368193,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6070,"num_read_kb":27721,"num_write":1113,"num_write_kb":14827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":15619889,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"165'725","reported_seq":4612,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372484+0000","last_change":"2024-09-17T07:31:14.176026+0000","last_active":"2024-09-17T07:31:37.372484+0000","last_peered":"2024-09-17T07:31:37.372484+0000","last_clean":"2024-09-17T07:31:37.372484+0000","last_became_active":"2024-09-17T07:28:43.080816+0000","last_became_peered":"2024-09-17T07:28:43.080816+0000","last_unstale":"2024-09-17T07:31:37.372484+0000","last_undegraded":"2024-09-17T07:31:37.372484+0000","last_fullsized":"2024-09-17T07:31:37.372484+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:05:06.879033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017783269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6005,"num_read_kb":48957,"num_write":832,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"164'458","reported_seq":8425,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372431+0000","last_change":"2024-09-17T07:31:26.155667+0000","last_active":"2024-09-17T07:31:37.372431+0000","last_peered":"2024-09-17T07:31:37.372431+0000","last_clean":"2024-09-17T07:31:37.372431+0000","last_became_active":"2024-09-17T07:31:26.154754+0000","last_became_peered":"2024-09-17T07:31:26.154754+0000","last_unstale":"2024-09-17T07:31:37.372431+0000","last_undegraded":"2024-09-17T07:31:37.372431+0000","last_fullsized":"2024-09-17T07:31:37.372431+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:47.550445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021346680000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7087,"num_read_kb":55690,"num_write":392,"num_write_kb":41376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":13797349,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"163'832","reported_seq":6338,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372621+0000","last_change":"2024-09-17T07:31:28.103546+0000","last_active":"2024-09-17T07:31:37.372621+0000","last_peered":"2024-09-17T07:31:37.372621+0000","last_clean":"2024-09-17T07:31:37.372621+0000","last_became_active":"2024-09-17T07:31:28.101799+0000","last_became_peered":"2024-09-17T07:31:28.101799+0000","last_unstale":"2024-09-17T07:31:37.372621+0000","last_undegraded":"2024-09-17T07:31:37.372621+0000","last_fullsized":"2024-09-17T07:31:37.372621+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:28.873325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021103049999999998,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6418,"num_read_kb":23171,"num_write":923,"num_write_kb":18541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":744,"num_bytes_recovered":25525458,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"164'401","reported_seq":2768,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216051+0000","last_change":"2024-09-17T07:31:14.720102+0000","last_active":"2024-09-17T07:31:38.216051+0000","last_peered":"2024-09-17T07:31:38.216051+0000","last_clean":"2024-09-17T07:31:38.216051+0000","last_became_active":"2024-09-17T07:28:43.077360+0000","last_became_peered":"2024-09-17T07:28:43.077360+0000","last_unstale":"2024-09-17T07:31:38.216051+0000","last_undegraded":"2024-09-17T07:31:38.216051+0000","last_fullsized":"2024-09-17T07:31:38.216051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:17.256040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061413699999999995,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2841,"num_read_kb":26198,"num_write":450,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"164'802","reported_seq":4337,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372196+0000","last_change":"2024-09-17T07:31:29.105135+0000","last_active":"2024-09-17T07:31:37.372196+0000","last_peered":"2024-09-17T07:31:37.372196+0000","last_clean":"2024-09-17T07:31:37.372196+0000","last_became_active":"2024-09-17T07:31:29.103626+0000","last_became_peered":"2024-09-17T07:31:29.103626+0000","last_unstale":"2024-09-17T07:31:37.372196+0000","last_undegraded":"2024-09-17T07:31:37.372196+0000","last_fullsized":"2024-09-17T07:31:37.372196+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:10.689986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001604722,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5310,"num_read_kb":45214,"num_write":937,"num_write_kb":26421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":468,"num_bytes_recovered":31676674,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"164'435","reported_seq":2745,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208126+0000","last_change":"2024-09-17T07:31:14.104829+0000","last_active":"2024-09-17T07:31:37.208126+0000","last_peered":"2024-09-17T07:31:37.208126+0000","last_clean":"2024-09-17T07:31:37.208126+0000","last_became_active":"2024-09-17T07:31:08.096240+0000","last_became_peered":"2024-09-17T07:31:08.096240+0000","last_unstale":"2024-09-17T07:31:37.208126+0000","last_undegraded":"2024-09-17T07:31:37.208126+0000","last_fullsized":"2024-09-17T07:31:37.208126+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:34.448388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098890999999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2231,"num_read_kb":44562,"num_write":412,"num_write_kb":29148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":24460350,"num_keys_recovered":37,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"164'772","reported_seq":4240,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372356+0000","last_change":"2024-09-17T07:31:29.104804+0000","last_active":"2024-09-17T07:31:37.372356+0000","last_peered":"2024-09-17T07:31:37.372356+0000","last_clean":"2024-09-17T07:31:37.372356+0000","last_became_active":"2024-09-17T07:31:29.103168+0000","last_became_peered":"2024-09-17T07:31:29.103168+0000","last_unstale":"2024-09-17T07:31:37.372356+0000","last_undegraded":"2024-09-17T07:31:37.372356+0000","last_fullsized":"2024-09-17T07:31:37.372356+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:31.292655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093421499999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4511,"num_read_kb":28124,"num_write":1074,"num_write_kb":16615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":21539718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":607,"num_read":166048,"num_read_kb":1208202,"num_write":23179,"num_write_kb":792760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9827,"num_bytes_recovered":666374238,"num_keys_recovered":891,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7902,"internal_metadata":0},"log_size":20553,"ondisk_log_size":20553,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":101032,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542186,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47324,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324516,"statfs":{"total":96636764160,"available":96588304384,"internally_reserved":0,"allocated":2301952,"data_stored":1679580,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33412,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338428,"statfs":{"total":96636764160,"available":96602550272,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55616,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316224,"statfs":{"total":96636764160,"available":96579813376,"internally_reserved":0,"allocated":2269184,"data_stored":1095102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542186,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91640,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280200,"statfs":{"total":96636764160,"available":96542924800,"internally_reserved":0,"allocated":2256896,"data_stored":1089231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056663,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43268,"kb_used_data":2212,"kb_used_omap":16,"kb_used_meta":41007,"kb_avail":94328572,"statfs":{"total":96636764160,"available":96592457728,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17058,"internal_metadata":41991518},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542186,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55612,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316228,"statfs":{"total":96636764160,"available":96579817472,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65336,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306504,"statfs":{"total":96636764160,"available":96569860096,"internally_reserved":0,"allocated":2260992,"data_stored":1089250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47340,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324500,"statfs":{"total":96636764160,"available":96588288000,"internally_reserved":0,"allocated":2318336,"data_stored":1679580,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7902,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:31:43.108 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T07:31:43.108 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:31:43.407 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:43.408 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:31:43.420 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-09-17T07:31:42.788723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":607,"num_read":166114,"num_read_kb":1208257,"num_write":23282,"num_write_kb":794390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9832,"num_bytes_recovered":667554974,"num_keys_recovered":891,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20628,"ondisk_log_size":20628,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439548,"kb_used_data":17776,"kb_used_omap":27,"kb_used_meta":421540,"kb_avail":754535172,"statfs":{"total":773094113280,"available":772644016128,"internally_reserved":0,"allocated":18202624,"data_stored":9917992,"data_compressed":66815,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":28217,"internal_metadata":431657415},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.592133"},"pg_stats":[{"pgid":"3.1f","version":"165'874","reported_seq":4617,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208112+0000","last_change":"2024-09-17T07:31:14.105507+0000","last_active":"2024-09-17T07:31:37.208112+0000","last_peered":"2024-09-17T07:31:37.208112+0000","last_clean":"2024-09-17T07:31:37.208112+0000","last_became_active":"2024-09-17T07:30:43.092456+0000","last_became_peered":"2024-09-17T07:30:43.092456+0000","last_unstale":"2024-09-17T07:31:37.208112+0000","last_undegraded":"2024-09-17T07:31:37.208112+0000","last_fullsized":"2024-09-17T07:31:37.208112+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:26.811072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016105900000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5801,"num_read_kb":34310,"num_write":997,"num_write_kb":41977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":26159662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"164'553","reported_seq":3285,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214114+0000","last_change":"2024-09-17T07:31:14.115590+0000","last_active":"2024-09-17T07:31:38.214114+0000","last_peered":"2024-09-17T07:31:38.214114+0000","last_clean":"2024-09-17T07:31:38.214114+0000","last_became_active":"2024-09-17T07:30:19.087213+0000","last_became_peered":"2024-09-17T07:30:19.087213+0000","last_unstale":"2024-09-17T07:31:38.214114+0000","last_undegraded":"2024-09-17T07:31:38.214114+0000","last_fullsized":"2024-09-17T07:31:38.214114+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:15.610729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029487620000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3721,"num_read_kb":30298,"num_write":527,"num_write_kb":15594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":15743718,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"163'550","reported_seq":3498,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214875+0000","last_change":"2024-09-17T07:31:26.150683+0000","last_active":"2024-09-17T07:31:38.214875+0000","last_peered":"2024-09-17T07:31:38.214875+0000","last_clean":"2024-09-17T07:31:38.214875+0000","last_became_active":"2024-09-17T07:31:26.149428+0000","last_became_peered":"2024-09-17T07:31:26.149428+0000","last_unstale":"2024-09-17T07:31:38.214875+0000","last_undegraded":"2024-09-17T07:31:38.214875+0000","last_fullsized":"2024-09-17T07:31:38.214875+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:58.834946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016933180000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":1948,"num_read_kb":28759,"num_write":579,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":27967854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"165'813","reported_seq":6628,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215109+0000","last_change":"2024-09-17T07:31:28.101214+0000","last_active":"2024-09-17T07:31:38.215109+0000","last_peered":"2024-09-17T07:31:38.215109+0000","last_clean":"2024-09-17T07:31:38.215109+0000","last_became_active":"2024-09-17T07:31:28.100048+0000","last_became_peered":"2024-09-17T07:31:28.100048+0000","last_unstale":"2024-09-17T07:31:38.215109+0000","last_undegraded":"2024-09-17T07:31:38.215109+0000","last_fullsized":"2024-09-17T07:31:38.215109+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:35.020631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010655949999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":7523,"num_read_kb":42454,"num_write":1061,"num_write_kb":38721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":26661650,"num_keys_recovered":180,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"165'606","reported_seq":5639,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208349+0000","last_change":"2024-09-17T07:31:28.104670+0000","last_active":"2024-09-17T07:31:37.208349+0000","last_peered":"2024-09-17T07:31:37.208349+0000","last_clean":"2024-09-17T07:31:37.208349+0000","last_became_active":"2024-09-17T07:31:28.102058+0000","last_became_peered":"2024-09-17T07:31:28.102058+0000","last_unstale":"2024-09-17T07:31:37.208349+0000","last_undegraded":"2024-09-17T07:31:37.208349+0000","last_fullsized":"2024-09-17T07:31:37.208349+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:07.797309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028238529999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5800,"num_read_kb":52463,"num_write":596,"num_write_kb":22879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":91784745,"num_keys_recovered":126,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"163'920","reported_seq":14894,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372319+0000","last_change":"2024-09-17T07:31:27.105439+0000","last_active":"2024-09-17T07:31:37.372319+0000","last_peered":"2024-09-17T07:31:37.372319+0000","last_clean":"2024-09-17T07:31:37.372319+0000","last_became_active":"2024-09-17T07:31:27.103933+0000","last_became_peered":"2024-09-17T07:31:27.103933+0000","last_unstale":"2024-09-17T07:31:37.372319+0000","last_undegraded":"2024-09-17T07:31:37.372319+0000","last_fullsized":"2024-09-17T07:31:37.372319+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:06:20.908980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096620079999999997,"stat_sum":{"num_bytes":19,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":16715,"num_read_kb":60981,"num_write":1188,"num_write_kb":20597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":19562028,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"164'439","reported_seq":5685,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145330+0000","last_change":"2024-09-17T07:31:14.189296+0000","last_active":"2024-09-17T07:31:39.145330+0000","last_peered":"2024-09-17T07:31:39.145330+0000","last_clean":"2024-09-17T07:31:39.145330+0000","last_became_active":"2024-09-17T07:30:18.083918+0000","last_became_peered":"2024-09-17T07:30:18.083918+0000","last_unstale":"2024-09-17T07:31:39.145330+0000","last_undegraded":"2024-09-17T07:31:39.145330+0000","last_fullsized":"2024-09-17T07:31:39.145330+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:26.983426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000820972,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4352,"num_read_kb":37354,"num_write":400,"num_write_kb":43089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":25536146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'763","reported_seq":4669,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214008+0000","last_change":"2024-09-17T07:31:14.113881+0000","last_active":"2024-09-17T07:31:38.214008+0000","last_peered":"2024-09-17T07:31:38.214008+0000","last_clean":"2024-09-17T07:31:38.214008+0000","last_became_active":"2024-09-17T07:30:43.091692+0000","last_became_peered":"2024-09-17T07:30:43.091692+0000","last_unstale":"2024-09-17T07:31:38.214008+0000","last_undegraded":"2024-09-17T07:31:38.214008+0000","last_fullsized":"2024-09-17T07:31:38.214008+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:24.550163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014749730000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5969,"num_read_kb":25327,"num_write":792,"num_write_kb":16603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":29456184,"num_keys_recovered":176,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"163'436","reported_seq":4226,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208396+0000","last_change":"2024-09-17T07:31:14.106118+0000","last_active":"2024-09-17T07:31:37.208396+0000","last_peered":"2024-09-17T07:31:37.208396+0000","last_clean":"2024-09-17T07:31:37.208396+0000","last_became_active":"2024-09-17T07:28:43.080428+0000","last_became_peered":"2024-09-17T07:28:43.080428+0000","last_unstale":"2024-09-17T07:31:37.208396+0000","last_undegraded":"2024-09-17T07:31:37.208396+0000","last_fullsized":"2024-09-17T07:31:37.208396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:12:14.112234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020783120000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4961,"num_read_kb":22717,"num_write":365,"num_write_kb":13531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"164'513","reported_seq":4191,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214897+0000","last_change":"2024-09-17T07:31:14.106949+0000","last_active":"2024-09-17T07:31:38.214897+0000","last_peered":"2024-09-17T07:31:38.214897+0000","last_clean":"2024-09-17T07:31:38.214897+0000","last_became_active":"2024-09-17T07:28:43.082313+0000","last_became_peered":"2024-09-17T07:28:43.082313+0000","last_unstale":"2024-09-17T07:31:38.214897+0000","last_undegraded":"2024-09-17T07:31:38.214897+0000","last_fullsized":"2024-09-17T07:31:38.214897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:35.352537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045261,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6140,"num_read_kb":58359,"num_write":449,"num_write_kb":31855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"164'390","reported_seq":4504,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372480+0000","last_change":"2024-09-17T07:31:14.175336+0000","last_active":"2024-09-17T07:31:37.372480+0000","last_peered":"2024-09-17T07:31:37.372480+0000","last_clean":"2024-09-17T07:31:37.372480+0000","last_became_active":"2024-09-17T07:31:08.094940+0000","last_became_peered":"2024-09-17T07:31:08.094940+0000","last_unstale":"2024-09-17T07:31:37.372480+0000","last_undegraded":"2024-09-17T07:31:37.372480+0000","last_fullsized":"2024-09-17T07:31:37.372480+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:58.292675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011121970000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3586,"num_read_kb":32352,"num_write":371,"num_write_kb":22781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":431,"num_bytes_recovered":60567103,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"163'465","reported_seq":2104,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372357+0000","last_change":"2024-09-17T07:31:26.153704+0000","last_active":"2024-09-17T07:31:37.372357+0000","last_peered":"2024-09-17T07:31:37.372357+0000","last_clean":"2024-09-17T07:31:37.372357+0000","last_became_active":"2024-09-17T07:31:26.152423+0000","last_became_peered":"2024-09-17T07:31:26.152423+0000","last_unstale":"2024-09-17T07:31:37.372357+0000","last_undegraded":"2024-09-17T07:31:37.372357+0000","last_fullsized":"2024-09-17T07:31:37.372357+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:46.055274+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015069525,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1414,"num_read_kb":17974,"num_write":482,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":16518806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":363,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216133+0000","last_change":"2024-09-17T07:31:05.098106+0000","last_active":"2024-09-17T07:31:38.216133+0000","last_peered":"2024-09-17T07:31:38.216133+0000","last_clean":"2024-09-17T07:31:38.216133+0000","last_became_active":"2024-09-17T07:31:05.097821+0000","last_became_peered":"2024-09-17T07:31:05.097821+0000","last_unstale":"2024-09-17T07:31:38.216133+0000","last_undegraded":"2024-09-17T07:31:38.216133+0000","last_fullsized":"2024-09-17T07:31:38.216133+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:12.297890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"165'1030","reported_seq":2761,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208278+0000","last_change":"2024-09-17T07:31:14.105440+0000","last_active":"2024-09-17T07:31:37.208278+0000","last_peered":"2024-09-17T07:31:37.208278+0000","last_clean":"2024-09-17T07:31:37.208278+0000","last_became_active":"2024-09-17T07:28:43.079229+0000","last_became_peered":"2024-09-17T07:28:43.079229+0000","last_unstale":"2024-09-17T07:31:37.208278+0000","last_undegraded":"2024-09-17T07:31:37.208278+0000","last_fullsized":"2024-09-17T07:31:37.208278+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1030,"log_dups_size":0,"ondisk_log_size":1030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:32.835400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015639079999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4625,"num_read_kb":48744,"num_write":1479,"num_write_kb":29297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":83,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145446+0000","last_change":"2024-09-17T07:31:01.268641+0000","last_active":"2024-09-17T07:31:39.145446+0000","last_peered":"2024-09-17T07:31:39.145446+0000","last_clean":"2024-09-17T07:31:39.145446+0000","last_became_active":"2024-09-17T07:31:01.268257+0000","last_became_peered":"2024-09-17T07:31:01.268257+0000","last_unstale":"2024-09-17T07:31:39.145446+0000","last_undegraded":"2024-09-17T07:31:39.145446+0000","last_fullsized":"2024-09-17T07:31:39.145446+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:04.188817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"164'577","reported_seq":3701,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214250+0000","last_change":"2024-09-17T07:31:14.115776+0000","last_active":"2024-09-17T07:31:38.214250+0000","last_peered":"2024-09-17T07:31:38.214250+0000","last_clean":"2024-09-17T07:31:38.214250+0000","last_became_active":"2024-09-17T07:30:19.087787+0000","last_became_peered":"2024-09-17T07:30:19.087787+0000","last_unstale":"2024-09-17T07:31:38.214250+0000","last_undegraded":"2024-09-17T07:31:38.214250+0000","last_fullsized":"2024-09-17T07:31:38.214250+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:26:55.414577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029713869999999998,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":3995,"num_read_kb":36995,"num_write":614,"num_write_kb":22622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":15169480,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":350,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372624+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:31:37.372624+0000","last_peered":"2024-09-17T07:31:37.372624+0000","last_clean":"2024-09-17T07:31:37.372624+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:31:37.372624+0000","last_undegraded":"2024-09-17T07:31:37.372624+0000","last_fullsized":"2024-09-17T07:31:37.372624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"163'1063","reported_seq":3603,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145306+0000","last_change":"2024-09-17T07:31:14.189299+0000","last_active":"2024-09-17T07:31:39.145306+0000","last_peered":"2024-09-17T07:31:39.145306+0000","last_clean":"2024-09-17T07:31:39.145306+0000","last_became_active":"2024-09-17T07:28:43.239387+0000","last_became_peered":"2024-09-17T07:28:43.239387+0000","last_unstale":"2024-09-17T07:31:39.145306+0000","last_undegraded":"2024-09-17T07:31:39.145306+0000","last_fullsized":"2024-09-17T07:31:39.145306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1063,"log_dups_size":0,"ondisk_log_size":1063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:09.407494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080983699999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6807,"num_read_kb":16873,"num_write":1444,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214910+0000","last_change":"2024-09-17T07:30:15.200774+0000","last_active":"2024-09-17T07:31:38.214910+0000","last_peered":"2024-09-17T07:31:38.214910+0000","last_clean":"2024-09-17T07:31:38.214910+0000","last_became_active":"2024-09-17T07:30:15.200470+0000","last_became_peered":"2024-09-17T07:30:15.200470+0000","last_unstale":"2024-09-17T07:31:38.214910+0000","last_undegraded":"2024-09-17T07:31:38.214910+0000","last_fullsized":"2024-09-17T07:31:38.214910+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:58.984439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"164'569","reported_seq":3303,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215021+0000","last_change":"2024-09-17T07:31:14.107519+0000","last_active":"2024-09-17T07:31:38.215021+0000","last_peered":"2024-09-17T07:31:38.215021+0000","last_clean":"2024-09-17T07:31:38.215021+0000","last_became_active":"2024-09-17T07:30:19.085184+0000","last_became_peered":"2024-09-17T07:30:19.085184+0000","last_unstale":"2024-09-17T07:31:38.215021+0000","last_undegraded":"2024-09-17T07:31:38.215021+0000","last_fullsized":"2024-09-17T07:31:38.215021+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:37.515132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015649349999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2687,"num_read_kb":46297,"num_write":632,"num_write_kb":27531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":35691417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"165'668","reported_seq":3157,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372462+0000","last_change":"2024-09-17T07:31:27.102404+0000","last_active":"2024-09-17T07:31:37.372462+0000","last_peered":"2024-09-17T07:31:37.372462+0000","last_clean":"2024-09-17T07:31:37.372462+0000","last_became_active":"2024-09-17T07:31:27.101204+0000","last_became_peered":"2024-09-17T07:31:27.101204+0000","last_unstale":"2024-09-17T07:31:37.372462+0000","last_undegraded":"2024-09-17T07:31:37.372462+0000","last_fullsized":"2024-09-17T07:31:37.372462+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:55.417631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028913630000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":2237,"num_read_kb":35376,"num_write":632,"num_write_kb":16684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":26727758,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":355,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214818+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:31:38.214818+0000","last_peered":"2024-09-17T07:31:38.214818+0000","last_clean":"2024-09-17T07:31:38.214818+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:31:38.214818+0000","last_undegraded":"2024-09-17T07:31:38.214818+0000","last_fullsized":"2024-09-17T07:31:38.214818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"164'825","reported_seq":11345,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145432+0000","last_change":"2024-09-17T07:31:14.189768+0000","last_active":"2024-09-17T07:31:39.145432+0000","last_peered":"2024-09-17T07:31:39.145432+0000","last_clean":"2024-09-17T07:31:39.145432+0000","last_became_active":"2024-09-17T07:28:43.239409+0000","last_became_peered":"2024-09-17T07:28:43.239409+0000","last_unstale":"2024-09-17T07:31:39.145432+0000","last_undegraded":"2024-09-17T07:31:39.145432+0000","last_fullsized":"2024-09-17T07:31:39.145432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":825,"log_dups_size":0,"ondisk_log_size":825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:13.253211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012368310000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":12617,"num_read_kb":91706,"num_write":1240,"num_write_kb":58366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":365,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.380263+0000","last_change":"2024-09-17T07:30:15.201999+0000","last_active":"2024-09-17T07:31:38.380263+0000","last_peered":"2024-09-17T07:31:38.380263+0000","last_clean":"2024-09-17T07:31:38.380263+0000","last_became_active":"2024-09-17T07:30:15.201642+0000","last_became_peered":"2024-09-17T07:30:15.201642+0000","last_unstale":"2024-09-17T07:31:38.380263+0000","last_undegraded":"2024-09-17T07:31:38.380263+0000","last_fullsized":"2024-09-17T07:31:38.380263+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:47.871565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"164'427","reported_seq":5376,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208231+0000","last_change":"2024-09-17T07:31:29.105514+0000","last_active":"2024-09-17T07:31:37.208231+0000","last_peered":"2024-09-17T07:31:37.208231+0000","last_clean":"2024-09-17T07:31:37.208231+0000","last_became_active":"2024-09-17T07:31:29.103782+0000","last_became_peered":"2024-09-17T07:31:29.103782+0000","last_unstale":"2024-09-17T07:31:37.208231+0000","last_undegraded":"2024-09-17T07:31:37.208231+0000","last_fullsized":"2024-09-17T07:31:37.208231+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:52.018922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094238099999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3788,"num_read_kb":51682,"num_write":424,"num_write_kb":28012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":37615005,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":362,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372172+0000","last_change":"2024-09-17T07:31:01.231085+0000","last_active":"2024-09-17T07:31:37.372172+0000","last_peered":"2024-09-17T07:31:37.372172+0000","last_clean":"2024-09-17T07:31:37.372172+0000","last_became_active":"2024-09-17T07:31:01.230794+0000","last_became_peered":"2024-09-17T07:31:01.230794+0000","last_unstale":"2024-09-17T07:31:37.372172+0000","last_undegraded":"2024-09-17T07:31:37.372172+0000","last_fullsized":"2024-09-17T07:31:37.372172+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:25.321726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"165'405","reported_seq":2497,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214012+0000","last_change":"2024-09-17T07:31:14.114890+0000","last_active":"2024-09-17T07:31:38.214012+0000","last_peered":"2024-09-17T07:31:38.214012+0000","last_clean":"2024-09-17T07:31:38.214012+0000","last_became_active":"2024-09-17T07:30:18.084247+0000","last_became_peered":"2024-09-17T07:30:18.084247+0000","last_unstale":"2024-09-17T07:31:38.214012+0000","last_undegraded":"2024-09-17T07:31:38.214012+0000","last_fullsized":"2024-09-17T07:31:38.214012+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:54:33.014267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002225922,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2958,"num_read_kb":9261,"num_write":364,"num_write_kb":6317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":6805602,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":138,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208474+0000","last_change":"2024-09-17T07:30:34.116563+0000","last_active":"2024-09-17T07:31:37.208474+0000","last_peered":"2024-09-17T07:31:37.208474+0000","last_clean":"2024-09-17T07:31:37.208474+0000","last_became_active":"2024-09-17T07:30:34.113598+0000","last_became_peered":"2024-09-17T07:30:34.113598+0000","last_unstale":"2024-09-17T07:31:37.208474+0000","last_undegraded":"2024-09-17T07:31:37.208474+0000","last_fullsized":"2024-09-17T07:31:37.208474+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:52.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":79,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372168+0000","last_change":"2024-09-17T07:31:05.102195+0000","last_active":"2024-09-17T07:31:37.372168+0000","last_peered":"2024-09-17T07:31:37.372168+0000","last_clean":"2024-09-17T07:31:37.372168+0000","last_became_active":"2024-09-17T07:31:05.096988+0000","last_became_peered":"2024-09-17T07:31:05.096988+0000","last_unstale":"2024-09-17T07:31:37.372168+0000","last_undegraded":"2024-09-17T07:31:37.372168+0000","last_fullsized":"2024-09-17T07:31:37.372168+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:01.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"164'520","reported_seq":2435,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214008+0000","last_change":"2024-09-17T07:31:14.113864+0000","last_active":"2024-09-17T07:31:38.214008+0000","last_peered":"2024-09-17T07:31:38.214008+0000","last_clean":"2024-09-17T07:31:38.214008+0000","last_became_active":"2024-09-17T07:30:19.086777+0000","last_became_peered":"2024-09-17T07:30:19.086777+0000","last_unstale":"2024-09-17T07:31:38.214008+0000","last_undegraded":"2024-09-17T07:31:38.214008+0000","last_fullsized":"2024-09-17T07:31:38.214008+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:55.561658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014795710000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1640,"num_read_kb":24382,"num_write":527,"num_write_kb":33950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":24920122,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"164'551","reported_seq":3917,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372211+0000","last_change":"2024-09-17T07:31:27.105903+0000","last_active":"2024-09-17T07:31:37.372211+0000","last_peered":"2024-09-17T07:31:37.372211+0000","last_clean":"2024-09-17T07:31:37.372211+0000","last_became_active":"2024-09-17T07:31:27.103978+0000","last_became_peered":"2024-09-17T07:31:27.103978+0000","last_unstale":"2024-09-17T07:31:37.372211+0000","last_undegraded":"2024-09-17T07:31:37.372211+0000","last_fullsized":"2024-09-17T07:31:37.372211+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:13.427259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001973519,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4515,"num_read_kb":21074,"num_write":530,"num_write_kb":12196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":11434290,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"164'739","reported_seq":9612,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216065+0000","last_change":"2024-09-17T07:31:28.103185+0000","last_active":"2024-09-17T07:31:38.216065+0000","last_peered":"2024-09-17T07:31:38.216065+0000","last_clean":"2024-09-17T07:31:38.216065+0000","last_became_active":"2024-09-17T07:31:28.101926+0000","last_became_peered":"2024-09-17T07:31:28.101926+0000","last_unstale":"2024-09-17T07:31:38.216065+0000","last_undegraded":"2024-09-17T07:31:38.216065+0000","last_fullsized":"2024-09-17T07:31:38.216065+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:17:26.947693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000464964,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":26,"num_read":9368,"num_read_kb":52923,"num_write":688,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":604,"num_bytes_recovered":24254406,"num_keys_recovered":93,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"163'559","reported_seq":2911,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214086+0000","last_change":"2024-09-17T07:31:14.114917+0000","last_active":"2024-09-17T07:31:38.214086+0000","last_peered":"2024-09-17T07:31:38.214086+0000","last_clean":"2024-09-17T07:31:38.214086+0000","last_became_active":"2024-09-17T07:30:19.087515+0000","last_became_peered":"2024-09-17T07:30:19.087515+0000","last_unstale":"2024-09-17T07:31:38.214086+0000","last_undegraded":"2024-09-17T07:31:38.214086+0000","last_fullsized":"2024-09-17T07:31:38.214086+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:42.469193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023177290000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2408,"num_read_kb":29904,"num_write":665,"num_write_kb":15633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":11178824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"165'873","reported_seq":5432,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215146+0000","last_change":"2024-09-17T07:31:14.107447+0000","last_active":"2024-09-17T07:31:38.215146+0000","last_peered":"2024-09-17T07:31:38.215146+0000","last_clean":"2024-09-17T07:31:38.215146+0000","last_became_active":"2024-09-17T07:30:19.085668+0000","last_became_peered":"2024-09-17T07:30:19.085668+0000","last_unstale":"2024-09-17T07:31:38.215146+0000","last_undegraded":"2024-09-17T07:31:38.215146+0000","last_fullsized":"2024-09-17T07:31:38.215146+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:47.391518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001368193,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6070,"num_read_kb":27721,"num_write":1113,"num_write_kb":14827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":15619889,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"165'725","reported_seq":4612,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372484+0000","last_change":"2024-09-17T07:31:14.176026+0000","last_active":"2024-09-17T07:31:37.372484+0000","last_peered":"2024-09-17T07:31:37.372484+0000","last_clean":"2024-09-17T07:31:37.372484+0000","last_became_active":"2024-09-17T07:28:43.080816+0000","last_became_peered":"2024-09-17T07:28:43.080816+0000","last_unstale":"2024-09-17T07:31:37.372484+0000","last_undegraded":"2024-09-17T07:31:37.372484+0000","last_fullsized":"2024-09-17T07:31:37.372484+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:05:06.879033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017783269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6005,"num_read_kb":48957,"num_write":832,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"164'458","reported_seq":8425,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372431+0000","last_change":"2024-09-17T07:31:26.155667+0000","last_active":"2024-09-17T07:31:37.372431+0000","last_peered":"2024-09-17T07:31:37.372431+0000","last_clean":"2024-09-17T07:31:37.372431+0000","last_became_active":"2024-09-17T07:31:26.154754+0000","last_became_peered":"2024-09-17T07:31:26.154754+0000","last_unstale":"2024-09-17T07:31:37.372431+0000","last_undegraded":"2024-09-17T07:31:37.372431+0000","last_fullsized":"2024-09-17T07:31:37.372431+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:47.550445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021346680000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7087,"num_read_kb":55690,"num_write":392,"num_write_kb":41376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":13797349,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"163'832","reported_seq":6338,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372621+0000","last_change":"2024-09-17T07:31:28.103546+0000","last_active":"2024-09-17T07:31:37.372621+0000","last_peered":"2024-09-17T07:31:37.372621+0000","last_clean":"2024-09-17T07:31:37.372621+0000","last_became_active":"2024-09-17T07:31:28.101799+0000","last_became_peered":"2024-09-17T07:31:28.101799+0000","last_unstale":"2024-09-17T07:31:37.372621+0000","last_undegraded":"2024-09-17T07:31:37.372621+0000","last_fullsized":"2024-09-17T07:31:37.372621+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:28.873325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021103049999999998,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6418,"num_read_kb":23171,"num_write":923,"num_write_kb":18541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":744,"num_bytes_recovered":25525458,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"164'401","reported_seq":2768,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216051+0000","last_change":"2024-09-17T07:31:14.720102+0000","last_active":"2024-09-17T07:31:38.216051+0000","last_peered":"2024-09-17T07:31:38.216051+0000","last_clean":"2024-09-17T07:31:38.216051+0000","last_became_active":"2024-09-17T07:28:43.077360+0000","last_became_peered":"2024-09-17T07:28:43.077360+0000","last_unstale":"2024-09-17T07:31:38.216051+0000","last_undegraded":"2024-09-17T07:31:38.216051+0000","last_fullsized":"2024-09-17T07:31:38.216051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:17.256040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061413699999999995,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2841,"num_read_kb":26198,"num_write":450,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"164'802","reported_seq":4337,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372196+0000","last_change":"2024-09-17T07:31:29.105135+0000","last_active":"2024-09-17T07:31:37.372196+0000","last_peered":"2024-09-17T07:31:37.372196+0000","last_clean":"2024-09-17T07:31:37.372196+0000","last_became_active":"2024-09-17T07:31:29.103626+0000","last_became_peered":"2024-09-17T07:31:29.103626+0000","last_unstale":"2024-09-17T07:31:37.372196+0000","last_undegraded":"2024-09-17T07:31:37.372196+0000","last_fullsized":"2024-09-17T07:31:37.372196+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:10.689986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001604722,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5310,"num_read_kb":45214,"num_write":937,"num_write_kb":26421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":468,"num_bytes_recovered":31676674,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"164'435","reported_seq":2745,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.208126+0000","last_change":"2024-09-17T07:31:14.104829+0000","last_active":"2024-09-17T07:31:37.208126+0000","last_peered":"2024-09-17T07:31:37.208126+0000","last_clean":"2024-09-17T07:31:37.208126+0000","last_became_active":"2024-09-17T07:31:08.096240+0000","last_became_peered":"2024-09-17T07:31:08.096240+0000","last_unstale":"2024-09-17T07:31:37.208126+0000","last_undegraded":"2024-09-17T07:31:37.208126+0000","last_fullsized":"2024-09-17T07:31:37.208126+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:34.448388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098890999999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2231,"num_read_kb":44562,"num_write":412,"num_write_kb":29148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":24460350,"num_keys_recovered":37,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"164'772","reported_seq":4240,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-09-17T07:31:37.372356+0000","last_change":"2024-09-17T07:31:29.104804+0000","last_active":"2024-09-17T07:31:37.372356+0000","last_peered":"2024-09-17T07:31:37.372356+0000","last_clean":"2024-09-17T07:31:37.372356+0000","last_became_active":"2024-09-17T07:31:29.103168+0000","last_became_peered":"2024-09-17T07:31:29.103168+0000","last_unstale":"2024-09-17T07:31:37.372356+0000","last_undegraded":"2024-09-17T07:31:37.372356+0000","last_fullsized":"2024-09-17T07:31:37.372356+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:31.292655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093421499999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4511,"num_read_kb":28124,"num_write":1074,"num_write_kb":16615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":21539718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":607,"num_read":166048,"num_read_kb":1208202,"num_write":23179,"num_write_kb":792760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9827,"num_bytes_recovered":666374238,"num_keys_recovered":891,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7902,"internal_metadata":0},"log_size":20553,"ondisk_log_size":20553,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":101032,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542186,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47324,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324516,"statfs":{"total":96636764160,"available":96588304384,"internally_reserved":0,"allocated":2301952,"data_stored":1679580,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33412,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338428,"statfs":{"total":96636764160,"available":96602550272,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55616,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316224,"statfs":{"total":96636764160,"available":96579813376,"internally_reserved":0,"allocated":2269184,"data_stored":1095102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542186,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91640,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280200,"statfs":{"total":96636764160,"available":96542924800,"internally_reserved":0,"allocated":2256896,"data_stored":1089231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056663,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43268,"kb_used_data":2212,"kb_used_omap":16,"kb_used_meta":41007,"kb_avail":94328572,"statfs":{"total":96636764160,"available":96592457728,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17058,"internal_metadata":41991518},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542186,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55612,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316228,"statfs":{"total":96636764160,"available":96579817472,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542186,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65336,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306504,"statfs":{"total":96636764160,"available":96569860096,"internally_reserved":0,"allocated":2260992,"data_stored":1089250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47340,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324500,"statfs":{"total":96636764160,"available":96588288000,"internally_reserved":0,"allocated":2318336,"data_stored":1679580,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7902,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:31:43.422 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T07:31:43.721 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:43.722 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":188,"fsid":"b283e6d5-0f4a-4271-a443-15e695e35364","created":"2024-09-17T07:28:12.276015+0000","modified":"2024-09-17T07:31:38.206026+0000","last_up_change":"2024-09-17T07:29:58.120417+0000","last_in_change":"2024-09-17T07:31:22.090468+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T07:28:24.767696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T07:28:30.249487+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T07:28:41.831004+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":165,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7794e6a8-ebf4-4b7c-bf3d-63cd015234ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6801","nonce":2039818160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6803","nonce":2039818160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6807","nonce":2039818160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":2039818160},{"type":"v1","addr":"172.21.15.45:6805","nonce":2039818160}]},"public_addr":"172.21.15.45:6801/2039818160","cluster_addr":"172.21.15.45:6803/2039818160","heartbeat_back_addr":"172.21.15.45:6807/2039818160","heartbeat_front_addr":"172.21.15.45:6805/2039818160","state":["exists","up"]},{"osd":1,"uuid":"f374d4ce-2785-4b73-8b6d-10fe6fcaf8d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6809","nonce":1158817756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6811","nonce":1158817756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6815","nonce":1158817756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":1158817756},{"type":"v1","addr":"172.21.15.45:6813","nonce":1158817756}]},"public_addr":"172.21.15.45:6809/1158817756","cluster_addr":"172.21.15.45:6811/1158817756","heartbeat_back_addr":"172.21.15.45:6815/1158817756","heartbeat_front_addr":"172.21.15.45:6813/1158817756","state":["exists","up"]},{"osd":2,"uuid":"195a9db1-568d-459a-bb50-2a1bb2445f53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6825","nonce":3298886784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6827","nonce":3298886784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6831","nonce":3298886784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":3298886784},{"type":"v1","addr":"172.21.15.45:6829","nonce":3298886784}]},"public_addr":"172.21.15.45:6825/3298886784","cluster_addr":"172.21.15.45:6827/3298886784","heartbeat_back_addr":"172.21.15.45:6831/3298886784","heartbeat_front_addr":"172.21.15.45:6829/3298886784","state":["exists","up"]},{"osd":3,"uuid":"b3fda98b-a10a-4354-905e-1b37a5124b98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":36,"up_from":90,"up_thru":157,"down_at":37,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":724784918},{"type":"v1","addr":"172.21.15.45:6817","nonce":724784918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":724784918},{"type":"v1","addr":"172.21.15.45:6819","nonce":724784918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":724784918},{"type":"v1","addr":"172.21.15.45:6823","nonce":724784918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":724784918},{"type":"v1","addr":"172.21.15.45:6821","nonce":724784918}]},"public_addr":"172.21.15.45:6817/724784918","cluster_addr":"172.21.15.45:6819/724784918","heartbeat_back_addr":"172.21.15.45:6823/724784918","heartbeat_front_addr":"172.21.15.45:6821/724784918","state":["exists","up"]},{"osd":4,"uuid":"6ab67b48-2d83-4e68-8898-302a1adc0a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":176,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6825","nonce":1644784047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6827","nonce":1644784047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6831","nonce":1644784047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1644784047},{"type":"v1","addr":"172.21.15.192:6829","nonce":1644784047}]},"public_addr":"172.21.15.192:6825/1644784047","cluster_addr":"172.21.15.192:6827/1644784047","heartbeat_back_addr":"172.21.15.192:6831/1644784047","heartbeat_front_addr":"172.21.15.192:6829/1644784047","state":["exists","up"]},{"osd":5,"uuid":"ae3c787c-63d2-4e88-8f99-6d4f19837a3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6817","nonce":4077426583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6819","nonce":4077426583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6823","nonce":4077426583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":4077426583},{"type":"v1","addr":"172.21.15.192:6821","nonce":4077426583}]},"public_addr":"172.21.15.192:6817/4077426583","cluster_addr":"172.21.15.192:6819/4077426583","heartbeat_back_addr":"172.21.15.192:6823/4077426583","heartbeat_front_addr":"172.21.15.192:6821/4077426583","state":["exists","up"]},{"osd":6,"uuid":"f792f18d-ad9e-49a8-b194-24522398fb25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6809","nonce":2922229639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6811","nonce":2922229639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6815","nonce":2922229639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":2922229639},{"type":"v1","addr":"172.21.15.192:6813","nonce":2922229639}]},"public_addr":"172.21.15.192:6809/2922229639","cluster_addr":"172.21.15.192:6811/2922229639","heartbeat_back_addr":"172.21.15.192:6815/2922229639","heartbeat_front_addr":"172.21.15.192:6813/2922229639","state":["exists","up"]},{"osd":7,"uuid":"4a589d43-b0b9-42de-989c-46928d693c24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":178,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6801","nonce":780298944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6803","nonce":780298944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6807","nonce":780298944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":780298944},{"type":"v1","addr":"172.21.15.192:6805","nonce":780298944}]},"public_addr":"172.21.15.192:6801/780298944","cluster_addr":"172.21.15.192:6803/780298944","heartbeat_back_addr":"172.21.15.192:6807/780298944","heartbeat_front_addr":"172.21.15.192:6805/780298944","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.586514+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.833505+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:21.059899+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.499470+0000","dead_epoch":37},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:21.353342+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.586673+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.767847+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T07:28:20.859615+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.4","osds":[5,3]}],"pg_upmap_items":[{"pgid":"2.3","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T07:31:44.735 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T07:31:44.735 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T07:31:44.876 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:44.877 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T07:31:44.877 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:44.887 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T07:31:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T07:31:45.199 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T07:31:45.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T07:31:45.327 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:45.327 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T07:31:45.327 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:45.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T07:31:45.633 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T07:31:45.646 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T07:31:45.646 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T07:31:45.773 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:45.773 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T07:31:45.773 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:45.784 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T07:31:46.081 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T07:31:46.094 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T07:31:46.095 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T07:31:46.221 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:46.221 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T07:31:46.222 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:46.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T07:31:46.527 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T07:31:46.540 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T07:31:46.540 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T07:31:46.670 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:46.670 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T07:31:46.670 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:46.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T07:31:46.984 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T07:31:46.997 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T07:31:46.997 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T07:31:47.129 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:47.129 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T07:31:47.129 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:47.139 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T07:31:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T07:31:47.455 INFO:tasks.ceph:Scrubbing osd.6 2024-09-17T07:31:47.455 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-17T07:31:47.582 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:47.582 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T07:31:47.582 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:47.593 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-17T07:31:47.899 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 6 to deep-scrub 2024-09-17T07:31:47.912 INFO:tasks.ceph:Scrubbing osd.7 2024-09-17T07:31:47.912 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-17T07:31:48.040 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-17T07:31:48.040 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T07:31:48.040 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-17T07:31:48.050 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-17T07:31:48.357 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 7 to deep-scrub 2024-09-17T07:31:48.372 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:31:48.676 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:31:48.677 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:31:48.688 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-09-17T07:31:46.789485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":607,"num_read":166114,"num_read_kb":1208257,"num_write":23282,"num_write_kb":794390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9832,"num_bytes_recovered":667554974,"num_keys_recovered":891,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20628,"ondisk_log_size":20628,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439592,"kb_used_data":17824,"kb_used_omap":27,"kb_used_meta":421540,"kb_avail":754535128,"statfs":{"total":773094113280,"available":772643971072,"internally_reserved":0,"allocated":18251776,"data_stored":9941476,"data_compressed":66815,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":28217,"internal_metadata":431657415},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.583281"},"pg_stats":[{"pgid":"3.1f","version":"165'874","reported_seq":4619,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.130392+0000","last_change":"2024-09-17T07:31:14.105507+0000","last_active":"2024-09-17T07:31:39.130392+0000","last_peered":"2024-09-17T07:31:39.130392+0000","last_clean":"2024-09-17T07:31:39.130392+0000","last_became_active":"2024-09-17T07:30:43.092456+0000","last_became_peered":"2024-09-17T07:30:43.092456+0000","last_unstale":"2024-09-17T07:31:39.130392+0000","last_undegraded":"2024-09-17T07:31:39.130392+0000","last_fullsized":"2024-09-17T07:31:39.130392+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:26.811072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016105900000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5801,"num_read_kb":34310,"num_write":997,"num_write_kb":41977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":26159662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"164'553","reported_seq":3285,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214114+0000","last_change":"2024-09-17T07:31:14.115590+0000","last_active":"2024-09-17T07:31:38.214114+0000","last_peered":"2024-09-17T07:31:38.214114+0000","last_clean":"2024-09-17T07:31:38.214114+0000","last_became_active":"2024-09-17T07:30:19.087213+0000","last_became_peered":"2024-09-17T07:30:19.087213+0000","last_unstale":"2024-09-17T07:31:38.214114+0000","last_undegraded":"2024-09-17T07:31:38.214114+0000","last_fullsized":"2024-09-17T07:31:38.214114+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:15.610729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029487620000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3721,"num_read_kb":30298,"num_write":527,"num_write_kb":15594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":15743718,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"163'550","reported_seq":3498,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214875+0000","last_change":"2024-09-17T07:31:26.150683+0000","last_active":"2024-09-17T07:31:38.214875+0000","last_peered":"2024-09-17T07:31:38.214875+0000","last_clean":"2024-09-17T07:31:38.214875+0000","last_became_active":"2024-09-17T07:31:26.149428+0000","last_became_peered":"2024-09-17T07:31:26.149428+0000","last_unstale":"2024-09-17T07:31:38.214875+0000","last_undegraded":"2024-09-17T07:31:38.214875+0000","last_fullsized":"2024-09-17T07:31:38.214875+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:58.834946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016933180000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":1948,"num_read_kb":28759,"num_write":579,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":27967854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"165'813","reported_seq":6628,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215109+0000","last_change":"2024-09-17T07:31:28.101214+0000","last_active":"2024-09-17T07:31:38.215109+0000","last_peered":"2024-09-17T07:31:38.215109+0000","last_clean":"2024-09-17T07:31:38.215109+0000","last_became_active":"2024-09-17T07:31:28.100048+0000","last_became_peered":"2024-09-17T07:31:28.100048+0000","last_unstale":"2024-09-17T07:31:38.215109+0000","last_undegraded":"2024-09-17T07:31:38.215109+0000","last_fullsized":"2024-09-17T07:31:38.215109+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:35.020631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010655949999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":7523,"num_read_kb":42454,"num_write":1061,"num_write_kb":38721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":26661650,"num_keys_recovered":180,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"165'606","reported_seq":5641,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.130102+0000","last_change":"2024-09-17T07:31:28.104670+0000","last_active":"2024-09-17T07:31:39.130102+0000","last_peered":"2024-09-17T07:31:39.130102+0000","last_clean":"2024-09-17T07:31:39.130102+0000","last_became_active":"2024-09-17T07:31:28.102058+0000","last_became_peered":"2024-09-17T07:31:28.102058+0000","last_unstale":"2024-09-17T07:31:39.130102+0000","last_undegraded":"2024-09-17T07:31:39.130102+0000","last_fullsized":"2024-09-17T07:31:39.130102+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:07.797309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028238529999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5800,"num_read_kb":52463,"num_write":596,"num_write_kb":22879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":91784745,"num_keys_recovered":126,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"163'920","reported_seq":14896,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.683913+0000","last_change":"2024-09-17T07:31:27.105439+0000","last_active":"2024-09-17T07:31:39.683913+0000","last_peered":"2024-09-17T07:31:39.683913+0000","last_clean":"2024-09-17T07:31:39.683913+0000","last_became_active":"2024-09-17T07:31:27.103933+0000","last_became_peered":"2024-09-17T07:31:27.103933+0000","last_unstale":"2024-09-17T07:31:39.683913+0000","last_undegraded":"2024-09-17T07:31:39.683913+0000","last_fullsized":"2024-09-17T07:31:39.683913+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:06:20.908980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096620079999999997,"stat_sum":{"num_bytes":19,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":16715,"num_read_kb":60981,"num_write":1188,"num_write_kb":20597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":19562028,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"164'439","reported_seq":5685,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145330+0000","last_change":"2024-09-17T07:31:14.189296+0000","last_active":"2024-09-17T07:31:39.145330+0000","last_peered":"2024-09-17T07:31:39.145330+0000","last_clean":"2024-09-17T07:31:39.145330+0000","last_became_active":"2024-09-17T07:30:18.083918+0000","last_became_peered":"2024-09-17T07:30:18.083918+0000","last_unstale":"2024-09-17T07:31:39.145330+0000","last_undegraded":"2024-09-17T07:31:39.145330+0000","last_fullsized":"2024-09-17T07:31:39.145330+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:47:26.983426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000820972,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4352,"num_read_kb":37354,"num_write":400,"num_write_kb":43089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":25536146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'763","reported_seq":4669,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214008+0000","last_change":"2024-09-17T07:31:14.113881+0000","last_active":"2024-09-17T07:31:38.214008+0000","last_peered":"2024-09-17T07:31:38.214008+0000","last_clean":"2024-09-17T07:31:38.214008+0000","last_became_active":"2024-09-17T07:30:43.091692+0000","last_became_peered":"2024-09-17T07:30:43.091692+0000","last_unstale":"2024-09-17T07:31:38.214008+0000","last_undegraded":"2024-09-17T07:31:38.214008+0000","last_fullsized":"2024-09-17T07:31:38.214008+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:24.550163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014749730000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5969,"num_read_kb":25327,"num_write":792,"num_write_kb":16603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":29456184,"num_keys_recovered":176,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"163'436","reported_seq":4228,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.130346+0000","last_change":"2024-09-17T07:31:14.106118+0000","last_active":"2024-09-17T07:31:39.130346+0000","last_peered":"2024-09-17T07:31:39.130346+0000","last_clean":"2024-09-17T07:31:39.130346+0000","last_became_active":"2024-09-17T07:28:43.080428+0000","last_became_peered":"2024-09-17T07:28:43.080428+0000","last_unstale":"2024-09-17T07:31:39.130346+0000","last_undegraded":"2024-09-17T07:31:39.130346+0000","last_fullsized":"2024-09-17T07:31:39.130346+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:12:14.112234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020783120000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4961,"num_read_kb":22717,"num_write":365,"num_write_kb":13531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"164'513","reported_seq":4191,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214897+0000","last_change":"2024-09-17T07:31:14.106949+0000","last_active":"2024-09-17T07:31:38.214897+0000","last_peered":"2024-09-17T07:31:38.214897+0000","last_clean":"2024-09-17T07:31:38.214897+0000","last_became_active":"2024-09-17T07:28:43.082313+0000","last_became_peered":"2024-09-17T07:28:43.082313+0000","last_unstale":"2024-09-17T07:31:38.214897+0000","last_undegraded":"2024-09-17T07:31:38.214897+0000","last_fullsized":"2024-09-17T07:31:38.214897+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:35.352537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045261,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6140,"num_read_kb":58359,"num_write":449,"num_write_kb":31855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"164'390","reported_seq":4506,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.683606+0000","last_change":"2024-09-17T07:31:14.175336+0000","last_active":"2024-09-17T07:31:39.683606+0000","last_peered":"2024-09-17T07:31:39.683606+0000","last_clean":"2024-09-17T07:31:39.683606+0000","last_became_active":"2024-09-17T07:31:08.094940+0000","last_became_peered":"2024-09-17T07:31:08.094940+0000","last_unstale":"2024-09-17T07:31:39.683606+0000","last_undegraded":"2024-09-17T07:31:39.683606+0000","last_fullsized":"2024-09-17T07:31:39.683606+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:58.292675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011121970000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3586,"num_read_kb":32352,"num_write":371,"num_write_kb":22781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":431,"num_bytes_recovered":60567103,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"163'465","reported_seq":2106,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.683750+0000","last_change":"2024-09-17T07:31:26.153704+0000","last_active":"2024-09-17T07:31:39.683750+0000","last_peered":"2024-09-17T07:31:39.683750+0000","last_clean":"2024-09-17T07:31:39.683750+0000","last_became_active":"2024-09-17T07:31:26.152423+0000","last_became_peered":"2024-09-17T07:31:26.152423+0000","last_unstale":"2024-09-17T07:31:39.683750+0000","last_undegraded":"2024-09-17T07:31:39.683750+0000","last_fullsized":"2024-09-17T07:31:39.683750+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:46.055274+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015069525,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1414,"num_read_kb":17974,"num_write":482,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":16518806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":363,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216133+0000","last_change":"2024-09-17T07:31:05.098106+0000","last_active":"2024-09-17T07:31:38.216133+0000","last_peered":"2024-09-17T07:31:38.216133+0000","last_clean":"2024-09-17T07:31:38.216133+0000","last_became_active":"2024-09-17T07:31:05.097821+0000","last_became_peered":"2024-09-17T07:31:05.097821+0000","last_unstale":"2024-09-17T07:31:38.216133+0000","last_undegraded":"2024-09-17T07:31:38.216133+0000","last_fullsized":"2024-09-17T07:31:38.216133+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:12.297890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"165'1030","reported_seq":2763,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.130153+0000","last_change":"2024-09-17T07:31:14.105440+0000","last_active":"2024-09-17T07:31:39.130153+0000","last_peered":"2024-09-17T07:31:39.130153+0000","last_clean":"2024-09-17T07:31:39.130153+0000","last_became_active":"2024-09-17T07:28:43.079229+0000","last_became_peered":"2024-09-17T07:28:43.079229+0000","last_unstale":"2024-09-17T07:31:39.130153+0000","last_undegraded":"2024-09-17T07:31:39.130153+0000","last_fullsized":"2024-09-17T07:31:39.130153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1030,"log_dups_size":0,"ondisk_log_size":1030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:32.835400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015639079999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4625,"num_read_kb":48744,"num_write":1479,"num_write_kb":29297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":83,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145446+0000","last_change":"2024-09-17T07:31:01.268641+0000","last_active":"2024-09-17T07:31:39.145446+0000","last_peered":"2024-09-17T07:31:39.145446+0000","last_clean":"2024-09-17T07:31:39.145446+0000","last_became_active":"2024-09-17T07:31:01.268257+0000","last_became_peered":"2024-09-17T07:31:01.268257+0000","last_unstale":"2024-09-17T07:31:39.145446+0000","last_undegraded":"2024-09-17T07:31:39.145446+0000","last_fullsized":"2024-09-17T07:31:39.145446+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:04.188817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"164'577","reported_seq":3701,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214250+0000","last_change":"2024-09-17T07:31:14.115776+0000","last_active":"2024-09-17T07:31:38.214250+0000","last_peered":"2024-09-17T07:31:38.214250+0000","last_clean":"2024-09-17T07:31:38.214250+0000","last_became_active":"2024-09-17T07:30:19.087787+0000","last_became_peered":"2024-09-17T07:30:19.087787+0000","last_unstale":"2024-09-17T07:31:38.214250+0000","last_undegraded":"2024-09-17T07:31:38.214250+0000","last_fullsized":"2024-09-17T07:31:38.214250+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:26:55.414577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029713869999999998,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":3995,"num_read_kb":36995,"num_write":614,"num_write_kb":22622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":15169480,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":352,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.683727+0000","last_change":"2024-09-17T07:28:33.951359+0000","last_active":"2024-09-17T07:31:39.683727+0000","last_peered":"2024-09-17T07:31:39.683727+0000","last_clean":"2024-09-17T07:31:39.683727+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:31:39.683727+0000","last_undegraded":"2024-09-17T07:31:39.683727+0000","last_fullsized":"2024-09-17T07:31:39.683727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:41.941706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"163'1063","reported_seq":3603,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145306+0000","last_change":"2024-09-17T07:31:14.189299+0000","last_active":"2024-09-17T07:31:39.145306+0000","last_peered":"2024-09-17T07:31:39.145306+0000","last_clean":"2024-09-17T07:31:39.145306+0000","last_became_active":"2024-09-17T07:28:43.239387+0000","last_became_peered":"2024-09-17T07:28:43.239387+0000","last_unstale":"2024-09-17T07:31:39.145306+0000","last_undegraded":"2024-09-17T07:31:39.145306+0000","last_fullsized":"2024-09-17T07:31:39.145306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":1063,"log_dups_size":0,"ondisk_log_size":1063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:09.407494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080983699999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6807,"num_read_kb":16873,"num_write":1444,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214910+0000","last_change":"2024-09-17T07:30:15.200774+0000","last_active":"2024-09-17T07:31:38.214910+0000","last_peered":"2024-09-17T07:31:38.214910+0000","last_clean":"2024-09-17T07:31:38.214910+0000","last_became_active":"2024-09-17T07:30:15.200470+0000","last_became_peered":"2024-09-17T07:30:15.200470+0000","last_unstale":"2024-09-17T07:31:38.214910+0000","last_undegraded":"2024-09-17T07:31:38.214910+0000","last_fullsized":"2024-09-17T07:31:38.214910+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:59:58.984439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"164'569","reported_seq":3303,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215021+0000","last_change":"2024-09-17T07:31:14.107519+0000","last_active":"2024-09-17T07:31:38.215021+0000","last_peered":"2024-09-17T07:31:38.215021+0000","last_clean":"2024-09-17T07:31:38.215021+0000","last_became_active":"2024-09-17T07:30:19.085184+0000","last_became_peered":"2024-09-17T07:30:19.085184+0000","last_unstale":"2024-09-17T07:31:38.215021+0000","last_undegraded":"2024-09-17T07:31:38.215021+0000","last_fullsized":"2024-09-17T07:31:38.215021+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:37.515132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015649349999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2687,"num_read_kb":46297,"num_write":632,"num_write_kb":27531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":35691417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"165'668","reported_seq":3159,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.684087+0000","last_change":"2024-09-17T07:31:27.102404+0000","last_active":"2024-09-17T07:31:39.684087+0000","last_peered":"2024-09-17T07:31:39.684087+0000","last_clean":"2024-09-17T07:31:39.684087+0000","last_became_active":"2024-09-17T07:31:27.101204+0000","last_became_peered":"2024-09-17T07:31:27.101204+0000","last_unstale":"2024-09-17T07:31:39.684087+0000","last_undegraded":"2024-09-17T07:31:39.684087+0000","last_fullsized":"2024-09-17T07:31:39.684087+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:55.417631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028913630000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":2237,"num_read_kb":35376,"num_write":632,"num_write_kb":16684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":26727758,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":355,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214818+0000","last_change":"2024-09-17T07:28:33.361821+0000","last_active":"2024-09-17T07:31:38.214818+0000","last_peered":"2024-09-17T07:31:38.214818+0000","last_clean":"2024-09-17T07:31:38.214818+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:31:38.214818+0000","last_undegraded":"2024-09-17T07:31:38.214818+0000","last_fullsized":"2024-09-17T07:31:38.214818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:16.630015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"164'825","reported_seq":11345,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.145432+0000","last_change":"2024-09-17T07:31:14.189768+0000","last_active":"2024-09-17T07:31:39.145432+0000","last_peered":"2024-09-17T07:31:39.145432+0000","last_clean":"2024-09-17T07:31:39.145432+0000","last_became_active":"2024-09-17T07:28:43.239409+0000","last_became_peered":"2024-09-17T07:28:43.239409+0000","last_unstale":"2024-09-17T07:31:39.145432+0000","last_undegraded":"2024-09-17T07:31:39.145432+0000","last_fullsized":"2024-09-17T07:31:39.145432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":825,"log_dups_size":0,"ondisk_log_size":825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:13.253211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012368310000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":12617,"num_read_kb":91706,"num_write":1240,"num_write_kb":58366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":365,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.380263+0000","last_change":"2024-09-17T07:30:15.201999+0000","last_active":"2024-09-17T07:31:38.380263+0000","last_peered":"2024-09-17T07:31:38.380263+0000","last_clean":"2024-09-17T07:31:38.380263+0000","last_became_active":"2024-09-17T07:30:15.201642+0000","last_became_peered":"2024-09-17T07:30:15.201642+0000","last_unstale":"2024-09-17T07:31:38.380263+0000","last_undegraded":"2024-09-17T07:31:38.380263+0000","last_fullsized":"2024-09-17T07:31:38.380263+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:47.871565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"164'427","reported_seq":5378,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.130174+0000","last_change":"2024-09-17T07:31:29.105514+0000","last_active":"2024-09-17T07:31:39.130174+0000","last_peered":"2024-09-17T07:31:39.130174+0000","last_clean":"2024-09-17T07:31:39.130174+0000","last_became_active":"2024-09-17T07:31:29.103782+0000","last_became_peered":"2024-09-17T07:31:29.103782+0000","last_unstale":"2024-09-17T07:31:39.130174+0000","last_undegraded":"2024-09-17T07:31:39.130174+0000","last_fullsized":"2024-09-17T07:31:39.130174+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:52.018922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094238099999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3788,"num_read_kb":51682,"num_write":424,"num_write_kb":28012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":37615005,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":364,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.683493+0000","last_change":"2024-09-17T07:31:01.231085+0000","last_active":"2024-09-17T07:31:39.683493+0000","last_peered":"2024-09-17T07:31:39.683493+0000","last_clean":"2024-09-17T07:31:39.683493+0000","last_became_active":"2024-09-17T07:31:01.230794+0000","last_became_peered":"2024-09-17T07:31:01.230794+0000","last_unstale":"2024-09-17T07:31:39.683493+0000","last_undegraded":"2024-09-17T07:31:39.683493+0000","last_fullsized":"2024-09-17T07:31:39.683493+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:25.321726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"165'405","reported_seq":2497,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214012+0000","last_change":"2024-09-17T07:31:14.114890+0000","last_active":"2024-09-17T07:31:38.214012+0000","last_peered":"2024-09-17T07:31:38.214012+0000","last_clean":"2024-09-17T07:31:38.214012+0000","last_became_active":"2024-09-17T07:30:18.084247+0000","last_became_peered":"2024-09-17T07:30:18.084247+0000","last_unstale":"2024-09-17T07:31:38.214012+0000","last_undegraded":"2024-09-17T07:31:38.214012+0000","last_fullsized":"2024-09-17T07:31:38.214012+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:54:33.014267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002225922,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2958,"num_read_kb":9261,"num_write":364,"num_write_kb":6317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":6805602,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":140,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.130225+0000","last_change":"2024-09-17T07:30:34.116563+0000","last_active":"2024-09-17T07:31:39.130225+0000","last_peered":"2024-09-17T07:31:39.130225+0000","last_clean":"2024-09-17T07:31:39.130225+0000","last_became_active":"2024-09-17T07:30:34.113598+0000","last_became_peered":"2024-09-17T07:30:34.113598+0000","last_unstale":"2024-09-17T07:31:39.130225+0000","last_undegraded":"2024-09-17T07:31:39.130225+0000","last_fullsized":"2024-09-17T07:31:39.130225+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:30.341349+0000","last_clean_scrub_stamp":"2024-09-17T07:28:30.341349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:52.003957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":81,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.683423+0000","last_change":"2024-09-17T07:31:05.102195+0000","last_active":"2024-09-17T07:31:39.683423+0000","last_peered":"2024-09-17T07:31:39.683423+0000","last_clean":"2024-09-17T07:31:39.683423+0000","last_became_active":"2024-09-17T07:31:05.096988+0000","last_became_peered":"2024-09-17T07:31:05.096988+0000","last_unstale":"2024-09-17T07:31:39.683423+0000","last_undegraded":"2024-09-17T07:31:39.683423+0000","last_fullsized":"2024-09-17T07:31:39.683423+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:25.321032+0000","last_clean_scrub_stamp":"2024-09-17T07:28:25.321032+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:01.112355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"164'520","reported_seq":2435,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214008+0000","last_change":"2024-09-17T07:31:14.113864+0000","last_active":"2024-09-17T07:31:38.214008+0000","last_peered":"2024-09-17T07:31:38.214008+0000","last_clean":"2024-09-17T07:31:38.214008+0000","last_became_active":"2024-09-17T07:30:19.086777+0000","last_became_peered":"2024-09-17T07:30:19.086777+0000","last_unstale":"2024-09-17T07:31:38.214008+0000","last_undegraded":"2024-09-17T07:31:38.214008+0000","last_fullsized":"2024-09-17T07:31:38.214008+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:55.561658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014795710000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1640,"num_read_kb":24382,"num_write":527,"num_write_kb":33950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":24920122,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"164'551","reported_seq":3919,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.683803+0000","last_change":"2024-09-17T07:31:27.105903+0000","last_active":"2024-09-17T07:31:39.683803+0000","last_peered":"2024-09-17T07:31:39.683803+0000","last_clean":"2024-09-17T07:31:39.683803+0000","last_became_active":"2024-09-17T07:31:27.103978+0000","last_became_peered":"2024-09-17T07:31:27.103978+0000","last_unstale":"2024-09-17T07:31:39.683803+0000","last_undegraded":"2024-09-17T07:31:39.683803+0000","last_fullsized":"2024-09-17T07:31:39.683803+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:13.427259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001973519,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4515,"num_read_kb":21074,"num_write":530,"num_write_kb":12196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":11434290,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"164'739","reported_seq":9612,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216065+0000","last_change":"2024-09-17T07:31:28.103185+0000","last_active":"2024-09-17T07:31:38.216065+0000","last_peered":"2024-09-17T07:31:38.216065+0000","last_clean":"2024-09-17T07:31:38.216065+0000","last_became_active":"2024-09-17T07:31:28.101926+0000","last_became_peered":"2024-09-17T07:31:28.101926+0000","last_unstale":"2024-09-17T07:31:38.216065+0000","last_undegraded":"2024-09-17T07:31:38.216065+0000","last_fullsized":"2024-09-17T07:31:38.216065+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:17:26.947693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000464964,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":26,"num_read":9368,"num_read_kb":52923,"num_write":688,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":604,"num_bytes_recovered":24254406,"num_keys_recovered":93,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"163'559","reported_seq":2911,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.214086+0000","last_change":"2024-09-17T07:31:14.114917+0000","last_active":"2024-09-17T07:31:38.214086+0000","last_peered":"2024-09-17T07:31:38.214086+0000","last_clean":"2024-09-17T07:31:38.214086+0000","last_became_active":"2024-09-17T07:30:19.087515+0000","last_became_peered":"2024-09-17T07:30:19.087515+0000","last_unstale":"2024-09-17T07:31:38.214086+0000","last_undegraded":"2024-09-17T07:31:38.214086+0000","last_fullsized":"2024-09-17T07:31:38.214086+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:42.469193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023177290000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2408,"num_read_kb":29904,"num_write":665,"num_write_kb":15633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":11178824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"165'873","reported_seq":5432,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.215146+0000","last_change":"2024-09-17T07:31:14.107447+0000","last_active":"2024-09-17T07:31:38.215146+0000","last_peered":"2024-09-17T07:31:38.215146+0000","last_clean":"2024-09-17T07:31:38.215146+0000","last_became_active":"2024-09-17T07:30:19.085668+0000","last_became_peered":"2024-09-17T07:30:19.085668+0000","last_unstale":"2024-09-17T07:31:38.215146+0000","last_undegraded":"2024-09-17T07:31:38.215146+0000","last_fullsized":"2024-09-17T07:31:38.215146+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:20:47.391518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001368193,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6070,"num_read_kb":27721,"num_write":1113,"num_write_kb":14827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":15619889,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"165'725","reported_seq":4614,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.683373+0000","last_change":"2024-09-17T07:31:14.176026+0000","last_active":"2024-09-17T07:31:39.683373+0000","last_peered":"2024-09-17T07:31:39.683373+0000","last_clean":"2024-09-17T07:31:39.683373+0000","last_became_active":"2024-09-17T07:28:43.080816+0000","last_became_peered":"2024-09-17T07:28:43.080816+0000","last_unstale":"2024-09-17T07:31:39.683373+0000","last_undegraded":"2024-09-17T07:31:39.683373+0000","last_fullsized":"2024-09-17T07:31:39.683373+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:05:06.879033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017783269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6005,"num_read_kb":48957,"num_write":832,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"164'458","reported_seq":8427,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.683557+0000","last_change":"2024-09-17T07:31:26.155667+0000","last_active":"2024-09-17T07:31:39.683557+0000","last_peered":"2024-09-17T07:31:39.683557+0000","last_clean":"2024-09-17T07:31:39.683557+0000","last_became_active":"2024-09-17T07:31:26.154754+0000","last_became_peered":"2024-09-17T07:31:26.154754+0000","last_unstale":"2024-09-17T07:31:39.683557+0000","last_undegraded":"2024-09-17T07:31:39.683557+0000","last_fullsized":"2024-09-17T07:31:39.683557+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:47.550445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021346680000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7087,"num_read_kb":55690,"num_write":392,"num_write_kb":41376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":13797349,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"163'832","reported_seq":6340,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.683997+0000","last_change":"2024-09-17T07:31:28.103546+0000","last_active":"2024-09-17T07:31:39.683997+0000","last_peered":"2024-09-17T07:31:39.683997+0000","last_clean":"2024-09-17T07:31:39.683997+0000","last_became_active":"2024-09-17T07:31:28.101799+0000","last_became_peered":"2024-09-17T07:31:28.101799+0000","last_unstale":"2024-09-17T07:31:39.683997+0000","last_undegraded":"2024-09-17T07:31:39.683997+0000","last_fullsized":"2024-09-17T07:31:39.683997+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:28.873325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021103049999999998,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6418,"num_read_kb":23171,"num_write":923,"num_write_kb":18541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":744,"num_bytes_recovered":25525458,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"164'401","reported_seq":2768,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:38.216051+0000","last_change":"2024-09-17T07:31:14.720102+0000","last_active":"2024-09-17T07:31:38.216051+0000","last_peered":"2024-09-17T07:31:38.216051+0000","last_clean":"2024-09-17T07:31:38.216051+0000","last_became_active":"2024-09-17T07:28:43.077360+0000","last_became_peered":"2024-09-17T07:28:43.077360+0000","last_unstale":"2024-09-17T07:31:38.216051+0000","last_undegraded":"2024-09-17T07:31:38.216051+0000","last_fullsized":"2024-09-17T07:31:38.216051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:17.256040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061413699999999995,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2841,"num_read_kb":26198,"num_write":450,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"164'802","reported_seq":4339,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.683617+0000","last_change":"2024-09-17T07:31:29.105135+0000","last_active":"2024-09-17T07:31:39.683617+0000","last_peered":"2024-09-17T07:31:39.683617+0000","last_clean":"2024-09-17T07:31:39.683617+0000","last_became_active":"2024-09-17T07:31:29.103626+0000","last_became_peered":"2024-09-17T07:31:29.103626+0000","last_unstale":"2024-09-17T07:31:39.683617+0000","last_undegraded":"2024-09-17T07:31:39.683617+0000","last_fullsized":"2024-09-17T07:31:39.683617+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:10.689986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001604722,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5310,"num_read_kb":45214,"num_write":937,"num_write_kb":26421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":468,"num_bytes_recovered":31676674,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"164'435","reported_seq":2747,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.130102+0000","last_change":"2024-09-17T07:31:14.104829+0000","last_active":"2024-09-17T07:31:39.130102+0000","last_peered":"2024-09-17T07:31:39.130102+0000","last_clean":"2024-09-17T07:31:39.130102+0000","last_became_active":"2024-09-17T07:31:08.096240+0000","last_became_peered":"2024-09-17T07:31:08.096240+0000","last_unstale":"2024-09-17T07:31:39.130102+0000","last_undegraded":"2024-09-17T07:31:39.130102+0000","last_fullsized":"2024-09-17T07:31:39.130102+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:34.448388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098890999999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2231,"num_read_kb":44562,"num_write":412,"num_write_kb":29148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":24460350,"num_keys_recovered":37,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"164'772","reported_seq":4242,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:39.683703+0000","last_change":"2024-09-17T07:31:29.104804+0000","last_active":"2024-09-17T07:31:39.683703+0000","last_peered":"2024-09-17T07:31:39.683703+0000","last_clean":"2024-09-17T07:31:39.683703+0000","last_became_active":"2024-09-17T07:31:29.103168+0000","last_became_peered":"2024-09-17T07:31:29.103168+0000","last_unstale":"2024-09-17T07:31:39.683703+0000","last_undegraded":"2024-09-17T07:31:39.683703+0000","last_fullsized":"2024-09-17T07:31:39.683703+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:28:42.059932+0000","last_clean_scrub_stamp":"2024-09-17T07:28:42.059932+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:31.292655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093421499999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4511,"num_read_kb":28124,"num_write":1074,"num_write_kb":16615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":21539718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":607,"num_read":166048,"num_read_kb":1208202,"num_write":23179,"num_write_kb":792760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9827,"num_bytes_recovered":666374238,"num_keys_recovered":891,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7902,"internal_metadata":0},"log_size":20553,"ondisk_log_size":20553,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":101032,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47340,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324500,"statfs":{"total":96636764160,"available":96588288000,"internally_reserved":0,"allocated":2314240,"data_stored":1685451,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33412,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338428,"statfs":{"total":96636764160,"available":96602550272,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55616,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316224,"statfs":{"total":96636764160,"available":96579813376,"internally_reserved":0,"allocated":2269184,"data_stored":1095102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91644,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280196,"statfs":{"total":96636764160,"available":96542920704,"internally_reserved":0,"allocated":2269184,"data_stored":1095102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056663,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43268,"kb_used_data":2212,"kb_used_omap":16,"kb_used_meta":41007,"kb_avail":94328572,"statfs":{"total":96636764160,"available":96592457728,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17058,"internal_metadata":41991518},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542187,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55612,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316228,"statfs":{"total":96636764160,"available":96579817472,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65344,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306496,"statfs":{"total":96636764160,"available":96569851904,"internally_reserved":0,"allocated":2273280,"data_stored":1095121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47356,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324484,"statfs":{"total":96636764160,"available":96588271616,"internally_reserved":0,"allocated":2330624,"data_stored":1685451,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7902,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:31:48.694 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.694 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.694 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.694 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.694 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.694 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.694 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.694 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.694 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.695 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.695 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.695 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.695 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-17T07:28:30.341349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=30, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.695 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.695 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-17T07:28:30.341349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=30, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.695 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.695 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-17T07:28:30.341349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=30, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.695 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.696 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-17T07:28:30.341349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=30, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.696 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.696 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.696 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-17T07:28:30.341349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=30, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.696 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.696 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-17T07:28:30.341349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=30, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.696 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.696 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-17T07:28:30.341349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=30, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.697 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.697 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-17T07:28:30.341349+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=30, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.697 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T07:28:25.321032+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=25, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.697 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.697 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.697 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.697 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.697 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.697 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.698 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.698 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.698 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.698 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.698 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.698 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-17T07:28:42.059932+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=28, tm_sec=42, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=7, tm_min=31, tm_sec=43, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T07:31:48.698 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T07:32:08.700 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T07:32:09.015 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-17T07:32:09.016 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-17T07:32:09.027 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-17T07:32:08.794136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":607,"num_read":166114,"num_read_kb":1208257,"num_write":23282,"num_write_kb":794390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9832,"num_bytes_recovered":667554974,"num_keys_recovered":891,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21243,"ondisk_log_size":21243,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":443680,"kb_used_data":17824,"kb_used_omap":27,"kb_used_meta":425700,"kb_avail":754531040,"statfs":{"total":773094113280,"available":772639784960,"internally_reserved":0,"allocated":18251776,"data_stored":9941476,"data_compressed":66815,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":28217,"internal_metadata":435917255},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002194"},"pg_stats":[{"pgid":"3.1f","version":"188'892","reported_seq":4645,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:52.111664+0000","last_change":"2024-09-17T07:31:52.111664+0000","last_active":"2024-09-17T07:31:52.111664+0000","last_peered":"2024-09-17T07:31:52.111664+0000","last_clean":"2024-09-17T07:31:52.111664+0000","last_became_active":"2024-09-17T07:30:43.092456+0000","last_became_peered":"2024-09-17T07:30:43.092456+0000","last_unstale":"2024-09-17T07:31:52.111664+0000","last_undegraded":"2024-09-17T07:31:52.111664+0000","last_fullsized":"2024-09-17T07:31:52.111664+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'892","last_scrub_stamp":"2024-09-17T07:31:52.111547+0000","last_deep_scrub":"188'892","last_deep_scrub_stamp":"2024-09-17T07:31:52.111547+0000","last_clean_scrub_stamp":"2024-09-17T07:31:52.111547+0000","objects_scrubbed":18,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:54.730520+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.0016105900000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5801,"num_read_kb":34310,"num_write":997,"num_write_kb":41977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":26159662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"188'587","reported_seq":3328,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:51.956415+0000","last_change":"2024-09-17T07:31:51.956415+0000","last_active":"2024-09-17T07:31:51.956415+0000","last_peered":"2024-09-17T07:31:51.956415+0000","last_clean":"2024-09-17T07:31:51.956415+0000","last_became_active":"2024-09-17T07:30:19.087213+0000","last_became_peered":"2024-09-17T07:30:19.087213+0000","last_unstale":"2024-09-17T07:31:51.956415+0000","last_undegraded":"2024-09-17T07:31:51.956415+0000","last_fullsized":"2024-09-17T07:31:51.956415+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'587","last_scrub_stamp":"2024-09-17T07:31:51.956301+0000","last_deep_scrub":"188'587","last_deep_scrub_stamp":"2024-09-17T07:31:51.956301+0000","last_clean_scrub_stamp":"2024-09-17T07:31:51.956301+0000","objects_scrubbed":34,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:44:26.842002+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0.0029487620000000002,"stat_sum":{"num_bytes":0,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":34,"num_read":3721,"num_read_kb":30298,"num_write":527,"num_write_kb":15594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":15743718,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"188'578","reported_seq":3535,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:51.970445+0000","last_change":"2024-09-17T07:31:51.970445+0000","last_active":"2024-09-17T07:31:51.970445+0000","last_peered":"2024-09-17T07:31:51.970445+0000","last_clean":"2024-09-17T07:31:51.970445+0000","last_became_active":"2024-09-17T07:31:26.149428+0000","last_became_peered":"2024-09-17T07:31:26.149428+0000","last_unstale":"2024-09-17T07:31:51.970445+0000","last_undegraded":"2024-09-17T07:31:51.970445+0000","last_fullsized":"2024-09-17T07:31:51.970445+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'578","last_scrub_stamp":"2024-09-17T07:31:51.970363+0000","last_deep_scrub":"188'578","last_deep_scrub_stamp":"2024-09-17T07:31:51.970363+0000","last_clean_scrub_stamp":"2024-09-17T07:31:51.970363+0000","objects_scrubbed":28,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:53:23.532229+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0.0016933180000000001,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":1948,"num_read_kb":28759,"num_write":579,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":27967854,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"188'827","reported_seq":6650,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:53.951630+0000","last_change":"2024-09-17T07:31:53.951630+0000","last_active":"2024-09-17T07:31:53.951630+0000","last_peered":"2024-09-17T07:31:53.951630+0000","last_clean":"2024-09-17T07:31:53.951630+0000","last_became_active":"2024-09-17T07:31:28.100048+0000","last_became_peered":"2024-09-17T07:31:28.100048+0000","last_unstale":"2024-09-17T07:31:53.951630+0000","last_undegraded":"2024-09-17T07:31:53.951630+0000","last_fullsized":"2024-09-17T07:31:53.951630+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'827","last_scrub_stamp":"2024-09-17T07:31:53.951535+0000","last_deep_scrub":"188'827","last_deep_scrub_stamp":"2024-09-17T07:31:53.951535+0000","last_clean_scrub_stamp":"2024-09-17T07:31:53.951535+0000","objects_scrubbed":14,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:16:44.877422+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0010655949999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":7523,"num_read_kb":42454,"num_write":1061,"num_write_kb":38721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":26661650,"num_keys_recovered":180,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"188'619","reported_seq":5662,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:51.138972+0000","last_change":"2024-09-17T07:31:51.138972+0000","last_active":"2024-09-17T07:31:51.138972+0000","last_peered":"2024-09-17T07:31:51.138972+0000","last_clean":"2024-09-17T07:31:51.138972+0000","last_became_active":"2024-09-17T07:31:28.102058+0000","last_became_peered":"2024-09-17T07:31:28.102058+0000","last_unstale":"2024-09-17T07:31:51.138972+0000","last_undegraded":"2024-09-17T07:31:51.138972+0000","last_fullsized":"2024-09-17T07:31:51.138972+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'619","last_scrub_stamp":"2024-09-17T07:31:51.138866+0000","last_deep_scrub":"188'619","last_deep_scrub_stamp":"2024-09-17T07:31:51.138866+0000","last_clean_scrub_stamp":"2024-09-17T07:31:51.138866+0000","objects_scrubbed":13,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:21.557758+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0028238529999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5800,"num_read_kb":52463,"num_write":596,"num_write_kb":22879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":922,"num_bytes_recovered":91784745,"num_keys_recovered":126,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"188'939","reported_seq":14923,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:48.365954+0000","last_change":"2024-09-17T07:31:48.365954+0000","last_active":"2024-09-17T07:31:48.365954+0000","last_peered":"2024-09-17T07:31:48.365954+0000","last_clean":"2024-09-17T07:31:48.365954+0000","last_became_active":"2024-09-17T07:31:27.103933+0000","last_became_peered":"2024-09-17T07:31:27.103933+0000","last_unstale":"2024-09-17T07:31:48.365954+0000","last_undegraded":"2024-09-17T07:31:48.365954+0000","last_fullsized":"2024-09-17T07:31:48.365954+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'939","last_scrub_stamp":"2024-09-17T07:31:48.365848+0000","last_deep_scrub":"188'939","last_deep_scrub_stamp":"2024-09-17T07:31:48.365848+0000","last_clean_scrub_stamp":"2024-09-17T07:31:48.365848+0000","objects_scrubbed":19,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:33.547666+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0096620079999999997,"stat_sum":{"num_bytes":19,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":16715,"num_read_kb":60981,"num_write":1188,"num_write_kb":20597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":19562028,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"188'467","reported_seq":5722,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:48.581140+0000","last_change":"2024-09-17T07:31:48.581140+0000","last_active":"2024-09-17T07:31:48.581140+0000","last_peered":"2024-09-17T07:31:48.581140+0000","last_clean":"2024-09-17T07:31:48.581140+0000","last_became_active":"2024-09-17T07:30:18.083918+0000","last_became_peered":"2024-09-17T07:30:18.083918+0000","last_unstale":"2024-09-17T07:31:48.581140+0000","last_undegraded":"2024-09-17T07:31:48.581140+0000","last_fullsized":"2024-09-17T07:31:48.581140+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'467","last_scrub_stamp":"2024-09-17T07:31:48.581070+0000","last_deep_scrub":"188'467","last_deep_scrub_stamp":"2024-09-17T07:31:48.581070+0000","last_clean_scrub_stamp":"2024-09-17T07:31:48.581070+0000","objects_scrubbed":28,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:14:35.040290+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.000820972,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":4352,"num_read_kb":37354,"num_write":400,"num_write_kb":43089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":25536146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"188'782","reported_seq":4696,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:50.943898+0000","last_change":"2024-09-17T07:31:50.943898+0000","last_active":"2024-09-17T07:31:50.943898+0000","last_peered":"2024-09-17T07:31:50.943898+0000","last_clean":"2024-09-17T07:31:50.943898+0000","last_became_active":"2024-09-17T07:30:43.091692+0000","last_became_peered":"2024-09-17T07:30:43.091692+0000","last_unstale":"2024-09-17T07:31:50.943898+0000","last_undegraded":"2024-09-17T07:31:50.943898+0000","last_fullsized":"2024-09-17T07:31:50.943898+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'782","last_scrub_stamp":"2024-09-17T07:31:50.943767+0000","last_deep_scrub":"188'782","last_deep_scrub_stamp":"2024-09-17T07:31:50.943767+0000","last_clean_scrub_stamp":"2024-09-17T07:31:50.943767+0000","objects_scrubbed":19,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:56.405673+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0014749730000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5969,"num_read_kb":25327,"num_write":792,"num_write_kb":16603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":29456184,"num_keys_recovered":176,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"188'461","reported_seq":4262,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:49.094730+0000","last_change":"2024-09-17T07:31:49.094730+0000","last_active":"2024-09-17T07:31:49.094730+0000","last_peered":"2024-09-17T07:31:49.094730+0000","last_clean":"2024-09-17T07:31:49.094730+0000","last_became_active":"2024-09-17T07:28:43.080428+0000","last_became_peered":"2024-09-17T07:28:43.080428+0000","last_unstale":"2024-09-17T07:31:49.094730+0000","last_undegraded":"2024-09-17T07:31:49.094730+0000","last_fullsized":"2024-09-17T07:31:49.094730+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'461","last_scrub_stamp":"2024-09-17T07:31:49.094656+0000","last_deep_scrub":"188'461","last_deep_scrub_stamp":"2024-09-17T07:31:49.094656+0000","last_clean_scrub_stamp":"2024-09-17T07:31:49.094656+0000","objects_scrubbed":25,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:47.990055+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.0020783120000000001,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4961,"num_read_kb":22717,"num_write":365,"num_write_kb":13531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"188'531","reported_seq":4217,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:52.920836+0000","last_change":"2024-09-17T07:31:52.920836+0000","last_active":"2024-09-17T07:31:52.920836+0000","last_peered":"2024-09-17T07:31:52.920836+0000","last_clean":"2024-09-17T07:31:52.920836+0000","last_became_active":"2024-09-17T07:28:43.082313+0000","last_became_peered":"2024-09-17T07:28:43.082313+0000","last_unstale":"2024-09-17T07:31:52.920836+0000","last_undegraded":"2024-09-17T07:31:52.920836+0000","last_fullsized":"2024-09-17T07:31:52.920836+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'531","last_scrub_stamp":"2024-09-17T07:31:52.920704+0000","last_deep_scrub":"188'531","last_deep_scrub_stamp":"2024-09-17T07:31:52.920704+0000","last_clean_scrub_stamp":"2024-09-17T07:31:52.920704+0000","objects_scrubbed":18,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:12.438698+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.001045261,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":6140,"num_read_kb":58359,"num_write":449,"num_write_kb":31855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"188'404","reported_seq":4528,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:56.805441+0000","last_change":"2024-09-17T07:31:56.805441+0000","last_active":"2024-09-17T07:31:56.805441+0000","last_peered":"2024-09-17T07:31:56.805441+0000","last_clean":"2024-09-17T07:31:56.805441+0000","last_became_active":"2024-09-17T07:31:08.094940+0000","last_became_peered":"2024-09-17T07:31:08.094940+0000","last_unstale":"2024-09-17T07:31:56.805441+0000","last_undegraded":"2024-09-17T07:31:56.805441+0000","last_fullsized":"2024-09-17T07:31:56.805441+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'404","last_scrub_stamp":"2024-09-17T07:31:56.805343+0000","last_deep_scrub":"188'404","last_deep_scrub_stamp":"2024-09-17T07:31:56.805343+0000","last_clean_scrub_stamp":"2024-09-17T07:31:56.805343+0000","objects_scrubbed":14,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:01.303488+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0011121970000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3586,"num_read_kb":32352,"num_write":371,"num_write_kb":22781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":431,"num_bytes_recovered":60567103,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"188'481","reported_seq":2130,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:47.349695+0000","last_change":"2024-09-17T07:31:47.349695+0000","last_active":"2024-09-17T07:31:47.349695+0000","last_peered":"2024-09-17T07:31:47.349695+0000","last_clean":"2024-09-17T07:31:47.349695+0000","last_became_active":"2024-09-17T07:31:26.152423+0000","last_became_peered":"2024-09-17T07:31:26.152423+0000","last_unstale":"2024-09-17T07:31:47.349695+0000","last_undegraded":"2024-09-17T07:31:47.349695+0000","last_fullsized":"2024-09-17T07:31:47.349695+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'481","last_scrub_stamp":"2024-09-17T07:31:47.349589+0000","last_deep_scrub":"188'481","last_deep_scrub_stamp":"2024-09-17T07:31:47.349589+0000","last_clean_scrub_stamp":"2024-09-17T07:31:47.349589+0000","objects_scrubbed":16,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:50:51.754393+0000","scrub_duration":12,"objects_trimmed":2,"snaptrim_duration":0.015069525,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1414,"num_read_kb":17974,"num_write":482,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":16518806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":371,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:50.859283+0000","last_change":"2024-09-17T07:31:50.859283+0000","last_active":"2024-09-17T07:31:50.859283+0000","last_peered":"2024-09-17T07:31:50.859283+0000","last_clean":"2024-09-17T07:31:50.859283+0000","last_became_active":"2024-09-17T07:31:05.097821+0000","last_became_peered":"2024-09-17T07:31:05.097821+0000","last_unstale":"2024-09-17T07:31:50.859283+0000","last_undegraded":"2024-09-17T07:31:50.859283+0000","last_fullsized":"2024-09-17T07:31:50.859283+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:31:50.859189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:31:50.859189+0000","last_clean_scrub_stamp":"2024-09-17T07:31:50.859189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:39.147911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474377,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"188'1048","reported_seq":2789,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:47.071825+0000","last_change":"2024-09-17T07:31:47.071825+0000","last_active":"2024-09-17T07:31:47.071825+0000","last_peered":"2024-09-17T07:31:47.071825+0000","last_clean":"2024-09-17T07:31:47.071825+0000","last_became_active":"2024-09-17T07:28:43.079229+0000","last_became_peered":"2024-09-17T07:28:43.079229+0000","last_unstale":"2024-09-17T07:31:47.071825+0000","last_undegraded":"2024-09-17T07:31:47.071825+0000","last_fullsized":"2024-09-17T07:31:47.071825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'1048","last_scrub_stamp":"2024-09-17T07:31:47.071722+0000","last_deep_scrub":"188'1048","last_deep_scrub_stamp":"2024-09-17T07:31:47.071722+0000","last_clean_scrub_stamp":"2024-09-17T07:31:47.071722+0000","objects_scrubbed":18,"log_size":1048,"log_dups_size":0,"ondisk_log_size":1048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:20.993384+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0015639079999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4625,"num_read_kb":48744,"num_write":1479,"num_write_kb":29297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":91,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:46.634771+0000","last_change":"2024-09-17T07:31:46.634771+0000","last_active":"2024-09-17T07:31:46.634771+0000","last_peered":"2024-09-17T07:31:46.634771+0000","last_clean":"2024-09-17T07:31:46.634771+0000","last_became_active":"2024-09-17T07:31:01.268257+0000","last_became_peered":"2024-09-17T07:31:01.268257+0000","last_unstale":"2024-09-17T07:31:46.634771+0000","last_undegraded":"2024-09-17T07:31:46.634771+0000","last_fullsized":"2024-09-17T07:31:46.634771+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:31:46.634687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:31:46.634687+0000","last_clean_scrub_stamp":"2024-09-17T07:31:46.634687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:21.735566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"188'597","reported_seq":3729,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:47.866609+0000","last_change":"2024-09-17T07:31:47.866609+0000","last_active":"2024-09-17T07:31:47.866609+0000","last_peered":"2024-09-17T07:31:47.866609+0000","last_clean":"2024-09-17T07:31:47.866609+0000","last_became_active":"2024-09-17T07:30:19.087787+0000","last_became_peered":"2024-09-17T07:30:19.087787+0000","last_unstale":"2024-09-17T07:31:47.866609+0000","last_undegraded":"2024-09-17T07:31:47.866609+0000","last_fullsized":"2024-09-17T07:31:47.866609+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'597","last_scrub_stamp":"2024-09-17T07:31:47.866537+0000","last_deep_scrub":"188'597","last_deep_scrub_stamp":"2024-09-17T07:31:47.866537+0000","last_clean_scrub_stamp":"2024-09-17T07:31:47.866537+0000","objects_scrubbed":20,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:52.106659+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0029713869999999998,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":3995,"num_read_kb":36995,"num_write":614,"num_write_kb":22622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":229,"num_bytes_recovered":15169480,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":360,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:48.854126+0000","last_change":"2024-09-17T07:31:48.854126+0000","last_active":"2024-09-17T07:31:48.854126+0000","last_peered":"2024-09-17T07:31:48.854126+0000","last_clean":"2024-09-17T07:31:48.854126+0000","last_became_active":"2024-09-17T07:28:31.567409+0000","last_became_peered":"2024-09-17T07:28:31.567409+0000","last_unstale":"2024-09-17T07:31:48.854126+0000","last_undegraded":"2024-09-17T07:31:48.854126+0000","last_fullsized":"2024-09-17T07:31:48.854126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:31:48.853987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:31:48.853987+0000","last_clean_scrub_stamp":"2024-09-17T07:31:48.853987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:25.953551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"188'1080","reported_seq":3628,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:47.623620+0000","last_change":"2024-09-17T07:31:47.623620+0000","last_active":"2024-09-17T07:31:47.623620+0000","last_peered":"2024-09-17T07:31:47.623620+0000","last_clean":"2024-09-17T07:31:47.623620+0000","last_became_active":"2024-09-17T07:28:43.239387+0000","last_became_peered":"2024-09-17T07:28:43.239387+0000","last_unstale":"2024-09-17T07:31:47.623620+0000","last_undegraded":"2024-09-17T07:31:47.623620+0000","last_fullsized":"2024-09-17T07:31:47.623620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'1080","last_scrub_stamp":"2024-09-17T07:31:47.623532+0000","last_deep_scrub":"188'1080","last_deep_scrub_stamp":"2024-09-17T07:31:47.623532+0000","last_clean_scrub_stamp":"2024-09-17T07:31:47.623532+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:43.615617+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00080983699999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6807,"num_read_kb":16873,"num_write":1444,"num_write_kb":10814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":183,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:49.019078+0000","last_change":"2024-09-17T07:31:49.019078+0000","last_active":"2024-09-17T07:31:49.019078+0000","last_peered":"2024-09-17T07:31:49.019078+0000","last_clean":"2024-09-17T07:31:49.019078+0000","last_became_active":"2024-09-17T07:30:15.200470+0000","last_became_peered":"2024-09-17T07:30:15.200470+0000","last_unstale":"2024-09-17T07:31:49.019078+0000","last_undegraded":"2024-09-17T07:31:49.019078+0000","last_fullsized":"2024-09-17T07:31:49.019078+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:31:49.019003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:31:49.019003+0000","last_clean_scrub_stamp":"2024-09-17T07:31:49.019003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:29:07.030467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"188'591","reported_seq":3333,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:48.074338+0000","last_change":"2024-09-17T07:31:48.074338+0000","last_active":"2024-09-17T07:31:48.074338+0000","last_peered":"2024-09-17T07:31:48.074338+0000","last_clean":"2024-09-17T07:31:48.074338+0000","last_became_active":"2024-09-17T07:30:19.085184+0000","last_became_peered":"2024-09-17T07:30:19.085184+0000","last_unstale":"2024-09-17T07:31:48.074338+0000","last_undegraded":"2024-09-17T07:31:48.074338+0000","last_fullsized":"2024-09-17T07:31:48.074338+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'591","last_scrub_stamp":"2024-09-17T07:31:48.074224+0000","last_deep_scrub":"188'591","last_deep_scrub_stamp":"2024-09-17T07:31:48.074224+0000","last_clean_scrub_stamp":"2024-09-17T07:31:48.074224+0000","objects_scrubbed":22,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:10.613360+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.0015649349999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2687,"num_read_kb":46297,"num_write":632,"num_write_kb":27531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":35691417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"188'694","reported_seq":3195,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:49.339516+0000","last_change":"2024-09-17T07:31:49.339516+0000","last_active":"2024-09-17T07:31:49.339516+0000","last_peered":"2024-09-17T07:31:49.339516+0000","last_clean":"2024-09-17T07:31:49.339516+0000","last_became_active":"2024-09-17T07:31:27.101204+0000","last_became_peered":"2024-09-17T07:31:27.101204+0000","last_unstale":"2024-09-17T07:31:49.339516+0000","last_undegraded":"2024-09-17T07:31:49.339516+0000","last_fullsized":"2024-09-17T07:31:49.339516+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'694","last_scrub_stamp":"2024-09-17T07:31:49.339388+0000","last_deep_scrub":"188'694","last_deep_scrub_stamp":"2024-09-17T07:31:49.339388+0000","last_clean_scrub_stamp":"2024-09-17T07:31:49.339388+0000","objects_scrubbed":26,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:18.886619+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0.0028913630000000001,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":2237,"num_read_kb":35376,"num_write":632,"num_write_kb":16684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":26727758,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"188'3","reported_seq":364,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:49.972356+0000","last_change":"2024-09-17T07:31:49.972356+0000","last_active":"2024-09-17T07:31:49.972356+0000","last_peered":"2024-09-17T07:31:49.972356+0000","last_clean":"2024-09-17T07:31:49.972356+0000","last_became_active":"2024-09-17T07:28:31.359669+0000","last_became_peered":"2024-09-17T07:28:31.359669+0000","last_unstale":"2024-09-17T07:31:49.972356+0000","last_undegraded":"2024-09-17T07:31:49.972356+0000","last_fullsized":"2024-09-17T07:31:49.972356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'3","last_scrub_stamp":"2024-09-17T07:31:49.972261+0000","last_deep_scrub":"188'3","last_deep_scrub_stamp":"2024-09-17T07:31:49.972261+0000","last_clean_scrub_stamp":"2024-09-17T07:31:49.972261+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:33:34.120873+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00050909299999999996,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"188'855","reported_seq":11384,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:49.599160+0000","last_change":"2024-09-17T07:31:49.599160+0000","last_active":"2024-09-17T07:31:49.599160+0000","last_peered":"2024-09-17T07:31:49.599160+0000","last_clean":"2024-09-17T07:31:49.599160+0000","last_became_active":"2024-09-17T07:28:43.239409+0000","last_became_peered":"2024-09-17T07:28:43.239409+0000","last_unstale":"2024-09-17T07:31:49.599160+0000","last_undegraded":"2024-09-17T07:31:49.599160+0000","last_fullsized":"2024-09-17T07:31:49.599160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'855","last_scrub_stamp":"2024-09-17T07:31:49.599082+0000","last_deep_scrub":"188'855","last_deep_scrub_stamp":"2024-09-17T07:31:49.599082+0000","last_clean_scrub_stamp":"2024-09-17T07:31:49.599082+0000","objects_scrubbed":30,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:33:58.553066+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0012368310000000001,"stat_sum":{"num_bytes":0,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":30,"num_read":12617,"num_read_kb":91706,"num_write":1240,"num_write_kb":58366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":373,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:46.878148+0000","last_change":"2024-09-17T07:31:46.878148+0000","last_active":"2024-09-17T07:31:46.878148+0000","last_peered":"2024-09-17T07:31:46.878148+0000","last_clean":"2024-09-17T07:31:46.878148+0000","last_became_active":"2024-09-17T07:30:15.201642+0000","last_became_peered":"2024-09-17T07:30:15.201642+0000","last_unstale":"2024-09-17T07:31:46.878148+0000","last_undegraded":"2024-09-17T07:31:46.878148+0000","last_fullsized":"2024-09-17T07:31:46.878148+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:31:46.878075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:31:46.878075+0000","last_clean_scrub_stamp":"2024-09-17T07:31:46.878075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:29:46.830252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029346299999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"188'442","reported_seq":5401,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:50.138971+0000","last_change":"2024-09-17T07:31:50.138971+0000","last_active":"2024-09-17T07:31:50.138971+0000","last_peered":"2024-09-17T07:31:50.138971+0000","last_clean":"2024-09-17T07:31:50.138971+0000","last_became_active":"2024-09-17T07:31:29.103782+0000","last_became_peered":"2024-09-17T07:31:29.103782+0000","last_unstale":"2024-09-17T07:31:50.138971+0000","last_undegraded":"2024-09-17T07:31:50.138971+0000","last_fullsized":"2024-09-17T07:31:50.138971+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'442","last_scrub_stamp":"2024-09-17T07:31:50.138861+0000","last_deep_scrub":"188'442","last_deep_scrub_stamp":"2024-09-17T07:31:50.138861+0000","last_clean_scrub_stamp":"2024-09-17T07:31:50.138861+0000","objects_scrubbed":15,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:36.233841+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00094238099999999995,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3788,"num_read_kb":51682,"num_write":424,"num_write_kb":28012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":37615005,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":372,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:49.816277+0000","last_change":"2024-09-17T07:31:49.816277+0000","last_active":"2024-09-17T07:31:49.816277+0000","last_peered":"2024-09-17T07:31:49.816277+0000","last_clean":"2024-09-17T07:31:49.816277+0000","last_became_active":"2024-09-17T07:31:01.230794+0000","last_became_peered":"2024-09-17T07:31:01.230794+0000","last_unstale":"2024-09-17T07:31:49.816277+0000","last_undegraded":"2024-09-17T07:31:49.816277+0000","last_fullsized":"2024-09-17T07:31:49.816277+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T07:31:49.816162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T07:31:49.816162+0000","last_clean_scrub_stamp":"2024-09-17T07:31:49.816162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:08.533966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075755099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"188'420","reported_seq":2520,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:48.908693+0000","last_change":"2024-09-17T07:31:48.908693+0000","last_active":"2024-09-17T07:31:48.908693+0000","last_peered":"2024-09-17T07:31:48.908693+0000","last_clean":"2024-09-17T07:31:48.908693+0000","last_became_active":"2024-09-17T07:30:18.084247+0000","last_became_peered":"2024-09-17T07:30:18.084247+0000","last_unstale":"2024-09-17T07:31:48.908693+0000","last_undegraded":"2024-09-17T07:31:48.908693+0000","last_fullsized":"2024-09-17T07:31:48.908693+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'420","last_scrub_stamp":"2024-09-17T07:31:48.908622+0000","last_deep_scrub":"188'420","last_deep_scrub_stamp":"2024-09-17T07:31:48.908622+0000","last_clean_scrub_stamp":"2024-09-17T07:31:48.908622+0000","objects_scrubbed":15,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T18:58:29.177343+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.002225922,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2958,"num_read_kb":9261,"num_write":364,"num_write_kb":6317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":6805602,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"188'2","reported_seq":149,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:46.023314+0000","last_change":"2024-09-17T07:31:46.023314+0000","last_active":"2024-09-17T07:31:46.023314+0000","last_peered":"2024-09-17T07:31:46.023314+0000","last_clean":"2024-09-17T07:31:46.023314+0000","last_became_active":"2024-09-17T07:30:34.113598+0000","last_became_peered":"2024-09-17T07:30:34.113598+0000","last_unstale":"2024-09-17T07:31:46.023314+0000","last_undegraded":"2024-09-17T07:31:46.023314+0000","last_fullsized":"2024-09-17T07:31:46.023314+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'2","last_scrub_stamp":"2024-09-17T07:31:46.023222+0000","last_deep_scrub":"188'2","last_deep_scrub_stamp":"2024-09-17T07:31:46.023222+0000","last_clean_scrub_stamp":"2024-09-17T07:31:46.023222+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:50.578443+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"188'74","reported_seq":91,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:50.815705+0000","last_change":"2024-09-17T07:31:50.815705+0000","last_active":"2024-09-17T07:31:50.815705+0000","last_peered":"2024-09-17T07:31:50.815705+0000","last_clean":"2024-09-17T07:31:50.815705+0000","last_became_active":"2024-09-17T07:31:05.096988+0000","last_became_peered":"2024-09-17T07:31:05.096988+0000","last_unstale":"2024-09-17T07:31:50.815705+0000","last_undegraded":"2024-09-17T07:31:50.815705+0000","last_fullsized":"2024-09-17T07:31:50.815705+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'74","last_scrub_stamp":"2024-09-17T07:31:50.815610+0000","last_deep_scrub":"188'74","last_deep_scrub_stamp":"2024-09-17T07:31:50.815610+0000","last_clean_scrub_stamp":"2024-09-17T07:31:50.815610+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:07:42.202744+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"188'536","reported_seq":2459,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:46.853828+0000","last_change":"2024-09-17T07:31:46.853828+0000","last_active":"2024-09-17T07:31:46.853828+0000","last_peered":"2024-09-17T07:31:46.853828+0000","last_clean":"2024-09-17T07:31:46.853828+0000","last_became_active":"2024-09-17T07:30:19.086777+0000","last_became_peered":"2024-09-17T07:30:19.086777+0000","last_unstale":"2024-09-17T07:31:46.853828+0000","last_undegraded":"2024-09-17T07:31:46.853828+0000","last_fullsized":"2024-09-17T07:31:46.853828+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'536","last_scrub_stamp":"2024-09-17T07:31:46.853713+0000","last_deep_scrub":"188'536","last_deep_scrub_stamp":"2024-09-17T07:31:46.853713+0000","last_clean_scrub_stamp":"2024-09-17T07:31:46.853713+0000","objects_scrubbed":16,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:39.608459+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0014795710000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1640,"num_read_kb":24382,"num_write":527,"num_write_kb":33950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":24920122,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"188'565","reported_seq":3941,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:50.373129+0000","last_change":"2024-09-17T07:31:50.373129+0000","last_active":"2024-09-17T07:31:50.373129+0000","last_peered":"2024-09-17T07:31:50.373129+0000","last_clean":"2024-09-17T07:31:50.373129+0000","last_became_active":"2024-09-17T07:31:27.103978+0000","last_became_peered":"2024-09-17T07:31:27.103978+0000","last_unstale":"2024-09-17T07:31:50.373129+0000","last_undegraded":"2024-09-17T07:31:50.373129+0000","last_fullsized":"2024-09-17T07:31:50.373129+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'565","last_scrub_stamp":"2024-09-17T07:31:50.373020+0000","last_deep_scrub":"188'565","last_deep_scrub_stamp":"2024-09-17T07:31:50.373020+0000","last_clean_scrub_stamp":"2024-09-17T07:31:50.373020+0000","objects_scrubbed":14,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:37:25.658290+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.001973519,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4515,"num_read_kb":21074,"num_write":530,"num_write_kb":12196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":11434290,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"188'766","reported_seq":9648,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:48.836322+0000","last_change":"2024-09-17T07:31:48.836322+0000","last_active":"2024-09-17T07:31:48.836322+0000","last_peered":"2024-09-17T07:31:48.836322+0000","last_clean":"2024-09-17T07:31:48.836322+0000","last_became_active":"2024-09-17T07:31:28.101926+0000","last_became_peered":"2024-09-17T07:31:28.101926+0000","last_unstale":"2024-09-17T07:31:48.836322+0000","last_undegraded":"2024-09-17T07:31:48.836322+0000","last_fullsized":"2024-09-17T07:31:48.836322+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'766","last_scrub_stamp":"2024-09-17T07:31:48.836234+0000","last_deep_scrub":"188'766","last_deep_scrub_stamp":"2024-09-17T07:31:48.836234+0000","last_clean_scrub_stamp":"2024-09-17T07:31:48.836234+0000","objects_scrubbed":27,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:26.825648+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.000464964,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":26,"num_read":9368,"num_read_kb":52923,"num_write":688,"num_write_kb":21093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":604,"num_bytes_recovered":24254406,"num_keys_recovered":93,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"188'573","reported_seq":2933,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:49.920456+0000","last_change":"2024-09-17T07:31:49.920456+0000","last_active":"2024-09-17T07:31:49.920456+0000","last_peered":"2024-09-17T07:31:49.920456+0000","last_clean":"2024-09-17T07:31:49.920456+0000","last_became_active":"2024-09-17T07:30:19.087515+0000","last_became_peered":"2024-09-17T07:30:19.087515+0000","last_unstale":"2024-09-17T07:31:49.920456+0000","last_undegraded":"2024-09-17T07:31:49.920456+0000","last_fullsized":"2024-09-17T07:31:49.920456+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'573","last_scrub_stamp":"2024-09-17T07:31:49.920361+0000","last_deep_scrub":"188'573","last_deep_scrub_stamp":"2024-09-17T07:31:49.920361+0000","last_clean_scrub_stamp":"2024-09-17T07:31:49.920361+0000","objects_scrubbed":14,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:50:26.700269+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0023177290000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2408,"num_read_kb":29904,"num_write":665,"num_write_kb":15633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":11178824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"188'890","reported_seq":5457,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:50.940475+0000","last_change":"2024-09-17T07:31:50.940475+0000","last_active":"2024-09-17T07:31:50.940475+0000","last_peered":"2024-09-17T07:31:50.940475+0000","last_clean":"2024-09-17T07:31:50.940475+0000","last_became_active":"2024-09-17T07:30:19.085668+0000","last_became_peered":"2024-09-17T07:30:19.085668+0000","last_unstale":"2024-09-17T07:31:50.940475+0000","last_undegraded":"2024-09-17T07:31:50.940475+0000","last_fullsized":"2024-09-17T07:31:50.940475+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'890","last_scrub_stamp":"2024-09-17T07:31:50.940362+0000","last_deep_scrub":"188'890","last_deep_scrub_stamp":"2024-09-17T07:31:50.940362+0000","last_clean_scrub_stamp":"2024-09-17T07:31:50.940362+0000","objects_scrubbed":17,"log_size":890,"log_dups_size":0,"ondisk_log_size":890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:16.605351+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.001368193,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":6070,"num_read_kb":27721,"num_write":1113,"num_write_kb":14827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":15619889,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"188'737","reported_seq":4634,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:51.861606+0000","last_change":"2024-09-17T07:31:51.861606+0000","last_active":"2024-09-17T07:31:51.861606+0000","last_peered":"2024-09-17T07:31:51.861606+0000","last_clean":"2024-09-17T07:31:51.861606+0000","last_became_active":"2024-09-17T07:28:43.080816+0000","last_became_peered":"2024-09-17T07:28:43.080816+0000","last_unstale":"2024-09-17T07:31:51.861606+0000","last_undegraded":"2024-09-17T07:31:51.861606+0000","last_fullsized":"2024-09-17T07:31:51.861606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'737","last_scrub_stamp":"2024-09-17T07:31:51.861500+0000","last_deep_scrub":"188'737","last_deep_scrub_stamp":"2024-09-17T07:31:51.861500+0000","last_clean_scrub_stamp":"2024-09-17T07:31:51.861500+0000","objects_scrubbed":12,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:27:51.875123+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0017783269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6005,"num_read_kb":48957,"num_write":832,"num_write_kb":31224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"188'474","reported_seq":8451,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:52.827196+0000","last_change":"2024-09-17T07:31:52.827196+0000","last_active":"2024-09-17T07:31:52.827196+0000","last_peered":"2024-09-17T07:31:52.827196+0000","last_clean":"2024-09-17T07:31:52.827196+0000","last_became_active":"2024-09-17T07:31:26.154754+0000","last_became_peered":"2024-09-17T07:31:26.154754+0000","last_unstale":"2024-09-17T07:31:52.827196+0000","last_undegraded":"2024-09-17T07:31:52.827196+0000","last_fullsized":"2024-09-17T07:31:52.827196+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'474","last_scrub_stamp":"2024-09-17T07:31:52.827087+0000","last_deep_scrub":"188'474","last_deep_scrub_stamp":"2024-09-17T07:31:52.827087+0000","last_clean_scrub_stamp":"2024-09-17T07:31:52.827087+0000","objects_scrubbed":16,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:00.783250+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0021346680000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":7087,"num_read_kb":55690,"num_write":392,"num_write_kb":41376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":13797349,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"188'856","reported_seq":6372,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:53.823189+0000","last_change":"2024-09-17T07:31:53.823189+0000","last_active":"2024-09-17T07:31:53.823189+0000","last_peered":"2024-09-17T07:31:53.823189+0000","last_clean":"2024-09-17T07:31:53.823189+0000","last_became_active":"2024-09-17T07:31:28.101799+0000","last_became_peered":"2024-09-17T07:31:28.101799+0000","last_unstale":"2024-09-17T07:31:53.823189+0000","last_undegraded":"2024-09-17T07:31:53.823189+0000","last_fullsized":"2024-09-17T07:31:53.823189+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'856","last_scrub_stamp":"2024-09-17T07:31:53.823040+0000","last_deep_scrub":"188'856","last_deep_scrub_stamp":"2024-09-17T07:31:53.823040+0000","last_clean_scrub_stamp":"2024-09-17T07:31:53.823040+0000","objects_scrubbed":24,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:22:28.627554+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0021103049999999998,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":6418,"num_read_kb":23171,"num_write":923,"num_write_kb":18541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":744,"num_bytes_recovered":25525458,"num_keys_recovered":56,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"188'418","reported_seq":2793,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:49.849936+0000","last_change":"2024-09-17T07:31:49.849936+0000","last_active":"2024-09-17T07:31:49.849936+0000","last_peered":"2024-09-17T07:31:49.849936+0000","last_clean":"2024-09-17T07:31:49.849936+0000","last_became_active":"2024-09-17T07:28:43.077360+0000","last_became_peered":"2024-09-17T07:28:43.077360+0000","last_unstale":"2024-09-17T07:31:49.849936+0000","last_undegraded":"2024-09-17T07:31:49.849936+0000","last_fullsized":"2024-09-17T07:31:49.849936+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'418","last_scrub_stamp":"2024-09-17T07:31:49.849802+0000","last_deep_scrub":"188'418","last_deep_scrub_stamp":"2024-09-17T07:31:49.849802+0000","last_clean_scrub_stamp":"2024-09-17T07:31:49.849802+0000","objects_scrubbed":17,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:38:55.659050+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00061413699999999995,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2841,"num_read_kb":26198,"num_write":450,"num_write_kb":12235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"188'816","reported_seq":4361,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:55.780393+0000","last_change":"2024-09-17T07:31:55.780393+0000","last_active":"2024-09-17T07:31:55.780393+0000","last_peered":"2024-09-17T07:31:55.780393+0000","last_clean":"2024-09-17T07:31:55.780393+0000","last_became_active":"2024-09-17T07:31:29.103626+0000","last_became_peered":"2024-09-17T07:31:29.103626+0000","last_unstale":"2024-09-17T07:31:55.780393+0000","last_undegraded":"2024-09-17T07:31:55.780393+0000","last_fullsized":"2024-09-17T07:31:55.780393+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'816","last_scrub_stamp":"2024-09-17T07:31:55.780255+0000","last_deep_scrub":"188'816","last_deep_scrub_stamp":"2024-09-17T07:31:55.780255+0000","last_clean_scrub_stamp":"2024-09-17T07:31:55.780255+0000","objects_scrubbed":14,"log_size":816,"log_dups_size":0,"ondisk_log_size":816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T19:04:24.675571+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001604722,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":5310,"num_read_kb":45214,"num_write":937,"num_write_kb":26421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":468,"num_bytes_recovered":31676674,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"188'452","reported_seq":2772,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:48.066443+0000","last_change":"2024-09-17T07:31:48.066443+0000","last_active":"2024-09-17T07:31:48.066443+0000","last_peered":"2024-09-17T07:31:48.066443+0000","last_clean":"2024-09-17T07:31:48.066443+0000","last_became_active":"2024-09-17T07:31:08.096240+0000","last_became_peered":"2024-09-17T07:31:08.096240+0000","last_unstale":"2024-09-17T07:31:48.066443+0000","last_undegraded":"2024-09-17T07:31:48.066443+0000","last_fullsized":"2024-09-17T07:31:48.066443+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'452","last_scrub_stamp":"2024-09-17T07:31:48.066292+0000","last_deep_scrub":"188'452","last_deep_scrub_stamp":"2024-09-17T07:31:48.066292+0000","last_clean_scrub_stamp":"2024-09-17T07:31:48.066292+0000","objects_scrubbed":17,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:45.076165+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00098890999999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":2231,"num_read_kb":44562,"num_write":412,"num_write_kb":29148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":24460350,"num_keys_recovered":37,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"188'786","reported_seq":4264,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-17T07:31:54.771074+0000","last_change":"2024-09-17T07:31:54.771074+0000","last_active":"2024-09-17T07:31:54.771074+0000","last_peered":"2024-09-17T07:31:54.771074+0000","last_clean":"2024-09-17T07:31:54.771074+0000","last_became_active":"2024-09-17T07:31:29.103168+0000","last_became_peered":"2024-09-17T07:31:29.103168+0000","last_unstale":"2024-09-17T07:31:54.771074+0000","last_undegraded":"2024-09-17T07:31:54.771074+0000","last_fullsized":"2024-09-17T07:31:54.771074+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"188'786","last_scrub_stamp":"2024-09-17T07:31:54.770975+0000","last_deep_scrub":"188'786","last_deep_scrub_stamp":"2024-09-17T07:31:54.770975+0000","last_clean_scrub_stamp":"2024-09-17T07:31:54.770975+0000","objects_scrubbed":14,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:12.741034+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00093421499999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":4511,"num_read_kb":28124,"num_write":1074,"num_write_kb":16615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":21539718,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":611,"num_object_clones":0,"num_object_copies":1222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":611,"num_whiteouts":607,"num_read":166048,"num_read_kb":1208202,"num_write":23179,"num_write_kb":792760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9827,"num_bytes_recovered":666374238,"num_keys_recovered":891,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7902,"internal_metadata":0},"log_size":21164,"ondisk_log_size":21164,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":101032,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47332,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324508,"statfs":{"total":96636764160,"available":96588296192,"internally_reserved":0,"allocated":2314240,"data_stored":1685451,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542192,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33396,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338444,"statfs":{"total":96636764160,"available":96602566656,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55608,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316232,"statfs":{"total":96636764160,"available":96579821568,"internally_reserved":0,"allocated":2269184,"data_stored":1095102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91640,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":89406,"kb_avail":94280200,"statfs":{"total":96636764160,"available":96542924800,"internally_reserved":0,"allocated":2269184,"data_stored":1095102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":91552197},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":90,"seq":386547056668,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47412,"kb_used_data":2212,"kb_used_omap":16,"kb_used_meta":45167,"kb_avail":94324428,"statfs":{"total":96636764160,"available":96588214272,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17058,"internal_metadata":46251358},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542192,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55604,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316236,"statfs":{"total":96636764160,"available":96579825664,"internally_reserved":0,"allocated":2265088,"data_stored":1095083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65340,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306500,"statfs":{"total":96636764160,"available":96569856000,"internally_reserved":0,"allocated":2273280,"data_stored":1095121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542192,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47348,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324492,"statfs":{"total":96636764160,"available":96588279808,"internally_reserved":0,"allocated":2330624,"data_stored":1685451,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32598,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34217,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7902,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T07:32:09.031 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T07:32:09.350 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T07:32:09.350 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T07:32:09.350 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T07:32:09.350 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T07:32:09.388 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T07:32:09.389 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T07:32:09.389 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T07:32:09.469 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T07:32:09.469 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T07:32:09.469 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T07:32:09.540 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T07:32:09.540 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T07:32:09.540 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T07:32:09.620 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T07:32:09.620 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T07:32:09.620 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T07:32:09.660 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T07:32:09.661 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T07:32:09.661 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T07:32:09.743 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T07:32:09.744 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-17T07:32:09.744 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T07:32:09.792 INFO:tasks.ceph.osd.6:Stopped 2024-09-17T07:32:09.792 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-17T07:32:09.792 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T07:32:09.838 INFO:tasks.ceph.osd.7:Stopped 2024-09-17T07:32:09.839 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T07:32:09.839 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-17T07:32:09.839 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T07:32:09.866 INFO:tasks.ceph.mgr.y:Stopped 2024-09-17T07:32:09.866 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T07:32:09.866 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T07:32:09.892 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T07:32:09.893 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T07:32:09.893 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T07:32:09.893 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T07:32:09.917 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T07:32:09.918 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T07:32:09.918 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T07:32:09.969 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T07:32:09.969 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T07:32:09.970 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T07:32:09.983 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T07:32:09.983 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T07:32:09.983 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-17T07:32:10.049 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-17T07:32:10.050 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T07:32:10.246 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-17T07:32:10.246 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T07:32:10.454 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-17T07:32:10.454 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T07:32:10.513 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi045.front.sepia.ceph.com 2024-09-17T07:32:10.513 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T07:32:10.587 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-17T07:32:10.588 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T07:32:10.837 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-17T07:32:10.837 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T07:32:10.913 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-17T07:32:10.914 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-17T07:32:10.955 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-17T07:32:10.955 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-17T07:32:11.047 INFO:tasks.ceph:Archiving mon data... 2024-09-17T07:32:11.048 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/dis-2024-09-17_07:04:29-rbd-wip-dis-testing-distro-default-smithi/7908707/data/mon.a.tgz 2024-09-17T07:32:11.048 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-17T07:32:11.066 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.wXMPvhAMhF 2024-09-17T07:32:11.066 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.wXMPvhAMhF 2024-09-17T07:32:11.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.wXMPvhAMhF 2024-09-17T07:32:11.323 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.wXMPvhAMhF is 433KB 2024-09-17T07:32:11.422 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.wXMPvhAMhF 2024-09-17T07:32:11.437 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/dis-2024-09-17_07:04:29-rbd-wip-dis-testing-distro-default-smithi/7908707/data/mon.c.tgz 2024-09-17T07:32:11.438 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-17T07:32:11.492 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.a5hX8nag0n 2024-09-17T07:32:11.492 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.a5hX8nag0n 2024-09-17T07:32:11.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.a5hX8nag0n 2024-09-17T07:32:11.765 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.a5hX8nag0n is 583KB 2024-09-17T07:32:11.865 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.a5hX8nag0n 2024-09-17T07:32:11.881 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/dis-2024-09-17_07:04:29-rbd-wip-dis-testing-distro-default-smithi/7908707/data/mon.b.tgz 2024-09-17T07:32:11.882 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-09-17T07:32:11.901 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.tcwskmZzF1 2024-09-17T07:32:11.901 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.tcwskmZzF1 2024-09-17T07:32:12.095 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.tcwskmZzF1 2024-09-17T07:32:12.178 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.tcwskmZzF1 is 583KB 2024-09-17T07:32:12.248 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.tcwskmZzF1 2024-09-17T07:32:12.263 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T07:32:12.264 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T07:32:12.266 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T07:32:12.347 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T07:32:12.384 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T07:32:12.384 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T07:32:12.385 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2024-09-17_07:04:29-rbd-wip-dis-testing-distro-default-smithi/7908707/remote/smithi045/crash 2024-09-17T07:32:12.386 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T07:32:12.414 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2024-09-17_07:04:29-rbd-wip-dis-testing-distro-default-smithi/7908707/remote/smithi192/crash 2024-09-17T07:32:12.415 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T07:32:12.448 INFO:tasks.ceph:Compressing logs... 2024-09-17T07:32:12.448 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T07:32:12.459 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T07:32:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T07:32:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59142.log 2024-09-17T07:32:12.483 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48620.log 2024-09-17T07:32:12.529 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32970.log 2024-09-17T07:32:12.529 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-17T07:32:12.529 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.32970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33106.log 2024-09-17T07:32:12.529 INFO:teuthology.orchestra.run.smithi192.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32970.log.gz 2024-09-17T07:32:12.530 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33186.log 2024-09-17T07:32:12.530 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T07:32:12.530 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.33106.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33106.log.gz 2024-09-17T07:32:12.530 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.33186.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T07:32:12.530 INFO:teuthology.orchestra.run.smithi192.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33186.log.gz 2024-09-17T07:32:12.530 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33010.log 2024-09-17T07:32:12.530 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33226.log 2024-09-17T07:32:12.530 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.33010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33146.log 2024-09-17T07:32:12.530 INFO:teuthology.orchestra.run.smithi192.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33010.log.gz 2024-09-17T07:32:12.530 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.33226.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.33226.log.gz 2024-09-17T07:32:12.530 INFO:teuthology.orchestra.run.smithi192.stderr: --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-17T07:32:12.530 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-17T07:32:12.530 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.33146.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33146.log.gz 2024-09-17T07:32:12.531 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T07:32:12.531 INFO:teuthology.orchestra.run.smithi192.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T07:32:12.531 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T07:32:12.531 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33266.log 2024-09-17T07:32:12.531 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T07:32:12.531 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.33266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33266.log.gz 2024-09-17T07:32:12.531 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33066.log 2024-09-17T07:32:12.531 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.33066.log: 85.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T07:32:12.531 INFO:teuthology.orchestra.run.smithi192.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33066.log.gz 2024-09-17T07:32:12.531 INFO:teuthology.orchestra.run.smithi192.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T07:32:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.59142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39506.log 2024-09-17T07:32:12.597 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59142.log.gz 2024-09-17T07:32:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48620.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48620.log.gz -5 2024-09-17T07:32:12.597 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45568.log 2024-09-17T07:32:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38923.log 2024-09-17T07:32:12.597 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39506.log.gz 2024-09-17T07:32:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.45568.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58170.log 2024-09-17T07:32:12.597 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45568.log.gz 2024-09-17T07:32:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60191.log 2024-09-17T07:32:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38923.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38923.log.gz 2024-09-17T07:32:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58170.log.gz 2024-09-17T07:32:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56377.log 2024-09-17T07:32:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60317.log 2024-09-17T07:32:12.598 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60191.log.gz 2024-09-17T07:32:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45528.log 2024-09-17T07:32:12.598 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56377.log.gz 2024-09-17T07:32:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37125.log 2024-09-17T07:32:12.598 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60317.log.gz 2024-09-17T07:32:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45528.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45528.log.gz 2024-09-17T07:32:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61325.log 2024-09-17T07:32:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45326.log 2024-09-17T07:32:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37125.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37125.log.gz 2024-09-17T07:32:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62333.log 2024-09-17T07:32:12.599 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61325.log.gz 2024-09-17T07:32:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61465.log 2024-09-17T07:32:12.599 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45326.log.gz 2024-09-17T07:32:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41228.log 2024-09-17T07:32:12.599 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62333.log.gz 2024-09-17T07:32:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55059.log 2024-09-17T07:32:12.599 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61465.log.gz 2024-09-17T07:32:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62249.log 2024-09-17T07:32:12.599 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41228.log.gz 2024-09-17T07:32:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45211.log 2024-09-17T07:32:12.599 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55059.log.gz 2024-09-17T07:32:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47497.log 2024-09-17T07:32:12.599 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62249.log.gz 2024-09-17T07:32:12.600 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51017.log 2024-09-17T07:32:12.600 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45211.log.gz 2024-09-17T07:32:12.600 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47497.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47497.log.gz 2024-09-17T07:32:12.600 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59580.log 2024-09-17T07:32:12.600 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59100.log 2024-09-17T07:32:12.600 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51017.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51017.log.gz 2024-09-17T07:32:12.600 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53714.log 2024-09-17T07:32:12.600 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59580.log.gz 2024-09-17T07:32:12.600 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38416.log 2024-09-17T07:32:12.600 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59100.log.gz 2024-09-17T07:32:12.600 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45885.log 2024-09-17T07:32:12.600 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53714.log.gz 2024-09-17T07:32:12.600 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37243.log 2024-09-17T07:32:12.601 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38416.log.gz 2024-09-17T07:32:12.601 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61464.log 2024-09-17T07:32:12.601 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45885.log.gz 2024-09-17T07:32:12.601 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38881.log 2024-09-17T07:32:12.601 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37243.log.gz 2024-09-17T07:32:12.601 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46060.log 2024-09-17T07:32:12.601 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61464.log.gz 2024-09-17T07:32:12.601 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60149.log 2024-09-17T07:32:12.601 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38881.log.gz 2024-09-17T07:32:12.601 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53917.log 2024-09-17T07:32:12.601 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46060.log.gz 2024-09-17T07:32:12.601 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60149.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T07:32:12.601 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60149.log.gz 2024-09-17T07:32:12.601 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57967.log 2024-09-17T07:32:12.602 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53917.log.gz 2024-09-17T07:32:12.602 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37838.log 2024-09-17T07:32:12.602 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50027.log 2024-09-17T07:32:12.602 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57967.log.gz 2024-09-17T07:32:12.602 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37838.log.gz 2024-09-17T07:32:12.602 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45488.log 2024-09-17T07:32:12.602 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60275.log 2024-09-17T07:32:12.602 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50027.log.gz 2024-09-17T07:32:12.602 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45488.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.56116.log 2024-09-17T07:32:12.602 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45488.log.gz 2024-09-17T07:32:12.602 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39166.log 2024-09-17T07:32:12.602 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60275.log.gz 2024-09-17T07:32:12.602 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60443.log 2024-09-17T07:32:12.602 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56116.log.gz 2024-09-17T07:32:12.603 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56646.log 2024-09-17T07:32:12.603 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39166.log.gz 2024-09-17T07:32:12.603 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60443.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.60443.log.gz 2024-09-17T07:32:12.603 INFO:teuthology.orchestra.run.smithi045.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38081.log 2024-09-17T07:32:12.603 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38797.log 2024-09-17T07:32:12.603 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56646.log.gz 2024-09-17T07:32:12.603 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38081.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38081.log.gz 2024-09-17T07:32:12.603 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61661.log 2024-09-17T07:32:12.603 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37505.log 2024-09-17T07:32:12.603 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38797.log.gz 2024-09-17T07:32:12.603 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61661.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61661.log.gz 2024-09-17T07:32:12.603 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59981.log 2024-09-17T07:32:12.603 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37505.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.42468.log 2024-09-17T07:32:12.603 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37505.log.gz 2024-09-17T07:32:12.604 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38474.log 2024-09-17T07:32:12.604 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59981.log.gz 2024-09-17T07:32:12.604 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/objectstore_tool.42468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44325.log 2024-09-17T07:32:12.604 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38474.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38474.log.gz -5 2024-09-17T07:32:12.604 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58600.log 2024-09-17T07:32:12.604 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46317.log 2024-09-17T07:32:12.604 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44325.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44325.log.gz 2024-09-17T07:32:12.604 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52458.log 2024-09-17T07:32:12.604 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58600.log.gz 2024-09-17T07:32:12.604 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37201.log 2024-09-17T07:32:12.604 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46317.log.gz 2024-09-17T07:32:12.604 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52458.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.52458.log.gz -- 2024-09-17T07:32:12.605 INFO:teuthology.orchestra.run.smithi045.stderr: /var/log/ceph/ceph-client.admin.38755.log 2024-09-17T07:32:12.605 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61129.log 2024-09-17T07:32:12.605 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37201.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37201.log.gz 2024-09-17T07:32:12.605 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38755.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.38755.log.gz 2024-09-17T07:32:12.605 INFO:teuthology.orchestra.run.smithi045.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48977.log 2024-09-17T07:32:12.605 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.41486.log/var/log/ceph/ceph-client.admin.61129.log: 2024-09-17T07:32:12.605 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61129.log.gz 2024-09-17T07:32:12.605 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48977.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.62291.log -- replaced with /var/log/ceph/ceph-client.admin.48977.log.gz 2024-09-17T07:32:12.605 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-17T07:32:12.605 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57157.log 2024-09-17T07:32:12.605 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/objectstore_tool.41486.log: /var/log/ceph/ceph-client.admin.62291.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.62291.log.gz -5 2024-09-17T07:32:12.605 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40722.log 2024-09-17T07:32:12.605 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53774.log 2024-09-17T07:32:12.605 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57157.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57157.log.gz 2024-09-17T07:32:12.606 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44955.log 2024-09-17T07:32:12.606 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40722.log.gz 2024-09-17T07:32:12.606 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.53774.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.53216.log 2024-09-17T07:32:12.606 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53774.log.gz 2024-09-17T07:32:12.606 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62459.log 2024-09-17T07:32:12.606 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44955.log.gz 2024-09-17T07:32:12.606 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53014.log 2024-09-17T07:32:12.606 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53216.log.gz 2024-09-17T07:32:12.606 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55938.log 2024-09-17T07:32:12.606 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62459.log.gz 2024-09-17T07:32:12.606 INFO:teuthology.orchestra.run.smithi045.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.42468.log.gz 2024-09-17T07:32:12.606 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44914.log 2024-09-17T07:32:12.606 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53014.log.gz 2024-09-17T07:32:12.606 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58372.log 2024-09-17T07:32:12.607 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55938.log.gz 2024-09-17T07:32:12.607 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52357.log 2024-09-17T07:32:12.607 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44914.log.gz 2024-09-17T07:32:12.607 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58372.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58372.log.gzgzip 2024-09-17T07:32:12.607 INFO:teuthology.orchestra.run.smithi045.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52180.log 2024-09-17T07:32:12.607 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60793.log 2024-09-17T07:32:12.607 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52357.log.gz 2024-09-17T07:32:12.607 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47123.log 0.0% 2024-09-17T07:32:12.607 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52180.log.gz 2024-09-17T07:32:12.607 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50839.log 2024-09-17T07:32:12.607 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60793.log.gz 2024-09-17T07:32:12.607 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47123.log.gz 2024-09-17T07:32:12.607 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43984.log 2024-09-17T07:32:12.608 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52707.log 2024-09-17T07:32:12.608 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50839.log.gz 2024-09-17T07:32:12.608 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43984.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.43984.log.gz -- 2024-09-17T07:32:12.608 INFO:teuthology.orchestra.run.smithi045.stderr: /var/log/ceph/ceph-client.admin.38180.log 2024-09-17T07:32:12.608 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62081.log 2024-09-17T07:32:12.608 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52707.log.gz 2024-09-17T07:32:12.608 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38180.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38180.log.gz -5 2024-09-17T07:32:12.608 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50299.log 2024-09-17T07:32:12.608 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38377.log 2024-09-17T07:32:12.608 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62081.log.gz 2024-09-17T07:32:12.608 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40503.log 2024-09-17T07:32:12.608 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50299.log.gz 2024-09-17T07:32:12.608 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38377.log: gzip -5 --verbose 94.9% 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.38377.log.gz -- replaced with /var/log/ceph/objectstore_tool.41486.log.gz 2024-09-17T07:32:12.609 INFO:teuthology.orchestra.run.smithi045.stderr: /var/log/ceph/ceph-client.admin.60789.log 2024-09-17T07:32:12.609 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-17T07:32:12.609 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45251.log 2024-09-17T07:32:12.609 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40503.log.gz 2024-09-17T07:32:12.609 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43100.log 2024-09-17T07:32:12.609 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60789.log.gz 2024-09-17T07:32:12.609 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39027.log 2024-09-17T07:32:12.609 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45251.log.gz 2024-09-17T07:32:12.609 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47165.log 2024-09-17T07:32:12.609 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43100.log.gz 2024-09-17T07:32:12.609 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62123.log 2024-09-17T07:32:12.609 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39027.log.gz 2024-09-17T07:32:12.609 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37844.log 2024-09-17T07:32:12.609 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47165.log.gz 2024-09-17T07:32:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59162.log 2024-09-17T07:32:12.610 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62123.log.gz 2024-09-17T07:32:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37844.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37844.log.gz 2024-09-17T07:32:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47719.log 2024-09-17T07:32:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46569.log 2024-09-17T07:32:12.610 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59162.log.gz 2024-09-17T07:32:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47719.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47719.log.gz 2024-09-17T07:32:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51700.log 2024-09-17T07:32:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48014.log 2024-09-17T07:32:12.610 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46569.log.gz 2024-09-17T07:32:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59622.log 2024-09-17T07:32:12.610 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51700.log.gz 2024-09-17T07:32:12.610 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48014.log.gzgzip 2024-09-17T07:32:12.610 INFO:teuthology.orchestra.run.smithi045.stderr: -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T07:32:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37836.log 2024-09-17T07:32:12.611 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59622.log.gz 2024-09-17T07:32:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59850.log 2024-09-17T07:32:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37836.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37836.log.gzgzip 2024-09-17T07:32:12.611 INFO:teuthology.orchestra.run.smithi045.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51101.log 2024-09-17T07:32:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38080.log 2024-09-17T07:32:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59850.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59850.log.gz 2024-09-17T07:32:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51101.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51101.log.gz 2024-09-17T07:32:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61118.log 2024-09-17T07:32:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55160.log 2024-09-17T07:32:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38080.log.gz 2024-09-17T07:32:12.611 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44061.log 2024-09-17T07:32:12.612 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61118.log.gz 2024-09-17T07:32:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.55160.log: /var/log/ceph/ceph-client.admin.38839.log 2024-09-17T07:32:12.612 INFO:teuthology.orchestra.run.smithi045.stderr: 93.7% 0.0% -- replaced with /var/log/ceph/ceph.audit.log.gz -- replaced with /var/log/ceph/ceph-client.admin.55160.log.gz 2024-09-17T07:32:12.612 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-17T07:32:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38713.log 2024-09-17T07:32:12.612 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44061.log.gz 2024-09-17T07:32:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38839.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.38839.log.gz /var/log/ceph/ceph-client.admin.54694.log 2024-09-17T07:32:12.612 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-17T07:32:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48055.log 2024-09-17T07:32:12.612 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38713.log.gz 2024-09-17T07:32:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43541.log 2024-09-17T07:32:12.612 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54694.log.gz 2024-09-17T07:32:12.612 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37497.log 2024-09-17T07:32:12.613 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48055.log.gz 2024-09-17T07:32:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43541.log.gz 2024-09-17T07:32:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61997.log 2024-09-17T07:32:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59766.log 2024-09-17T07:32:12.613 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37497.log.gz 2024-09-17T07:32:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57424.log 2024-09-17T07:32:12.613 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61997.log.gz 2024-09-17T07:32:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58777.log 2024-09-17T07:32:12.613 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59766.log.gz 2024-09-17T07:32:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57424.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57424.log.gz 2024-09-17T07:32:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45407.log 2024-09-17T07:32:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44637.log 2024-09-17T07:32:12.613 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58777.log.gz 2024-09-17T07:32:12.613 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56703.log 2024-09-17T07:32:12.614 INFO:teuthology.orchestra.run.smithi045.stderr: 26.4% -- replaced with /var/log/ceph/ceph-client.admin.45407.log.gz 2024-09-17T07:32:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50298.log 2024-09-17T07:32:12.614 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44637.log.gz 2024-09-17T07:32:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53395.log 2024-09-17T07:32:12.614 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56703.log.gz 2024-09-17T07:32:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61122.log 2024-09-17T07:32:12.614 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50298.log.gz 2024-09-17T07:32:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51401.log 2024-09-17T07:32:12.614 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53395.log.gz 2024-09-17T07:32:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51757.log 2024-09-17T07:32:12.614 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61122.log.gz 2024-09-17T07:32:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57083.log 2024-09-17T07:32:12.614 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51401.log.gz 2024-09-17T07:32:12.614 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46494.log 2024-09-17T07:32:12.615 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51757.log.gz 2024-09-17T07:32:12.615 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57083.log.gz 2024-09-17T07:32:12.615 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39161.log 2024-09-17T07:32:12.615 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59808.log 2024-09-17T07:32:12.615 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46494.log.gz 2024-09-17T07:32:12.615 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39161.log.gz 2024-09-17T07:32:12.615 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53998.log 2024-09-17T07:32:12.615 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60790.log 2024-09-17T07:32:12.615 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59808.log.gz 2024-09-17T07:32:12.615 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53998.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53998.log.gz 2024-09-17T07:32:12.615 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60359.log 2024-09-17T07:32:12.615 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56214.log 2024-09-17T07:32:12.615 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60790.log.gz 2024-09-17T07:32:12.615 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47955.log 2024-09-17T07:32:12.616 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60359.log.gz 2024-09-17T07:32:12.616 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62627.log 2024-09-17T07:32:12.616 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56214.log.gz 2024-09-17T07:32:12.616 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53833.log 2024-09-17T07:32:12.616 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47955.log.gz 2024-09-17T07:32:12.616 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53436.log 2024-09-17T07:32:12.616 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62627.log.gz 2024-09-17T07:32:12.616 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60792.log 2024-09-17T07:32:12.616 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53833.log.gz 2024-09-17T07:32:12.616 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53436.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27226.log 2024-09-17T07:32:12.616 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53436.log.gz 2024-09-17T07:32:12.616 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54836.log 2024-09-17T07:32:12.616 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60792.log.gz 2024-09-17T07:32:12.616 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph.tmp-client.admin.27226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60485.log 2024-09-17T07:32:12.617 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27226.log.gz 2024-09-17T07:32:12.617 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61913.log 2024-09-17T07:32:12.617 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54836.log.gz 2024-09-17T07:32:12.617 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62417.log 2024-09-17T07:32:12.617 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60485.log.gz 2024-09-17T07:32:12.617 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61913.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.61913.log.gz --verbose 2024-09-17T07:32:12.617 INFO:teuthology.orchestra.run.smithi045.stderr: -- /var/log/ceph/ceph-mon.c.log 2024-09-17T07:32:12.617 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38417.log 2024-09-17T07:32:12.617 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62417.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62417.log.gz 2024-09-17T07:32:12.617 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47854.log 2024-09-17T07:32:12.617 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38417.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38417.log.gz 2024-09-17T07:32:12.617 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33364.log 2024-09-17T07:32:12.617 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46846.log 2024-09-17T07:32:12.617 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47854.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47854.log.gz 2024-09-17T07:32:12.618 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33457.log 2024-09-17T07:32:12.618 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33364.log.gz 2024-09-17T07:32:12.618 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55243.log 2024-09-17T07:32:12.618 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46846.log.gz 2024-09-17T07:32:12.618 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62543.log 2024-09-17T07:32:12.618 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33457.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33457.log.gz 2024-09-17T07:32:12.618 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55243.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.60401.log 2024-09-17T07:32:12.618 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55243.log.gz 2024-09-17T07:32:12.618 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62543.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62543.log.gz 2024-09-17T07:32:12.618 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57966.log 2024-09-17T07:32:12.618 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38179.log 2024-09-17T07:32:12.618 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60401.log.gz 2024-09-17T07:32:12.618 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57966.log.gz 2024-09-17T07:32:12.619 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54736.log 2024-09-17T07:32:12.619 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39566.log 2024-09-17T07:32:12.619 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38179.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38179.log.gz 2024-09-17T07:32:12.619 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54736.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54736.log.gz 2024-09-17T07:32:12.619 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38079.log 2024-09-17T07:32:12.619 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51880.log 2024-09-17T07:32:12.619 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39566.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39566.log.gz 2024-09-17T07:32:12.619 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60791.log 2024-09-17T07:32:12.619 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38079.log.gz 2024-09-17T07:32:12.619 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51880.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.45191.log 2024-09-17T07:32:12.619 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51880.log.gz 2024-09-17T07:32:12.619 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51181.log 2024-09-17T07:32:12.619 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60791.log.gz 2024-09-17T07:32:12.619 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38181.log 2024-09-17T07:32:12.620 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45191.log.gz 2024-09-17T07:32:12.620 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57708.log 2024-09-17T07:32:12.620 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51181.log.gz 2024-09-17T07:32:12.620 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.38181.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37705.log 2024-09-17T07:32:12.620 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38181.log.gz 2024-09-17T07:32:12.620 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57708.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57708.log.gz 2024-09-17T07:32:12.620 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60233.log 2024-09-17T07:32:12.620 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45367.log 2024-09-17T07:32:12.620 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37705.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37705.log.gz 2024-09-17T07:32:12.620 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43943.log 2024-09-17T07:32:12.620 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60233.log.gz 2024-09-17T07:32:12.620 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44305.log 2024-09-17T07:32:12.620 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45367.log.gz 2024-09-17T07:32:12.620 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43943.log.gz 2024-09-17T07:32:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37841.log 2024-09-17T07:32:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57606.log 2024-09-17T07:32:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44305.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44305.log.gz 2024-09-17T07:32:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37841.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37841.log.gz -5 2024-09-17T07:32:12.621 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52957.log 2024-09-17T07:32:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44739.log 2024-09-17T07:32:12.621 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57606.log.gz 2024-09-17T07:32:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37069.log 2024-09-17T07:32:12.621 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52957.log.gz 2024-09-17T07:32:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44739.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T07:32:12.621 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44739.log.gz 2024-09-17T07:32:12.621 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37069.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.37069.log.gz 2024-09-17T07:32:12.621 INFO:teuthology.orchestra.run.smithi045.stderr: /var/log/ceph/ceph-client.admin.37501.log 2024-09-17T07:32:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37327.log 2024-09-17T07:32:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.37501.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37501.log.gz 2024-09-17T07:32:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59442.log 2024-09-17T07:32:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61128.log 2024-09-17T07:32:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37327.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37327.log.gz 2024-09-17T07:32:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59442.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.59442.log.gz -- 2024-09-17T07:32:12.622 INFO:teuthology.orchestra.run.smithi045.stderr: /var/log/ceph/ceph-client.admin.48660.log 2024-09-17T07:32:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59897.log 2024-09-17T07:32:12.622 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61128.log.gz 2024-09-17T07:32:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47739.log 2024-09-17T07:32:12.622 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48660.log.gz 2024-09-17T07:32:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59058.log 2024-09-17T07:32:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59897.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59897.log.gz 2024-09-17T07:32:12.622 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47739.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47739.log.gz 2024-09-17T07:32:12.622 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37500.log 2024-09-17T07:32:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47813.log 2024-09-17T07:32:12.623 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59058.log.gz 2024-09-17T07:32:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62039.log 2024-09-17T07:32:12.623 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37500.log.gz 2024-09-17T07:32:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58535.log 2024-09-17T07:32:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47813.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47813.log.gz/var/log/ceph/ceph-client.admin.62039.log: 2024-09-17T07:32:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62039.log.gz 2024-09-17T07:32:12.623 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58738.log 2024-09-17T07:32:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58535.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58535.log.gz 2024-09-17T07:32:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58454.log 2024-09-17T07:32:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43486.log 2024-09-17T07:32:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58738.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58738.log.gz 2024-09-17T07:32:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56604.log 2024-09-17T07:32:12.624 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58454.log.gz 2024-09-17T07:32:12.624 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43486.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43486.log.gz 2024-09-17T07:32:12.624 INFO:teuthology.orchestra.run.smithi045.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61955.log 2024-09-17T07:32:12.624 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62207.log 2024-09-17T07:32:12.624 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56604.log.gz 2024-09-17T07:32:12.624 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61955.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61955.log.gz 2024-09-17T07:32:12.624 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61787.log 2024-09-17T07:32:12.624 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49439.log 2024-09-17T07:32:12.624 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62207.log.gz 2024-09-17T07:32:12.624 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60023.log 2024-09-17T07:32:12.624 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61787.log.gz 2024-09-17T07:32:12.624 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49439.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T07:32:12.624 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49439.log.gz 2024-09-17T07:32:12.624 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55461.log 2024-09-17T07:32:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60023.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60023.log.gz 2024-09-17T07:32:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58111.log 2024-09-17T07:32:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55461.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55461.log.gz 2024-09-17T07:32:12.625 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38538.log 2024-09-17T07:32:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58111.log.gz 2024-09-17T07:32:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37150.log 2024-09-17T07:32:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38538.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38538.log.gz 2024-09-17T07:32:12.625 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61459.log 2024-09-17T07:32:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37150.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37150.log.gz 2024-09-17T07:32:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60527.log 2024-09-17T07:32:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61459.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61459.log.gz 2024-09-17T07:32:12.625 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54977.log 2024-09-17T07:32:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60527.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60527.log.gz 2024-09-17T07:32:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33062.log 2024-09-17T07:32:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54977.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54977.log.gz 2024-09-17T07:32:12.626 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38965.log 2024-09-17T07:32:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.33062.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33062.log.gz 2024-09-17T07:32:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43858.log 2024-09-17T07:32:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36969.log 2024-09-17T07:32:12.626 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38965.log.gz 2024-09-17T07:32:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38539.log 2024-09-17T07:32:12.626 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43858.log.gz 2024-09-17T07:32:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46357.log 2024-09-17T07:32:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.36969.log: /var/log/ceph/ceph-client.admin.38539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61462.log 2024-09-17T07:32:12.626 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38539.log.gz 2024-09-17T07:32:12.626 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36969.log.gz 2024-09-17T07:32:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46357.log.gz 2024-09-17T07:32:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54156.log 2024-09-17T07:32:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61462.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61462.log.gz 2024-09-17T07:32:12.627 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41959.log 2024-09-17T07:32:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54156.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54156.log.gz 2024-09-17T07:32:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37509.log 2024-09-17T07:32:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41959.log: gzip -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41959.log.gz 2024-09-17T07:32:12.627 INFO:teuthology.orchestra.run.smithi045.stderr: /var/log/ceph/ceph-client.admin.55681.log 2024-09-17T07:32:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37509.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37509.log.gz 2024-09-17T07:32:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-17T07:32:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43900.log 2024-09-17T07:32:12.627 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55681.log.gz 2024-09-17T07:32:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60569.log 2024-09-17T07:32:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50896.log 2024-09-17T07:32:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43900.log: /var/log/ceph/ceph-client.admin.60569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43900.log.gz 2024-09-17T07:32:12.628 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60569.log.gz 2024-09-17T07:32:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61123.log 2024-09-17T07:32:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46161.log 2024-09-17T07:32:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50896.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50896.log.gz 2024-09-17T07:32:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61123.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61101.log 2024-09-17T07:32:12.628 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-17T07:32:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46161.log.gz 2024-09-17T07:32:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52118.log 2024-09-17T07:32:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62585.log 2024-09-17T07:32:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61101.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61101.log.gz 2024-09-17T07:32:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52118.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52118.log.gz 2024-09-17T07:32:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61745.log 2024-09-17T07:32:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62585.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62585.log.gz 2024-09-17T07:32:12.628 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37837.log 2024-09-17T07:32:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T07:32:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61745.log.gz 2024-09-17T07:32:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37837.log.gz 2024-09-17T07:32:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44143.log 2024-09-17T07:32:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56294.log 2024-09-17T07:32:12.629 INFO:teuthology.orchestra.run.smithi045.stderr: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T07:32:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44143.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62709.log 2024-09-17T07:32:12.629 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-17T07:32:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56294.log.gz 2024-09-17T07:32:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38477.log 2024-09-17T07:32:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59642.log 2024-09-17T07:32:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62709.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62709.log.gz 2024-09-17T07:32:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38477.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38477.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40284.log 2024-09-17T07:32:12.629 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-17T07:32:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59642.log.gz 2024-09-17T07:32:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38484.log 2024-09-17T07:32:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.40974.log 2024-09-17T07:32:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40284.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40284.log.gz 2024-09-17T07:32:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38484.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.57767.log 2024-09-17T07:32:12.630 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38484.log.gz 2024-09-17T07:32:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/objectstore_tool.40974.log: 94.7% -- replaced with /var/log/ceph/objectstore_tool.40974.log.gz 2024-09-17T07:32:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56460.log 2024-09-17T07:32:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57767.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58677.log 2024-09-17T07:32:12.630 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57767.log.gz 2024-09-17T07:32:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37508.log 2024-09-17T07:32:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56460.log: /var/log/ceph/ceph-client.admin.58677.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56460.log.gz 2024-09-17T07:32:12.630 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58677.log.gz 2024-09-17T07:32:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60778.log 2024-09-17T07:32:12.631 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37508.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37508.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39160.log 2024-09-17T07:32:12.631 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-17T07:32:12.631 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58071.log 2024-09-17T07:32:12.631 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39160.log: /var/log/ceph/ceph-client.admin.60778.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60778.log.gz 2024-09-17T07:32:12.631 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39160.log.gz 2024-09-17T07:32:12.631 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62375.log 2024-09-17T07:32:12.631 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58071.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60611.log 2024-09-17T07:32:12.631 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58071.log.gz 2024-09-17T07:32:12.631 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61126.log 2024-09-17T07:32:12.631 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62375.log: /var/log/ceph/ceph-client.admin.60611.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62375.log.gz 2024-09-17T07:32:12.631 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60611.log.gz 2024-09-17T07:32:12.631 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61703.log 2024-09-17T07:32:12.631 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37507.log 2024-09-17T07:32:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61126.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61126.log.gz 2024-09-17T07:32:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61703.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61703.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50616.log 2024-09-17T07:32:12.632 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-17T07:32:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40264.log 2024-09-17T07:32:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37507.log: /var/log/ceph/ceph-client.admin.50616.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37507.log.gz 2024-09-17T07:32:12.632 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50616.log.gz 2024-09-17T07:32:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44542.log 2024-09-17T07:32:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54556.log 2024-09-17T07:32:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40264.log.gz 2024-09-17T07:32:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44542.log.gz 2024-09-17T07:32:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47934.log 2024-09-17T07:32:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40640.log 2024-09-17T07:32:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47934.log: /var/log/ceph/ceph-client.admin.54556.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54556.log.gz 2024-09-17T07:32:12.633 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47934.log.gz 2024-09-17T07:32:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57541.log 2024-09-17T07:32:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37506.log 2024-09-17T07:32:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40640.log.gz 2024-09-17T07:32:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57541.log.gz 2024-09-17T07:32:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39966.log 2024-09-17T07:32:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38178.log 2024-09-17T07:32:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39966.log: /var/log/ceph/ceph-client.admin.37506.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37506.log.gz 2024-09-17T07:32:12.633 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39966.log.gz 2024-09-17T07:32:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58312.log 2024-09-17T07:32:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43818.log 2024-09-17T07:32:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38178.log.gz 2024-09-17T07:32:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58312.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58312.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52576.log 2024-09-17T07:32:12.633 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-17T07:32:12.634 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38175.log 2024-09-17T07:32:12.634 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43818.log.gz 2024-09-17T07:32:12.634 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52576.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54415.log 2024-09-17T07:32:12.634 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52576.log.gz 2024-09-17T07:32:12.634 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.38175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45448.log 2024-09-17T07:32:12.634 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38175.log.gz 2024-09-17T07:32:12.634 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60787.log 2024-09-17T07:32:12.634 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45448.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54415.log.gz 2024-09-17T07:32:12.634 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45448.log.gz 2024-09-17T07:32:12.634 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39080.log 2024-09-17T07:32:12.634 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48836.log 2024-09-17T07:32:12.634 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60787.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60787.log.gz 2024-09-17T07:32:12.634 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.39080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50499.log 2024-09-17T07:32:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48836.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48836.log.gz 2024-09-17T07:32:12.635 INFO:teuthology.orchestra.run.smithi045.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.0.39080.log.gz 2024-09-17T07:32:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56561.log 2024-09-17T07:32:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60065.log 2024-09-17T07:32:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50499.log.gz 2024-09-17T07:32:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56561.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56561.log.gz 2024-09-17T07:32:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60786.log 2024-09-17T07:32:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59939.log 2024-09-17T07:32:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60786.log: /var/log/ceph/ceph-client.admin.60065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60065.log.gz 2024-09-17T07:32:12.635 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60786.log.gz 2024-09-17T07:32:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61460.log 2024-09-17T07:32:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48331.log 2024-09-17T07:32:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59939.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59939.log.gz 2024-09-17T07:32:12.635 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61460.log.gz 2024-09-17T07:32:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40859.log 2024-09-17T07:32:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61121.log 2024-09-17T07:32:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40859.log: /var/log/ceph/ceph-client.admin.48331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48331.log.gz 2024-09-17T07:32:12.636 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40859.log.gz 2024-09-17T07:32:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55639.log 2024-09-17T07:32:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53115.log 2024-09-17T07:32:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61121.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61121.log.gz 2024-09-17T07:32:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55639.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37285.log 2024-09-17T07:32:12.636 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55639.log.gz 2024-09-17T07:32:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62501.log 2024-09-17T07:32:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53115.log: /var/log/ceph/ceph-client.admin.37285.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53115.log.gz 2024-09-17T07:32:12.636 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37285.log.gz 2024-09-17T07:32:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54757.log 2024-09-17T07:32:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61461.log 2024-09-17T07:32:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62501.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62501.log.gz 2024-09-17T07:32:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54757.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54757.log.gz 2024-09-17T07:32:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48116.log 2024-09-17T07:32:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56019.log 2024-09-17T07:32:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48116.log: /var/log/ceph/ceph-client.admin.61461.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61461.log.gz 2024-09-17T07:32:12.637 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48116.log.gz 2024-09-17T07:32:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47894.log 2024-09-17T07:32:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48372.log 2024-09-17T07:32:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56019.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56019.log.gz 2024-09-17T07:32:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47894.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47894.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50143.log 2024-09-17T07:32:12.637 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-17T07:32:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61871.log 2024-09-17T07:32:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48372.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48372.log.gz 2024-09-17T07:32:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39747.log 2024-09-17T07:32:12.638 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50143.log.gz 2024-09-17T07:32:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47442.log 2024-09-17T07:32:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61871.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61871.log.gz 2024-09-17T07:32:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60107.log 2024-09-17T07:32:12.638 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39747.log.gz 2024-09-17T07:32:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54298.log 2024-09-17T07:32:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47442.log.gz 2024-09-17T07:32:12.638 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60107.log.gz 2024-09-17T07:32:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57042.log 2024-09-17T07:32:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51799.log 2024-09-17T07:32:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54298.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54298.log.gz 2024-09-17T07:32:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57042.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51361.log 2024-09-17T07:32:12.638 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57042.log.gz 2024-09-17T07:32:12.639 INFO:teuthology.orchestra.run.smithi192.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T07:32:12.650 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45803.log 2024-09-17T07:32:12.650 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51799.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51799.log.gz 2024-09-17T07:32:12.650 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47206.log 2024-09-17T07:32:12.650 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51361.log.gz 2024-09-17T07:32:12.650 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50936.log 2024-09-17T07:32:12.650 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47206.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45803.log.gz 2024-09-17T07:32:12.650 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47206.log.gz 2024-09-17T07:32:12.650 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39925.log 2024-09-17T07:32:12.651 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62165.log 2024-09-17T07:32:12.651 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50936.log.gz 2024-09-17T07:32:12.651 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.39925.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39925.log.gz 2024-09-17T07:32:12.651 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57323.log 2024-09-17T07:32:12.651 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56865.log 2024-09-17T07:32:12.651 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57323.log: /var/log/ceph/ceph-client.admin.62165.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62165.log.gz 2024-09-17T07:32:12.651 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57323.log.gz 2024-09-17T07:32:12.651 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46988.log 2024-09-17T07:32:12.651 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52638.log 2024-09-17T07:32:12.651 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56865.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56865.log.gz 2024-09-17T07:32:12.651 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46988.log.gz 2024-09-17T07:32:12.651 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43703.log 2024-09-17T07:32:12.651 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.41274.log 2024-09-17T07:32:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43703.log: /var/log/ceph/ceph-client.admin.52638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52638.log.gz 2024-09-17T07:32:12.652 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43703.log.gz 2024-09-17T07:32:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46791.log 2024-09-17T07:32:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T07:32:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/objectstore_tool.41274.log: /var/log/ceph/ceph-client.admin.46791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46791.log.gz 2024-09-17T07:32:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61829.log 2024-09-17T07:32:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.3.log: 94.4% -- replaced with /var/log/ceph/objectstore_tool.41274.log.gz 2024-09-17T07:32:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62669.log 2024-09-17T07:32:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61829.log.gz 2024-09-17T07:32:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61463.log 2024-09-17T07:32:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62669.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62669.log.gz 2024-09-17T07:32:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55300.log 2024-09-17T07:32:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61463.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61463.log.gz 2024-09-17T07:32:12.652 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59241.log 2024-09-17T07:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55300.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55300.log.gz 2024-09-17T07:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37845.log 2024-09-17T07:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59241.log.gz 2024-09-17T07:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57542.log 2024-09-17T07:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37845.log.gz 2024-09-17T07:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53875.log 2024-09-17T07:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57542.log.gz 2024-09-17T07:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45687.log 2024-09-17T07:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53875.log.gz 2024-09-17T07:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37842.log 2024-09-17T07:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45687.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45687.log.gz 2024-09-17T07:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.37842.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37842.log.gz 2024-09-17T07:32:12.653 INFO:teuthology.orchestra.run.smithi045.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-17T07:32:12.714 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T07:32:12.865 INFO:teuthology.orchestra.run.smithi045.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T07:32:13.288 INFO:teuthology.orchestra.run.smithi045.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T07:32:13.616 INFO:teuthology.orchestra.run.smithi045.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T07:32:14.318 INFO:teuthology.orchestra.run.smithi192.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-17T07:32:15.667 INFO:teuthology.orchestra.run.smithi045.stderr: 94.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T07:32:15.827 INFO:teuthology.orchestra.run.smithi192.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T07:32:15.865 INFO:teuthology.orchestra.run.smithi045.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T07:32:15.866 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-17T07:32:15.866 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m3.394s 2024-09-17T07:32:15.866 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m8.704s 2024-09-17T07:32:15.866 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m0.479s 2024-09-17T07:32:15.916 INFO:teuthology.orchestra.run.smithi192.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T07:32:16.590 INFO:teuthology.orchestra.run.smithi192.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-17T07:32:16.592 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-09-17T07:32:16.592 INFO:teuthology.orchestra.run.smithi192.stderr:real 0m4.085s 2024-09-17T07:32:16.592 INFO:teuthology.orchestra.run.smithi192.stderr:user 0m12.194s 2024-09-17T07:32:16.592 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m0.515s 2024-09-17T07:32:16.592 INFO:tasks.ceph:Archiving logs... 2024-09-17T07:32:16.592 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/dis-2024-09-17_07:04:29-rbd-wip-dis-testing-distro-default-smithi/7908707/remote/smithi045/log 2024-09-17T07:32:16.594 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T07:32:17.970 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/dis-2024-09-17_07:04:29-rbd-wip-dis-testing-distro-default-smithi/7908707/remote/smithi192/log 2024-09-17T07:32:17.971 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T07:32:19.656 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T07:32:19.666 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T07:32:19.667 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T07:32:19.669 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T07:32:19.751 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T07:32:19.759 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T07:32:19.760 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T07:32:19.762 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T07:32:19.778 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-09-17T07:32:19.780 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-09-17T07:32:19.782 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T07:32:19.783 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-09-17T07:32:19.783 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv01.front.sepia.ceph.com 3 6 377 39 +948us[ +471us] +/- 47ms 2024-09-17T07:32:19.783 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 -3869us[-4346us] +/- 55ms 2024-09-17T07:32:19.783 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +3221us[+3221us] +/- 72ms 2024-09-17T07:32:19.783 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T07:32:19.784 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T07:32:19.784 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-09-17T07:32:19.784 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv01.front.sepia.ceph.com 3 6 377 41 +946us[ +461us] +/- 47ms 2024-09-17T07:32:19.784 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 -3857us[-3857us] +/- 55ms 2024-09-17T07:32:19.785 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +3242us[+3242us] +/- 72ms 2024-09-17T07:32:19.785 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T07:32:19.785 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T07:32:19.794 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T07:32:19.794 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T07:32:19.802 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T07:32:19.912 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T07:32:19.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T07:32:20.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T07:32:20.023 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T07:32:20.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T07:32:20.054 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T07:32:20.065 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T07:32:20.075 INFO:teuthology.task.internal:Duration was 663.852230 seconds 2024-09-17T07:32:20.075 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T07:32:20.083 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T07:32:20.083 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T07:32:20.086 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T07:32:20.127 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T07:32:20.134 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T07:32:20.467 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T07:32:20.467 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-09-17T07:32:20.467 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T07:32:20.490 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-09-17T07:32:20.490 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T07:32:20.523 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T07:32:20.524 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T07:32:20.533 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T07:32:20.591 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T07:32:20.591 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T07:32:20.604 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T07:32:20.749 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T07:32:20.760 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T07:32:20.760 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T07:32:20.787 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T07:32:20.812 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T07:32:20.823 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T07:32:20.830 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T07:32:20.852 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-09-17T07:32:20.875 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-09-17T07:32:20.927 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T07:32:20.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T07:32:20.942 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T07:32:20.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T07:32:20.982 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T07:32:20.992 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T07:32:20.993 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2024-09-17_07:04:29-rbd-wip-dis-testing-distro-default-smithi/7908707/remote/smithi045 2024-09-17T07:32:20.993 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T07:32:21.038 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2024-09-17_07:04:29-rbd-wip-dis-testing-distro-default-smithi/7908707/remote/smithi192 2024-09-17T07:32:21.038 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T07:32:21.073 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T07:32:21.073 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T07:32:21.076 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T07:32:21.123 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T07:32:21.135 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T07:32:21.135 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T07:32:21.144 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T07:32:21.144 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T07:32:21.147 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T07:32:21.160 INFO:teuthology.orchestra.run.smithi045.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 07:32 /home/ubuntu/cephtest 2024-09-17T07:32:21.178 INFO:teuthology.orchestra.run.smithi192.stdout: 265638 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 07:32 /home/ubuntu/cephtest 2024-09-17T07:32:21.179 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T07:32:21.191 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T07:32:21.227 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writethrough} duration: 663.8522300720215 flavor: default owner: scheduled_dis@teuthology success: true 2024-09-17T07:32:21.227 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T07:32:21.312 INFO:teuthology.run:pass